Do you consider using MLX instead of whisper.cpp?

[quakeboy]

I still use this on a daily basis & wanted to ask / make a suggestion to try MLX version of whisper, which has been much more efficient and faster in terms of both startup and processing. Of course it would make it Apple Silicon only though
I used it for my telegram transcription bot, so just wanted to let you know.

[asmit203]

"mlx-community/whisper-large-v3-mlx" are you talking about this?

[quakeboy]

@asmit203

"mlx-community/whisper-large-v3-mlx" are you talking about this?

not particularly just that, but similar family - I use a MLX variant of 4bit quantised small whisper model which is a great balance of speed and accuracy for my telegram voice transcription bot.

[asmit203]

as per my understanding of this project goes this is a wrapper on top of whisper cpp which is installed on laptop to run this one on top of it for the mlx variant i think it will complicate the working as that will need a server like whisper.cpp to run it and serve it and other optimization needed for the server to be efficient

[human37]

Thanks for the suggestion. The blocker for MLX directly is that mlx-whisper is Python, so adopting it would mean adding a Python runtime on top of the current single-binary whisper-cpp dep, which is a big install-complexity jump.

A couple of notes: whisper.cpp on macOS already uses Metal acceleration in the Homebrew bottle, so on the small models used for dictation the gap to MLX is smaller than the headline benchmarks suggest. If I do go Apple Silicon-native later, WhisperKit (pure Swift, CoreML, Neural Engine, no Python) is probably the better fit than MLX-Whisper since it preserves the one-binary install.

Leaving this open as a tracking issue for that direction.