Nemotron 3.5 ASR Streaming 0.6B in native C++/GGML. RTF = 0.0066 on RTX 5090

[0xShug0]

Hi everyone,

I’d like to share audio.cpp support for Nemotron 3.5 ASR 0.6B in native C++/GGML:

https://github.com/0xShug0/audio.cpp

On my RTX 5090, the offline path transcribes a 327.6s audio file in 2.17s:

Some perf numbers on RTX 5090

Model	Mode	Dur.	TTFT	Wall	RTF	WER	VRAM
Nemotron	Offline	327.6s	N/A	2.17s	0.0066	3.18%	8294M
Nemotron	SSE	327.6s	308ms	11.62s	N/A	3.18%	4382M
Nemotron	SSE 1-shot	1800s	N/A	53.66s	0.0298	3.45%	4167M

The goal of audio.cpp is to make local audio models easier to run and integrate without needing a separate Python environment for every model.

Huge thanks to the Nemotron team for releasing the model and making this possible. Feedback, testing, and suggestions are very welcome.