[audio.cpp] CUDA-optimized C++ implementation of Nemotron-3.5-ASR-Streaming-0.6B. 400.0x realtime on RTX 5090

[0xShug0]

Hi NeMo team,

Thank you for releasing so many great models.

I'd like to share my C++/GGML based implmemantion of Nemotron-3.5-ASR-Streaming-0.6B in audio.cpp. Audio.cpp is an all-in-one, pure C++ inference engine for audio models, powered by ggml. Supports TTS, STT, VAD, voice conversion, music generation, and more, with highly optimized performance.

Would you consider adding the audio.cpp GGUF checkpoint to the “Featured Community Checkpoints” section? https://huggingface.co/audio-cpp/audio.cpp-gguf/tree/main/Nemotron-3.5-ASR-Streaming-0.6B-GGUF

Real-world ASR win: In TranscrIA benchmark on messy French meeting audio, audio.cpp’s Nemotron 3.5 ASR matched the same WER as other implementations while using about 1/4 of the wall time.

Tests on RTX 5090

ASR Offline Long-Lived Session (multiple requets)

Model	16-bit speed vs real time	Q8 speed vs real time	Q8 vs 16-bit	16-bit peak VRAM	Q8 peak VRAM
nemotron_asr	277.8x-384.6x	285.7x-400.0x	1.03x-1.12x	5125 MiB	4028 MiB

ASR Streaming Long Audio (30 mins)

Model	16-bit server TTFT	Q8 server TTFT	16-bit client TTFT	Q8 client TTFT	16-bit speed vs real time	Q8 speed vs real time	16-bit peak VRAM	Q8 peak VRAM
nemotron_asr	205.007 ms	214.822 ms	488.629 ms	499.453 ms	31.6x	33.4x	2816 MiB	2497 MiB