docs: STT whisper->parakeet evaluation and resident swap#3
Merged
Conversation
Replace the deployed STT engine (whisper.cpp ggml-small) with parakeet.cpp tdt_ctc-110m on :8178. On the Jetson Orin Nano Super, parakeet wins on every axis: WER 0.82% vs 1.02%, RTFx 116x vs 19x, per-call latency ~170ms vs ~500ms, peak RAM +410MB vs +726MB, model 170MB vs 466MB. Live LyraT captures transcribe correctly. Adds the comparison report (docs/stt-parakeet-evaluation.md) and reproducible deploy artifacts (deploy/parakeet-stt/): a resident `serve` patch for parakeet.cpp, an HTTP shim matching whisper-server's POST /inference, and a systemd unit. genie-core is unchanged; the swap is reversible. Addresses #2.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Replaces the deployed STT engine (whisper.cpp
ggml-small) with parakeet.cpptdt_ctc-110mon the Jetson Orin Nano Super, and documents the head-to-head that motivated it. Addresses the STT-accuracy open item in #2.Comparison (50 clean LibriSpeech clips, identical on-device conditions)
ggml-small(original)tdt_ctc-110m-q8(current)parakeet is more accurate, ~6× faster compute, ~3× lower per-call latency, and ~300 MB leaner — and it's the small parakeet model. Three live ESP32-LyraT mic captures transcribed correctly.
What's in the PR
docs/stt-parakeet-evaluation.md— full report: method, comparison, real-capture check, caveats, build notes, optional streaming.deploy/parakeet-stt/— reproducible artifacts:serve-mode.patch— adds a residentservesubcommand to parakeet.cpp (load model once; no per-call reload).parakeet-stt-server.py— HTTP shim with the samePOST /inference→{"text": ...}contract as whisper-server, on:8178.genie-parakeet.service— systemd unit (resident,Restart=always, on boot).README.md— build / install / revert.Integration & reversibility
genie-core is unchanged (it still posts to
:8178; parakeet now answers there). Swap =systemctl disable --now genie-whisper && systemctl enable --now genie-parakeet; revert is the inverse.Caveats
Clean LibriSpeech subset (both score low) — noisy-24 kHz real-capture WER is not yet quantified at scale (only the single live capture).
tdt_ctc-110mis the lean model;tdt-0.6b-v2is more accurate but heavier (swap via the service'sPK_MODEL).q8_0quant, device-specific.