Add VibeVoice-ASR (Microsoft) — OpenAI-compatible audio transcription

[Alfarouk-Baseten]

Summary

Adds vibevoice-asr/ — a passthrough Truss for microsoft/VibeVoice-ASR running on vllm/vllm-openai:v0.14.1.

• Native OpenAI chat completions API — any OpenAI-compatible client works
• Returns transcripts as JSON-stringified segments with speaker labels and timestamps
• Single H100, ~3-min cold start

Files

vibevoice-asr/
├── config.yaml      (101 lines — docker_server passthrough)
├── data/
│   └── patch.py     (112 lines — applies 3 plugin patches at boot)
└── README.md        (74 lines)

About data/patch.py

Microsoft's vllm_plugin/model.py has three known incompatibilities with current vLLM releases:

1. KV-cache delegation forwarders on VibeVoiceForCausalLM — required for vLLM 0.14.1; without it, startup crashes with IndexError: list index out of range on available_gpu_memory[0]
2. get_data_parser on VibeVoiceProcessingInfo — forward-compat for vLLM 0.21+
3. mm_data_items rename try/except — forward-compat for vLLM 0.15+

patch.py applies these in-place at container boot with assert-checked anchor strings. The script is idempotent and fails loudly at startup if Microsoft pushes an upstream change that breaks the anchors (rather than silent miscompile).

When Microsoft fixes the plugin upstream, delete data/patch.py and remove the python3 /app/data/patch.py line from config.yaml.

Known limitations (documented in README)

• WAV / FLAC audio only — m4a fails to decode in vLLM 0.14.1's audio loader (returns 400)
• Size max_tokens proportional to audio length (~20 tok/sec + buffer) — model doesn't reliably emit EOS so over-large max_tokens produces a repetition loop
• Single audio clip per request (no batching of multiple clips in one chat)

[CLAassistant]

All committers have signed the CLA.

[Alfarouk-Baseten]

Heads-up on the failing lint check

The lint (pre-commit) check is failing, but not because of changes in this PR. The pre-commit hook runs ruff with --all-files, which touches a pre-existing issue on main that was introduced before this PR.

Where the failure is

triton-inference-server/tensorrtllm-backend/data/model_repository/tensorrt_llm/1/model.py, line 563:

raise pb_utils.TritonModelException(
    f"text_input is missing from the request")

ruff (rule F541: f-string-without-placeholders) wants to strip the unused f prefix:

raise pb_utils.TritonModelException(
    "text_input is missing from the request")

Why this affects our PR

The repo's CI runs pre-commit run --all-files, so every PR's lint check will hit this and fail until main is cleaned up.

Verified

Running ruff check --fix --fix-only --diff on only vibevoice-asr/data/patch.py against ruff v0.11.6 (the version pinned in .pre-commit-config.yaml) reports no changes needed. The file under triton-inference-server/... is the sole source of the failure.

Suggested resolution

Either:

1. Fix main first with a one-line PR (f"text_input..." → "text_input...")
2. Or merge this PR despite the red lint check, since the lint failure isn't from this PR's contents

[Alfarouk-Baseten]

Closing in favor of basetenlabs/model-registry#163, which is the canonical path for adding a Baseten Model Library entry. The model-registry sync will publish this Truss to truss-examples automatically on merge to main. No reviewer action needed here.