Skip to content

fix(vllm): drop reasoning-parser none on the Python sidecar launcher - #602

Merged
peilii merged 2 commits into
mainfrom
peili/vllm-sidecar-hybrid-drop-reasoning-parser-none
Oct 13, 2026
Merged

peilii merged 2 commits into
mainfrom
peili/vllm-sidecar-hybrid-drop-reasoning-parser-none

Conversation

@peilii

@peilii peilii commented Oct 13, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Dynamo's vLLM sidecar refuses to start when vLLM runs its own reasoning parser alongside the sidecar's --dyn-*-parser flags (ai-dynamo/dynamo#15684), and Dynamo's launch scripts start vLLM with vllm-rs serve --reasoning-parser none. In srtctl that value only works for single-node sidecar endpoints, which launch vllm-rs serve. Hybrid-DP endpoints and cross-node replicas launch through Python vllm serve (VLLM_USE_RUST_FRONTEND=1), and Python's engine args look every --reasoning-parser value up in the parser registry; its disabled value is the absent flag. With one recipe value, prefill (DEP2, vllm-rs) came up and every decode node (DEP16 over four nodes, hybrid) died at engine-config time:

KeyError: "Reasoning parser 'none' not found. Available parsers: cohere_command3, ..., deepseek_v41, ..."

This change drops reasoning-parser: none (case-insensitive) on the Python launcher only. That is semantics-preserving: the Rust frontend Python bootstraps defaults to no parser (default_py_bootstrap_parser_selection returns ParserSelection::None in vllm-rs), so the result equals vllm-rs serve --reasoning-parser none. auto has no equivalent on this launcher (Python has no auto selection and the bootstrapped frontend never sees the flag), so it is rejected with a clear ValueError rather than forwarded into the same KeyError. Explicit parser names pass through unchanged.

Validation

  • pytest tests/test_sidecar_backends.py: 67 passed, including three new tests (hybrid DP drops none/None and forwards a parser name; hybrid DP rejects auto; a single-node vllm-rs serve endpoint keeps --reasoning-parser none).
  • ruff check / ruff format --check clean.
  • Observed on oci-jhb with vllm/vllm-openai:nightly-aarch64-dev-20261012-59e75f4 (vLLM 0.31.1rc1.dev296+gfe4df3df0) and ai-dynamo 1.6.0.dev20261010: the failing 5P1D job (--reasoning-parser none on all roles) vs. the same recipe without the flag on decode, which reached ready with all nine sidecars serving. With this patch and reasoning-parser: none on every role, the same 5P1D deployment launched prefill as vllm-rs serve … --reasoning-parser none and decode through the Python launcher without the flag, hit no KeyError, and reached ready with all 9 sidecars serving (5 prefill, 4 decode) before the benchmark started.

Notes

  • docs/sidecars.md gets one sentence in the multi-node DP paragraph.
  • Longer term, if vllm-rs serve gains hybrid-DP startup, decode can leave the Python launcher and this translation becomes unnecessary.

Dynamo's vLLM sidecar requires vLLM to run without its own reasoning parser
when the sidecar's --dyn-*-parser flags are set (ai-dynamo/dynamo#15684), and
the documented way is `vllm-rs serve --reasoning-parser none`. srtctl launches
single-node sidecar endpoints with vllm-rs, which parses `none` itself, but
hybrid-DP and cross-node replicas go through Python `vllm serve`, whose engine
args treat every --reasoning-parser value as a parser-registry name (its
disabled value is omission). The same recipe value therefore started prefill
and killed every decode node with:

  KeyError: "Reasoning parser 'none' not found. Available parsers: ..."

Drop `reasoning-parser: none` (case-insensitive) on the Python launcher only.
Other selections, including `auto`, are still forwarded unchanged, since
silently dropping them would change the selection the Python-bootstrapped
Rust frontend makes.

Validation: seen on a 5P1D DeepSeek-V4.1-Flash deployment (prefill DEP2 via
vllm-rs, decode DEP16 over four nodes via the hybrid launcher) with the
vllm/vllm-openai nightly 2026-10-12 image and ai-dynamo 1.6.0.dev20261010;
the two new tests cover both launchers.

Signed-off-by: Pei Li <peili@nvidia.com>
@github-actions

github-actions Bot commented Oct 13, 2026 •

Copy link
Copy Markdown

src/srtctl diff: +15 -0

Self-review follow-up. Dropping `none` is semantics-preserving because the
Rust frontend that Python `vllm serve` bootstraps defaults to no parser
(vllm-rs `default_py_bootstrap_parser_selection`). `auto` has no equivalent
on that launcher: Python has no auto selection and the bootstrapped frontend
never sees the flag, so forwarding it only moves the failure into the same
KeyError. Reject it with a clear error instead; explicit parser names still
pass through. Adds the test and tightens the comment and docs sentence.

Signed-off-by: Pei Li <peili@nvidia.com>
@peilii
peilii merged commit fe957cc into main Oct 13, 2026
12 checks passed
Juntian777 added a commit to SemiAnalysisAI/InferenceX that referenced this pull request Oct 13, 2026
Picks up NVIDIA/srt-slurm#602: on the Python sidecar launcher (multi-node hybrid-DP decode) srtctl drops
`reasoning-parser: none`, which Python vLLM rejects as an unknown parser name, while single-node `vllm-rs serve`
endpoints keep it as Dynamo sidecars require.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants