Repository navigation
fix(vllm): drop reasoning-parser none on the Python sidecar launcher - #602
Merged
peilii merged 2 commits intoOct 13, 2026
Merged
Conversation
Dynamo's vLLM sidecar requires vLLM to run without its own reasoning parser when the sidecar's --dyn-*-parser flags are set (ai-dynamo/dynamo#15684), and the documented way is `vllm-rs serve --reasoning-parser none`. srtctl launches single-node sidecar endpoints with vllm-rs, which parses `none` itself, but hybrid-DP and cross-node replicas go through Python `vllm serve`, whose engine args treat every --reasoning-parser value as a parser-registry name (its disabled value is omission). The same recipe value therefore started prefill and killed every decode node with: KeyError: "Reasoning parser 'none' not found. Available parsers: ..." Drop `reasoning-parser: none` (case-insensitive) on the Python launcher only. Other selections, including `auto`, are still forwarded unchanged, since silently dropping them would change the selection the Python-bootstrapped Rust frontend makes. Validation: seen on a 5P1D DeepSeek-V4.1-Flash deployment (prefill DEP2 via vllm-rs, decode DEP16 over four nodes via the hybrid launcher) with the vllm/vllm-openai nightly 2026-10-12 image and ai-dynamo 1.6.0.dev20261010; the two new tests cover both launchers. Signed-off-by: Pei Li <peili@nvidia.com>
peilii
requested review from
alec-flowers,
csahithi,
ishandhanani and
nlevin-ui
as code owners
October 13, 2026 05:04
|
|
Self-review follow-up. Dropping `none` is semantics-preserving because the Rust frontend that Python `vllm serve` bootstraps defaults to no parser (vllm-rs `default_py_bootstrap_parser_selection`). `auto` has no equivalent on that launcher: Python has no auto selection and the bootstrapped frontend never sees the flag, so forwarding it only moves the failure into the same KeyError. Reject it with a clear error instead; explicit parser names still pass through. Adds the test and tightens the comment and docs sentence. Signed-off-by: Pei Li <peili@nvidia.com>
richardhuo-nv
approved these changes
Oct 13, 2026
Juntian777
added a commit
to SemiAnalysisAI/InferenceX
that referenced
this pull request
Oct 13, 2026
Picks up NVIDIA/srt-slurm#602: on the Python sidecar launcher (multi-node hybrid-DP decode) srtctl drops `reasoning-parser: none`, which Python vLLM rejects as an unknown parser name, while single-node `vllm-rs serve` endpoints keep it as Dynamo sidecars require. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Dynamo's vLLM sidecar refuses to start when vLLM runs its own reasoning parser alongside the sidecar's
--dyn-*-parserflags (ai-dynamo/dynamo#15684), and Dynamo's launch scripts start vLLM withvllm-rs serve --reasoning-parser none. In srtctl that value only works for single-node sidecar endpoints, which launchvllm-rs serve. Hybrid-DP endpoints and cross-node replicas launch through Pythonvllm serve(VLLM_USE_RUST_FRONTEND=1), and Python's engine args look every--reasoning-parservalue up in the parser registry; its disabled value is the absent flag. With one recipe value, prefill (DEP2, vllm-rs) came up and every decode node (DEP16 over four nodes, hybrid) died at engine-config time:This change drops
reasoning-parser: none(case-insensitive) on the Python launcher only. That is semantics-preserving: the Rust frontend Python bootstraps defaults to no parser (default_py_bootstrap_parser_selectionreturnsParserSelection::Nonein vllm-rs), so the result equalsvllm-rs serve --reasoning-parser none.autohas no equivalent on this launcher (Python has no auto selection and the bootstrapped frontend never sees the flag), so it is rejected with a clearValueErrorrather than forwarded into the same KeyError. Explicit parser names pass through unchanged.Validation
pytest tests/test_sidecar_backends.py: 67 passed, including three new tests (hybrid DP dropsnone/Noneand forwards a parser name; hybrid DP rejectsauto; a single-nodevllm-rs serveendpoint keeps--reasoning-parser none).ruff check/ruff format --checkclean.vllm/vllm-openai:nightly-aarch64-dev-20261012-59e75f4(vLLM0.31.1rc1.dev296+gfe4df3df0) and ai-dynamo1.6.0.dev20261010: the failing 5P1D job (--reasoning-parser noneon all roles) vs. the same recipe without the flag on decode, which reached ready with all nine sidecars serving. With this patch andreasoning-parser: noneon every role, the same 5P1D deployment launched prefill asvllm-rs serve … --reasoning-parser noneand decode through the Python launcher without the flag, hit no KeyError, and reached ready with all 9 sidecars serving (5 prefill, 4 decode) before the benchmark started.Notes
docs/sidecars.mdgets one sentence in the multi-node DP paragraph.vllm-rs servegains hybrid-DP startup, decode can leave the Python launcher and this translation becomes unnecessary.