Repository navigation
Add Muse Spark Open GB300 SGLang DFlash2 AgentX recipe - #4141
Conversation
Single-node SGLang TP4/EP4 on GB300 serving the staged NVFP4 checkpoint, with DFlash2 speculation and golden synthetic acceptance. The connector maps SGLang DFLASH to the dflash curve with num_speculative_tokens equal to speculative-num-draft-tokens minus one. 中文：在 GB300 上新增 Muse Spark Open 单节点 SGLang TP4/EP4 AgentX 配方，使用已预置的 NVFP4 权重、DFlash2 投机解码与黄金合成接受率；连接器将 SGLang DFLASH 映射为 dflash 曲线，num_speculative_tokens 为 speculative-num-draft-tokens 减一。
中文：将 perf-changelog 条目关联到 #4141。
|
Thanks for the contribution!
中文感谢你的贡献！
|
cquil11
added
full-sweep-enabled
中文：恢复 perf-changelog 历史条目的 pr-link。
cquil11
added
the
full-sweep-enabled
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
I reviewed this PR and didn't find any bugs. The NVIDIA master config and GB300 recipes have designated code owners, and the change sets single-node-models: staged for the whole gb300-nv cluster, so it still needs a look from an owner.
What was reviewed:
- The DFlash branch in
spec_parameters:speculative-num-draft-tokens: 8gives 7 drafts.curve_namemaps this tomusespark_dflash, and the committed curve gives AL 4.05 for 7 drafts, which is within the 1..8 bound. The new test case uses 4 draft tokens (3 drafts) against the fixture's 3.4. - The staged-model switch on
gb300-nv: the only other single-node GB300 configs serve DeepSeek-V4.1-Flash and Qwen3.8-Flash-Next. Neither has amodels.entriesrow, so they still resolve tohf:<MODEL>. - The recipe, master-config entry and changelog entry agree on TP4/EP4,
draft_model, and the concurrency list from 1 to 128. TheXXXpr-link placeholder follows the existing pattern in the changelog.
Extended reasoning...
The change adds one single-node SGLang DFlash recipe for Muse Spark Open on GB300 and a matching nvidia-master entry. It also adds a GB300 staged-model entry, flips single-node-models to staged for the cluster, and adds a small DFlash branch with a test in the golden-acceptance connector. It touches no security-sensitive surface such as auth, injection or secrets. The bug hunt ran until it stopped finding anything (dry_streak) and reported nothing. I deferred instead of approving for two reasons: CODEOWNERS assigns nvidia-master.yaml and the gb300-* recipe directories to specific NVIDIA owners, with SemiAnalysisAI/core covering everything else; and the single-node-models flip applies to the whole cluster, not just this one config.
This review covers commit dca7e02, which is no longer the latest commit on this pull request; later commits are not covered by it.
…urrency sweep Shrink the sliding-window pool (swa-full-tokens-ratio 0.1) and raise mem-fraction-static to 0.85, which lifts the full-attention KV pool from 2.1M to 5.5M tokens. Sweep concurrency 1-32 where throughput still rises. 中文：缩小滑动窗口 KV 池（swa-full-tokens-ratio 0.1）并将 mem-fraction-static 提高到 0.85，使全注意力 KV 池从 2.1M 提升到 5.5M token；并发扫描改为吞吐仍在上升的 1-32 区间。
|
View unofficial run (performance): https://inferencex.semianalysis.com/inference?unofficialRun=38835086328 View unofficial run (accuracy): https://inferencex.semianalysis.com/evaluation?unofficialRun=38835086328 |
cquil11
added
non-canary-full-sweep-enabled
|
[by Claude Code] The SGLang cookbook integration of this PR's AgentX config ( This PR has no authorized |
|
/use 38835086328 |
|
@functionstackx staged run 38835086328. This run remains available across future |
|
[by Claude Code] sgl-project/sglang#44654 has been merged. This PR's AgentX config ( This PR has no authorized |
3 participants
Adds a Muse Spark Open (
meta-models/Muse-Spark-Open-816B-A42B-NVFP4) AgentX config on GB300:musespark-fp4-gb300-sglang-agentic-dflash.lmsysorg/sglang:dev-muse-spark-open(pinned by digest), following the SGLang cookbook's verified GB300 NVFP4 cell: TP4/EP4,modelopt_fp4,flashinfer_trtllmMoE, fp8 KV, layernorm SP with MegaMoE, DFlash2 draftRadixArk/Muse-Spark-Open-DFlash2with an 8-token verification block and fa4 draft attention,musereasoning and tool-call parsers. KV stays on the GPU.swa-full-tokens-ratio0.8 the SWA pool sat about 4% used while full-attention KV saturated from concurrency 24, so the recipe setsswa-full-tokens-ratio0.1 andmem-fraction-static0.85 (full-attention pool 2.1M to 5.5M tokens).max-running-requestsequals concurrency).DFLASHto themusespark_dflashcurve withnum_speculative_tokens = speculative-num-draft-tokens - 1(same change as [Klaud Cold] Add Muse Spark Open MXFP4 MI355X SGLang DFlash2 AgentX recipe / [Klaud Cold] 新增 Muse Spark Open MXFP4 MI355X SGLang DFlash2 AgentX 配方 #4127), so this recipe gets AL 4.05 (7 drafts, thinking_on) injected; nothing is hard-coded./data/models/Muse-Spark-Open-816B-A42B-NVFP4togb300-nvmodels.entriesand setssingle-node-models: staged, so single-node jobs serve the staged weights instead of the gated Hub repo. No other GB300 single-node config has amodels.entriesrow, so their resolution is unchanged. The AgentX client tokenizes from the mounted checkpoint (TOKENIZER: /model).Validation
Run Sweep 38835086328 on
d3850c4: all 12 AgentX throughput jobs and the agentic GSM8K eval succeeded (GSM8K strict-match 0.974, 0 request errors). Mean server accept length was 4.05 against the 4.05 golden target. The throughput-vs-median-E2EL frontier has 10 points (c1 to c20).