[AgentX] GB300 DSV4.1 Flash SGLang: 20261012 nightly, 7-point curve with DEP2 and TP4 EP1 / [AgentX] GB300 DSV4.1 Flash SGLang：升级 20261012 nightly，精简为 7 个点（DEP2、TP4 EP1）

[weireweire]

Description

Config: dsv41flash-fp4-gb300-sglang-agentic-dspark (aggregated, single node). Recipe, master config and changelog only.

1. Image bump to lmsysorg/sglang:nightly-dev-cu13-20261012-e6072935, pinned by digest. This nightly:
   • makes the Rust radix-tree core the default ([Radix Cache] Sync Rust TreeCore and make it the default sgl-project/sglang#39627), removing a Python insert walk that delayed the last prefill chunk by 30–70 ms on 200K–400K-token AgentX prompts;
   • supports DP attention with the MegaMoE all-to-all backend ([1/N][DSv41] Enable DP attention with MegaMoE sgl-project/sglang#43228).
2. Seven points instead of sixteen: TP4 c1/c4/c16, TP2 c16/c32 and DEP2 c64/c128. The dropped points (TP2 c1/c2/c4/c8, TP4 c2/c8/c32/c64/c128) are dominated on the tok/s/GPU vs P90 tok/s/user Pareto by the ones kept.
3. TP4 c1: 16K prefill chunks with no prefill-decode interval, instead of 4K chunks with a 16-step interval.
4. TP4 c4/c16 run EP1 instead of EP4. At batch 1–5, EP4 makes every decode step wait for the rank holding the most-selected experts. c16 also gets a 192-step prefill-decode interval, 4 tokenizer workers and SGLANG_NUMA_BIND_V2=0: one prefill forward costs about 200 ms of host dispatch on Grace and stalls every running decode.
5. TP2 c16/c32: 16K chunks with decoder SWA bounded replay and the prefill CUDA graph disabled, a 96/48-step prefill-decode interval, expandable_segments and 4 tokenizer workers. c16 also sets SGLANG_NUMA_BIND_V2=0.
6. c64/c128 move from TP2 to DEP2:
   • TP2/DP2/EP2 with DP attention, DP LM head and MegaMoE, behind sglang-router (cache_aware, dp-aware, routing key from the AIPerf correlation id);
   • total_requests load balancing, shortest-prefill-first scheduling, 16K chunks and 4096 SWA tails;
   • a 24-step prefill-decode interval at c64 and a 16-step one at c128; c128 also uses mem-fraction-static 0.85 and admits 256.

Local results

One-hour AgentX runs on GB300 with the image in this PR, using the CI harness with synthetic acceptance 3.51. Throughput is in tok/s/GPU; interactivity is P90 (median) tok/s/user.

point	throughput	interactivity	P90 TTFT
TP4 c1	7,314	625.7 (670.3)	1.38 s
TP4 c4 (EP1)	9,338	505.9 (668.7)	1.66 s
TP4 c16 (EP1)	28,838	424.9 (538.2)	2.68 s
TP2 c16	55,560	264.6 (324.9)	2.36 s
TP2 c32	113,613	183.1 (242.7)	3.13 s
DEP2 c64	200,265	114.3 (132.7)	2.90 s
DEP2 c128	271,364	81.6 (93.0)	4.35 s

The TP2 points that DEP2 replaces measured 149,666 at 99.4 median (c64) and 147,763 at 37.9 median with a 51 s P90 TTFT (c128) in #3782.

TP4 c4/c16 on EP1 come from a separate config-only GB300 tuning campaign (20261006 nightly: c4 9,460 at 502.0 P90, c16 28,771 at 376.8 P90; on the same image there, the c16 interval took P90 from 217 to 334 and EP1 added +4.7% at c16 and +6.3% at c4). The published sweep has 10,017 at 471.5 (c4) and 28,694 at 222.2 (c16). On the 20261012 image, TP4 c16 also dominates the dropped TP4 c8 (16,226 at 415.8), TP2 c4 (18,119 at 383.5) and TP2 c1/c2 (12,443 at 431.7, 12,689 at 392.6); TP4 c1 dominates TP4 c2 (7,363 at 569.7).

TP4 c1 chunking A/B (30 min, with fastokens 0.3.2 in both arms): 5,826 → 5,957 tok/s/GPU, P90 interactivity 561.6 → 566.4, P90 TTFT 1.38 → 0.46 s, subagent P90 TTFT 2.51 → 0.70 s.

Notes

• The image ships fastokens 0.1.x, so the recipe does not enable the fastokens tokenizer backend; [Tokenizer] Require fastokens>=0.3.2 and include it in the all extra sgl-project/sglang#43104 adds support for 0.3.2. The TP2/DEP2 settings with fastokens 0.3.2 installed on top gave similar throughput with lower TTFT.

AI model disclosure

• Model/version: Claude Opus 5.5 (claude-opus-5-5)
• Role: ran the local experiments, wrote the recipe, master-config and changelog changes, and drafted this description.

Related Issue

Follows #3782.

Type of Change

• Bug fix
• New feature
• Configuration change
• Documentation update
• Other (please describe)

Checklist

• I have completed the AI model disclosure and kept it current
• I have tested my changes locally
• I have updated documentation if necessary
• For every change that can affect benchmark performance and every recipe addition or modification, I have appended a new entry to the physical end of inferencex-e2e/perf-changelog.yaml and have not edited historical entries
• If this PR can affect benchmark performance or adds or modifies a recipe, it carries exactly one primary sweep label (a maintainer applies it on fork PRs): full-sweep-fail-fast (recommended), full-sweep-enabled, or non-canary-full-sweep-enabled. Optional modifiers all-evals, evals-only, and agentx-fast require a primary label; the last two block reuse while applied.
• Before merging via reuse, an authorized maintainer (OWNER/MEMBER/COLLABORATOR) has commented /use <run_id> (or the legacy /reuse-sweep-run) on this PR. Do this only once there is a final full sweep that is all green with evals passing, since after this comment the primary sweep label will no longer automatically kick off new sweeps. Remove and re-add the primary sweep label to force a new sweep.

中文

配置：dsv41flash-fp4-gb300-sglang-agentic-dspark（聚合、单节点），只改配方、master 配置和 changelog。

1. 镜像升级到 lmsysorg/sglang:nightly-dev-cu13-20261012-e6072935，按 digest 固定。
   • 默认使用 Rust radix tree（[Radix Cache] Sync Rust TreeCore and make it the default sgl-project/sglang#39627），去掉了 Python 插入路径。
   • 支持 DP attention 与 MegaMoE 组合（[1/N][DSv41] Enable DP attention with MegaMoE sgl-project/sglang#43228）。
2. 点位由 16 个精简为 7 个：TP4 c1/c4/c16、TP2 c16/c32、DEP2 c64/c128。删掉的点在吞吐对交互性的 Pareto 上被保留的点压过。
3. TP4 c1：16K chunk，不设 prefill-decode interval。
4. TP4 c4/c16 改为 EP1：小 batch 下，EP4 每一步都要等负载最重的 rank。c16 另加 192 步 interval、4 个 tokenizer worker 和 SGLANG_NUMA_BIND_V2=0。
5. TP2 c16/c32：
   • 16K chunk，开启 bounded replay；
   • interval 分别为 96/48；
   • 开启 expandable_segments，使用 4 个 tokenizer worker。
6. c64/c128 改为 DEP2：
   • DP attention + MegaMoE，前端为 sglang-router；
   • shortest-prefill-first 调度；
   • c128 设 mem-fraction 0.85，最多接纳 256 个请求。

本地结果见上表：一小时运行，合成接受长度 3.51。TP4 c4/c16 的 EP1 配置来自另一组只改配置的 GB300 调优，20261012 镜像上复测：TP4 c4 9,338 @ 505.9，TP4 c16 28,838 @ 424.9（官方曲线为 222.2）。

说明：镜像内的 fastokens 为 0.1.x，未启用 fastokens 后端（sgl-project/sglang#43104 会支持 0.3.2）。

AI 模型使用说明：Claude Opus 5.5（claude-opus-5-5），负责本地实验、配方 / master 配置 / changelog 修改，并起草本说明。

[github-actions]

Thanks for the contribution!

• Review: If this PR changes files owned by someone other than a repository admin or @SemiAnalysisAI/core, ask one eligible CODEOWNER to complete the latest PR_REVIEW_CHECKLIST.md before contacting a core maintainer on Slack. Follow the template exactly, including As a PR reviewer and CODEOWNER, I have reviewed this and have, so sign-off verification triggers.
• PR verification: Sweeps run only when the PR appends an inferencex-e2e/perf-changelog.yaml entry and carries exactly one primary label: full-sweep-fail-fast (strongly recommended; canary plus per-matrix fail-fast), full-sweep-enabled (canary; matrix jobs continue after a failure), or non-canary-full-sweep-enabled (no canary or fail-fast). The modifiers all-evals, evals-only, and agentx-fast require a primary label. On fork PRs, a maintainer applies the label. See sweep labels and reuse.
• After merging: PR authors must ensure all GitHub Actions jobs pass. Transient failures often pass on rerun; see how to rerun failed jobs.

中文

感谢你的贡献！

• **审阅：**如果 PR 修改的文件归属于仓库管理员及 @SemiAnalysisAI/core 之外的 CODEOWNER，请先联系一位有资格的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，再通过 Slack 联系核心维护者。必须严格遵循模板，并保留 As a PR reviewer and CODEOWNER, I have reviewed this and have，才能触发签核验证。
• **PR 验证：**只有当 PR 在 inferencex-e2e/perf-changelog.yaml 末尾追加了条目，并且恰好带有一个主标签时，才会运行扫描：full-sweep-fail-fast（强烈推荐；canary 加逐矩阵 fail-fast）、full-sweep-enabled（有 canary；矩阵任务在失败后继续运行）或 non-canary-full-sweep-enabled（无 canary，也无 fail-fast）。修饰标签 all-evals、evals-only 和 agentx-fast 必须与主标签一起使用。fork PR 的标签由维护者添加。参见扫描标签与复用。
• **合并后：**PR 作者必须确保所有 GitHub Actions 任务通过。临时性失败通常可以通过重新运行恢复；参见重新运行失败任务的说明。

[claude]

I reviewed this PR and didn't find any bugs. Because it bumps a production GB300 benchmark recipe (image digest, DSpark sampling fold, per-point chunked-prefill/interval retuning) and both changed files (nvidia-master.yaml and the gb300-*/ recipe tree) are covered by CODEOWNERS, a human look would still be worthwhile.

What was reviewed: confirmed the image digest bump is applied consistently in both agentic.yaml and nvidia-master.yaml; checked perf-changelog.yaml additions are strictly appended at the tail with no edits to prior entries; cross-checked the retuned tp2_c128/tp4_c1/tp4_c64 overrides against the updated header comment for consistency; and noted the PR's own disclosure that its two stacked changelog blocks (PR #3782 and #XXX) depend on merge order.

Extended reasoning...

The diff is a config-only change: an SGLang nightly image/digest bump, a new env var, and per-point prefill-chunking/interval retunes in a GB300 benchmark recipe, plus the matching master-config image entry and two appended perf-changelog blocks; it touches no auth, crypto, or data-handling code. Both changed non-changelog files fall under CODEOWNERS (nvidia-master.yaml and the gb300-* recipe path), which per the approval guidelines rules out approving regardless of correctness. The bug hunt reported zero findings; three candidate issues (two implausible/unverifiable changelog numbers in the PR #3782 block, and a known reuse-merge ordering conflict from carrying two changelog blocks) were investigated and are either out of this PR's reviewable scope (belong to the not-yet-merged #3782) or already disclosed by the author as an expected, self-resolving sequencing issue, so none rises to a flagged finding.

This review covers commit 4c1105a, which is no longer the latest commit on this pull request; later commits are not covered by it.

[github-actions]

View unofficial run (performance): https://inferencex.semianalysis.com/inference?unofficialRun=38220651569

View unofficial run (accuracy): https://inferencex.semianalysis.com/evaluation?unofficialRun=38220651569

[nvpohanh]

[by Claude Code] The SGLang cookbook integration of this PR's AgentX config (dsv41flash-fp4-gb300-sglang-agentic-dspark) will happen in sgl-project/sglang#43799, in the Agentic Long-Context Deployment section of the DeepSeek-V4.1 cookbook page.

This PR has no authorized /use <run_id> comment yet, so the cookbook shows the recipe at this PR's head (e182134fd) as "in progress". It switches to the measured run at the next cookbook sync after a run is pinned or this PR merges.

[nvpohanh]

[by Claude Code] sgl-project/sglang#43799 has been merged. This PR's AgentX config (dsv41flash-fp4-gb300-sglang-agentic-dspark) is now in the Agentic Long-Context Deployment section of the SGLang DeepSeek-V4.1 cookbook: https://docs.sglang.io/cookbook/autoregressive/DeepSeek/DeepSeek-V4_1#agentic-long-context-deployment

This PR has no authorized /use <run_id> comment yet, so the cookbook shows the recipe at this PR's head (e182134fd) as "in progress". It switches to the measured run at the next cookbook sync after a run is pinned or this PR merges.