[Doc] cookbook: Refresh SemiAnalysis AgentX launches to InferenceX main (GLM-5.2 NVFP4 B300 agg TP4, Dynamo frontend counts)

[nvpohanh]

[by Claude Code]

Motivation

Refresh the cookbook's interactive Agentic Long-Context Deployment sections (#42726, #43083, #43799, #43983) on GLM-5.2, DeepSeek-V4, DeepSeek-V4.1, Qwen3.5 and Qwen3.8-Flash-Next against the latest InferenceX main. Only two open InferenceX PRs stay overlaid. The configurations are mirrored from SemiAnalysis submissions as raw launch commands, each linked to the run that measured it.

This PR adds the GLM-5.2 NVFP4 B300 Dynamo+SGLang aggregated TP4 submission from InferenceX #3953. It stops overlaying four open InferenceX PRs that are not ready for submission. Lanes that exist only in those PRs leave the pages, and lanes they had revised go back to their measured main versions. It also corrects two pipeline errors: the number of Dynamo frontends shown, and four GLM-5.2 GB200 lanes that were pinned to outdated runs. The Deployment panels are unchanged.

Source

InferenceX main (01e3b46a), with these open InferenceX PRs overlaid:

PR	Page	Lane(s)	Pinned to
#3953 GLM-5.2 NVFP4 B300 Dynamo+SGLang	GLM-5.2	glm5.2-fp4-b300-dynamo-sglang-agentic-agg-tp4	no /use yet: in progress, PR head 56b2949a
#3927 DSV4.1 GB200 SGLang	DeepSeek-V4.1	dsv41flash-fp4-gb200-sglang-agentic-dspark	run 38266927261 (/use), verified

A lane from a PR without an authorized /use run is rendered at the PR head and labeled "In progress in SA InferenceX AgentX". It switches to the measured run on the next sync after a /use comment or a merge.

Changes in the overlay list since #43983:

• Merged into InferenceX main. InferenceX #3859 (dsv41flash-fp4-gb300-sglang-agentic-dspark) and InferenceX #3836 (glm5.2-fp8-b300-sglang-agentic-mtp). Both lanes are now pinned to their dashboard runs on main: run 38221097558 and run 38304664439. These are the runs their /use comments named, and the commands are unchanged.
• No longer overlaid (not ready for submission). InferenceX #3817, InferenceX #2978, InferenceX #3606 and InferenceX #3860. The effect on each page is listed below.

What this PR does

Page	Lanes	Deployments	Points	Hardware
GLM-5.2	16	27	67	GB300, GB200, B300, B200, H200
DeepSeek-V4	10	20	61	GB300, GB200, B300, B200, H200
DeepSeek-V4.1	9	19	66	GB300, GB200, B300, B200, H200, H100
Qwen3.5	18	46	216	GB300, GB200, B300, B200, H200, H100
Qwen3.8-Flash-Next	4	4	22	GB300, B300, B200, H200

Only the generated data files change. The component (_agentx.jsx), page text, Deployment panels and Playgrounds are untouched.

• GLM-5.2 NVFP4 B300 Dynamo+SGLang (InferenceX #3953), new. Two deployments of one aggregated TP4 engine on 4 of a B300 node's 8 GPUs, behind a Dynamo frontend with the KV router. "Agg TP4" covers c1, c8, c16 and c20. "Agg TP4·DPA4" (DP4 attention with the prefill delayer) covers c24, c32 and c48. Both run EAGLE MTP 5-1-6 on the checkpoint's built-in NextN head with HiCache (host DRAM), on lmsysorg/sglang:nightly-dev-cu13-20260901-07c8f729 with ai-dynamo 1.5.0.dev20260903. The lane is in progress at the PR head until it gets a /use run. The Mooncake tier is shown as not available because this image predates the external linker.

• Lanes of the PRs that are no longer overlaid. The pages now show what InferenceX main has for these lanes:

  • DeepSeek-V4: the B300 Dynamo+SGLang deployments from InferenceX #3817 (dsv4-fp4-b300-dynamo-sglang-agentic-agg and -disagg, 5 deployments) are removed. Those keys exist only in that PR.
  • DeepSeek-V4: dsv4-fp4-b300-sglang-agentic-hicache-mtp goes back from the in-progress revision of InferenceX #3606 (Agg TP4, TP8 and DEP8) to its measured main version: InferenceX #2918, run 34595563156, nightly-dev-20260901-07c8f729. That version has Agg TP8 at c1–c32 and Agg DEP8 with HiCache at c32–c576.
  • DeepSeek-V4: the 12 B300 EAGLE points that run without KV offload (dsv4-fp4-b300-sglang-agentic-hicache-eagle, Agg TP4 and Agg TP8 at c1–c32) have a derived HiCache variant. It took its flags from [Bug] #3817's lane and now takes them from the B300 SGLang HiCache lane above, so --hicache-ratio 2.75 --hicache-write-policy write_through becomes --hicache-ratio 3 --hicache-write-policy write_back.
  • DeepSeek-V4.1: dsv41flash-fp4-gb300-dynamo-sglang-agentic-disagg goes back from the 8 in-progress deployments of InferenceX #3860 to the 3 measured on main: InferenceX #3598, run 37260940238. These are 1P2D at c16–c64, 1P1D at c96 and c160, and 2P1D at c256.
  • Qwen3.5: the B300 Dynamo+SGLang pareto deployments from InferenceX #2978 (qwen3.5-fp4-b300-dynamo-sglang-agentic-pareto, 4 deployments) are removed. That key exists only in that PR.

• GLM-5.2 GB200 Dynamo+SGLang re-pinned (fix). Since InferenceX #3401 merged on 2026-10-07, the dashboard has served glm5.2-fp4-gb200-dynamo-sglang-agentic-agg, -disagg, -mtp and -mtp-agg from run 37506762704. The pipeline had cached that day's dashboard changelog before Failed run multiple-nodes between two nodes #3401 landed, so earlier cookbook PRs kept these lanes on the August runs of InferenceX #2620 and InferenceX #2642. Now they are pinned to the Failed run multiple-nodes between two nodes #3401 run. The workers set SGLANG_NVFP4_CKPT_FP8_NEXTN_MOE=0 instead of 1 (Failed run multiple-nodes between two nodes #3401: "Preserve draft precision"). The disaggregated and MTP lanes run lmsysorg/sglang:v0.5.18-cu130 (the arm64 digest SA pinned) instead of v0.5.17-cu130 and nightly-dev-cu13-20260805-211ee642. Nothing else in the commands changes.

• Dynamo frontend counts (fix). The pipeline previously rendered each launch on the job's nodes plus two spare nodes. With multiple frontends enabled, srt-slurm puts a frontend on every non-head node, so these deployments showed two more dynamo.frontend replicas than SemiAnalysis launched. Single-node deployments also showed an nginx that never ran. The launches are now rendered on exactly the nodes each job allocated, using srt-slurm's own sbatch node count. This was checked against four real SemiAnalysis job logs. A 6-node GLM-5.2 GB200 1P4D job ran 5 frontends, where the page showed 7. A 1-node Qwen3.5 B300 1P1D job ran one frontend with no nginx, and pinned prefill and decode to GPUs 0–3 and 4–7 of the same node. Affected: 29 deployments in 19 lanes.

  • GLM-5.2: B200 agg / disagg, GB200 agg / disagg / MTP / MTP-agg, GB300 agg / disagg, H200 MTP agg / 1P1D / 2P2D.
  • DeepSeek-V4: B200 disagg, H200 agg.
  • Qwen3.5: GB300 FP4 agg (SGLang-router variant) / disagg, GB300 FP8 PP pareto / midcurve, B200 FP8 disagg-MTP, B300 FP8 disagg.

  These deployments change in three ways. The frontend replica count is corrected. nginx is removed where the job had one node, and the frontend then listens on port 8000. The two single-node Qwen3.5 disaggregated lanes also change their worker GPU pinning. Worker commands are unchanged.

How the commands are produced

The pipeline and rules are the same as in #42726, #43083, #43799 and #43983, with these changes:

• Frontend node count. The fix described above.
• InferenceX's composed recipes. InferenceX main now builds each srt-slurm recipe from a fragment plus shared blocks, and binds the matrix point at launch (InferenceX #3805, #3809, #3818, #3820 and #3824, merged on 2026-10-13). A lane pinned to such a commit (here only [Bug] RecursionError: maximum recursion depth exceeded while calling a Python object #3953, whose head merged that refactor) is rendered from the recipe that InferenceX's own binder (infx generate) produces at that commit. For [Bug] RecursionError: maximum recursion depth exceeded while calling a Python object #3953, this gives the same server commands as the pre-refactor head caa2199e did.
• Dashboard refresh. Recent dashboard dates are now re-fetched on every sync. This is how the GLM-5.2 GB200 re-pin above was found.
• New keep rule. --prefill-delayer-max-delay-passes, an SGLang scheduler setting for --enable-prefill-delayer that [Bug] RecursionError: maximum recursion depth exceeded while calling a Python object #3953 uses at c48.

Each lane is rendered at the exact commit of the run that produced its dashboard data. For an open PR, that is the run named by its authorized /use comment, or the PR head when there is none. SA-only settings are removed: the benchmark harness, cluster paths and caches, telemetry, and InferenceX's synthetic speculative acceptance (disclosed per deployment). Placeholders replace IPs, ports and NICs. Router and KV-offload variants that were not run are derived and labeled "Derived — not run".

• Not mirrored. qwen3.5-fp8-h100-sglang-agentic (H100 without MTP) has never produced dashboard data, so there is no run to pin it to.

The data files are generated, so there is no hand-written code to review here.

Validation

• mint validate (strict build) and mint broken-links --check-anchors --check-redirects pass. node docs/scripts/check_cookbook_configs.mjs passes, and so does pre-commit run --all-files.
• A headless browser clicked through every combination of hardware, checkpoint, deployment, concurrency, KV tier and router on all five pages under mint dev, opening every tab. That is 2,144 command sets: GLM-5.2 328, DeepSeek-V4 316, DeepSeek-V4.1 396, Qwen3.5 1,028 and Qwen3.8-Flash-Next 76. The rendered commands match the expected text exactly. For every deployment, the "Copy link" URL of a non-default selection was reopened in a fresh tab and restored the same selection and tab (116 links).
• The Deployment-panel footer link still appears on GLM-5.2, DeepSeek-V4, DeepSeek-V4.1 and Qwen3.8-Flash-Next and scrolls to the section. Qwen3.5's anchor sits directly above the explorer, and pages that use the Deployment panel without the prop still show no link.
• The component's decoder and KV-tier transform, run through Node, reproduce the Python-generated commands for all 2,144 combinations, including every point's measured tier. Every KV-cache flag in a shown command exists in the SGLang commit of its deployment's image (190 image/flag pairs).
• Compared with main, the only changes are the ones listed above. GLM-5.2 adds 28 [Bug] RecursionError: maximum recursion depth exceeded while calling a Python object #3953 command sets (7 points × 2 routers × 2 KV tiers). DeepSeek-V4, DeepSeek-V4.1 and Qwen3.5 drop 88, 12 and 20 command sets with the lanes and deployments removed above. The remaining changes are the GLM-5.2 GB200 re-pin, the reverted DeepSeek-V4 / DeepSeek-V4.1 lanes, the re-derived DeepSeek-V4 EAGLE HiCache variant, and the frontend topology. Every other deployment renders byte-identical commands on all five pages, including the derived router and KV-offload variants. That includes InferenceX #3836, InferenceX #3859 and InferenceX #3927, and all of Qwen3.8-Flash-Next.
• Single-node commands from bash launchers were accepted only where a dry run of the pinned launcher reproduced the server command in the run's job log (233 points, including the 12 of the DeepSeek-V4 B300 SGLang lane on main).
• Every submitted and derived sglang.launch_server, dynamo.sglang, sglang_router.launch_router and dynamo.frontend command (1,848 distinct, across 29 images) parses with the argument parsers of its own submission image (CPU only). The 58 commands that are new in this PR were parsed now:
  • The 24 GLM-5.2 GB200 commands in v0.5.18-cu130 ran in the amd64 build of that tag. SA pinned its arm64 digest, and both platforms carry the same SGLang build (71de97b2, one build run). Their Dynamo commands were checked against the Dynamo source build at 71eb001e that the run used.
  • The 24 re-derived DeepSeek-V4 B300 EAGLE HiCache commands ran in their own image, dev-cu13-nightly-09242. The 10 derived HiCache commands of the DeepSeek-V4 B300 SGLang lane's Agg TP8 points ran in theirs, nightly-dev-20260901-07c8f729.
  • The rest are identical to commands parsed for [Docs] Cookbook: interactive SemiAnalysis InferenceX AgentX launches for GLM-5.2, DeepSeek-V4 and DeepSeek-V4.1 #42726, [Docs] Cookbook: AgentX launches for Qwen3.5 / Qwen3.8-Flash-Next; refresh GLM-5.2, DeepSeek-V4, DeepSeek-V4.1 #43083, [Docs] Cookbook: refresh SemiAnalysis AgentX launches (GLM-5.2 B300, DeepSeek-V4.1 GB300) #43799, [Docs] Cookbook: refresh SemiAnalysis AgentX launches (DeepSeek-V4.1 GB200/GB300, GLM-5.2 FP8 B300) #43983 and the previous push of this PR.

No serving or benchmark runs were performed for this PR; the numbers live in the linked SemiAnalysis runs. SGLang CI was not triggered.

🤖 Generated with Claude Code

---

CI States

Latest PR Test (Base): ✅ Run #38510642994
Latest PR Test (Extra): ❌ Run #38510642945
Latest PR Test (AMD ROCm 10): ➖ No AMD PR run found for this commit.

[faradawn]

LGTM

[nvpohanh]

looks good locally. force-merge to unblock SA AgentX submission.