Repository navigation
[AgentX] tune Muse Spark Open B200 for TP8/EP8 / [AgentX] 调优 Muse Spark Open B200 TP8/EP8 配置 - #4155
Conversation
将 B200 AgentX 配置调整为 TP8/EP8，保留 TP4 的全部并发点并增加并发 48。
|
Thanks for the contribution!
中文感谢你的贡献！
|
为 B200 TP8 性能记录补充 PR 链接。
edwingao28
added
the
full-sweep-fail-fast
|
View unofficial run (performance): https://inferencex.semianalysis.com/inference?unofficialRun=38851010434 View unofficial run (accuracy): https://inferencex.semianalysis.com/evaluation?unofficialRun=38851010434 |
8 ranks x (216 GB main + ~63 GB draft-model host pool) exceeded the node's ~1.86 TiB cgroup, so the draft pool allocation failed at startup. 160 GB per rank leaves ~1.65 TB total including the draft sidecar.
Oseltamivir
added
full-sweep-fail-fast
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
| description: | ||
| - "Update Muse Spark Open B200 from TP4/EP4 to TP8/EP8 using the LMSYS AgentX recipe, with the same NVFP4 checkpoint, pinned SGLang image and DFlash2 draft." | ||
| - "Enable block verification and breakable prefill CUDA graphs; use mem-fraction-static 0.9, SWA/full ratio 0.4, admission 192 and decode graph batch 128." | ||
| - "Sweep concurrency 1, 2, 4, 8 and 12 with GPU-resident KV; enable layernorm SP, MegaMoE and HiCache at 16, 24, 32 and 48, with 120 GB per rank at 16 and 216 GB per rank at 24/32/48 within a 90% host-DRAM budget." |
There was a problem hiding this comment.
🟡 (optional) Readers of the append-only changelog get a host-cache size the benchmark never uses: it says 216 GB per rank at 24/32/48. The recipe sets hicache-size: 160 for override_tp8_c24, override_tp8_c32 and override_tp8_c48, so the entry does not describe the final state. Fix: make the entry and the recipe agree before merge, either by writing "160 GB per rank at 24/32/48" or by changing those three overrides to hicache-size: 216. Historical entries cannot be edited later. [also at: inferencex-e2e/perf-changelog.yaml:9925 - Readers of the perf changelog are told the high-concurrency runs use 216 GB of host cache per rank, but the recipe actually runs 160 GB.]
Why this was flagged
The new entry at inferencex-e2e/perf-changelog.yaml:9924 says "120 GB per rank at 16 and 216 GB per rank at 24/32/48". The recipe inferencex-e2e/benchmarks/single_node/srt-slurm-recipes/musespark/sglang/b200-fp4-mtp/agentic.yaml sets hicache-size: 160 in override_tp8_c24, override_tp8_c32 and override_tp8_c48. The PR description also claims 216 GB. 216 GB matches the per-rank share of the 1731 GB B200 budget at dram-utilization 0.90 used by other B200 recipes (1731/8 = 216). So either the recipe under-sizes HiCache compared with the stated intent, or the changelog misreports it. AGENTS.md treats the file as append-only, and a PR's block must describe its final state. Once merged, the wrong 216 GB figure stays permanently attached to these results, and no generator or schema check compares the description text with the recipe.
Verification: The new perf-changelog.yaml entry (line 9924) says "...120 GB per rank at 16 and 216 GB per rank at 24/32/48 within a 90% host-DRAM budget." The recipe musespark/sglang/b200-fp4-mtp/agentic.yaml sets hicache-size: 120 in override_tp8_c16, which matches, but sets hicache-size: 160 in override_tp8_c24, override_tp8_c32 and override_tp8_c48. Once merged, the wrong number stays in the history for good.
# Conflicts: # inferencex-e2e/perf-changelog.yaml
Oseltamivir
added
non-canary-full-sweep-enabled
|
/use 38851010434 |
|
@Oseltamivir staged run 38851010434. This run remains available across future |
2 participants
Summary
Update B200 from TP4/EP4 to TP8/EP8 using the LMSYS AgentX settings, retaining every TP4 concurrency point and adding 48. Enable HiCache from concurrency 16; keep the NVFP4 model, pinned image, DFlash2 draft and golden acceptance unchanged. High concurrency uses 216 GB of host cache per rank.
AI model disclosure
PR checklist and change type
Type of Change
Checklist
python -m infx.matrix.changed --base origin/mainlists affected config keys, I have appended one new entry covering them to the physical end ofinferencex-e2e/perf-changelog.yaml; if it lists none, I have added none. I have not edited historical entriesinferencex-e2e/perf-changelog.yamlentry, it carries exactly one primary sweep label (a maintainer applies it on fork PRs):full-sweep-fail-fast(recommended),full-sweep-enabled, ornon-canary-full-sweep-enabled. Optional modifiersall-evals,evals-only, andagentx-fastrequire a primary label; the last two block reuse while applied.OWNER/MEMBER/COLLABORATOR) has commented/use <run_id>(or the legacy/reuse-sweep-run) on this PR. Do this only once there is a final full sweep that is all green with evals passing, since after this comment the primary sweep label will no longer automatically kick off new sweeps. Remove and re-add the primary sweep label to force a new sweep.中文
将 B200 从 TP4/EP4 调整为 TP8/EP8，采用上方 LMSYS AgentX 配置，保留 TP4 的全部并发点并增加 48。从并发 16 起启用 HiCache；NVFP4 模型、固定镜像、DFlash2 草稿模型和 golden acceptance 保持不变。高并发时每个 rank 使用 216 GB 主机缓存。
改动类型：配置变更。AI 模型：无法从运行环境确认精确型号或版本；负责配置修改与验证。