Skip to content

[Klaud Cold] Update DSR1 FP4 B200 SGLang MTP image to v0.5.21-cu130 / [Klaud Cold] 将 DSR1 FP4 B200 SGLang MTP 镜像更新至 v0.5.21-cu130 - #4088

Closed
Klaud-Cold wants to merge 1 commit into
mainfrom
klaud/auto-72260f70af9bbbbc-555679de269db36f
Closed

Klaud-Cold wants to merge 1 commit into
mainfrom
klaud/auto-72260f70af9bbbbc-555679de269db36f

Conversation

@Klaud-Cold

@Klaud-Cold Klaud-Cold commented Oct 15, 2026 •

Copy link
Copy Markdown
Collaborator

Goal: Update SGLang image from lmsysorg/sglang:v0.5.16-cu130 to lmsysorg/sglang:v0.5.21-cu130.
Baseline: 2026-08-06 · lmsysorg/sglang:v0.5.16-cu130
Mean latency · Sources: API 1, API 2

Point Total tok/s/GPU ↑ Output tok/s/GPU ↑ TTFT ms ↓ TPOT ms ↓
8k/1k c1 TP4 EP1 1ecad2 664.03 74.17 205.09 3.16
8k/1k c2 TP4 EP1 b79296 1,120.85 125.76 250.4 3.59
8k/1k c4 TP4 EP1 d56e03 1,649.89 183.51 296.12 4.98
8k/1k c8 TP4 EP1 298215 2,321.26 260.92 412.66 7
8k/1k c16 TP4 EP1 c2ec41 2,940.82 325.82 826.07 11.1
8k/1k c32 TP4 EP1 fb2b10 3,839.76 429.34 1,288.53 16.88
8k/1k c64 TP4 EP4 6b9bd7 5,786.09 641.84 1,462.11 22.72
8k/1k c128 TP4 EP4 c02deb 7,290.73 807.46 1,991.22 36.79
8k/1k c256 TP4 EP4 98a050 8,783.42 976.13 6,426.93 57.6

Eval: N/A

中文

**目标:**将 SGLang 镜像从 lmsysorg/sglang:v0.5.16-cu130 更新至 lmsysorg/sglang:v0.5.21-cu130。
**基线:**2026-08-06 · lmsysorg/sglang:v0.5.16-cu130
平均延迟 · 来源: API 1, API 2;数值及异常说明见上表。

Pin lmsysorg/sglang:v0.5.21-cu130 by digest and replace the CLI aliases
v0.5.21 removed (--cuda-graph-max-bs, --disable-piecewise-cuda-graph)
with --cuda-graph-max-bs-decode and --cuda-graph-backend-prefill=disabled.

将 dsr1-fp4-b200-sglang-mtp 的 SGLang 镜像更新至 v0.5.21-cu130(按 digest 固定),
并将 v0.5.21 已移除的 CLI 别名替换为 --cuda-graph-max-bs-decode 和
--cuda-graph-backend-prefill=disabled。

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

Thanks for the contribution!

  • Review: If this PR changes files owned by someone other than a repository admin or @SemiAnalysisAI/core, ask one eligible CODEOWNER to complete the latest PR_REVIEW_CHECKLIST.md before contacting a core maintainer on Slack. Follow the template exactly, including As a PR reviewer and CODEOWNER, I have reviewed this and have, so sign-off verification triggers.
  • PR verification: Sweeps run only when the PR appends an inferencex-e2e/perf-changelog.yaml entry and carries exactly one primary label: full-sweep-fail-fast (strongly recommended; canary plus per-matrix fail-fast), full-sweep-enabled (canary; matrix jobs continue after a failure), or non-canary-full-sweep-enabled (no canary or fail-fast). The modifiers all-evals, evals-only, and agentx-fast require a primary label. On fork PRs, a maintainer applies the label. See sweep labels and reuse.
  • After merging: PR authors must ensure all GitHub Actions jobs pass. Transient failures often pass on rerun; see how to rerun failed jobs.
中文

感谢你的贡献!

  • **审阅:**如果 PR 修改的文件归属于仓库管理员及 @SemiAnalysisAI/core 之外的 CODEOWNER,请先联系一位有资格的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md,再通过 Slack 联系核心维护者。必须严格遵循模板,并保留 As a PR reviewer and CODEOWNER, I have reviewed this and have,才能触发签核验证。
  • **PR 验证:**只有当 PR 在 inferencex-e2e/perf-changelog.yaml 末尾追加了条目,并且恰好带有一个主标签时,才会运行扫描:full-sweep-fail-fast(强烈推荐;canary 加逐矩阵 fail-fast)、full-sweep-enabled(有 canary;矩阵任务在失败后继续运行)或 non-canary-full-sweep-enabled(无 canary,也无 fail-fast)。修饰标签 all-evals、evals-only 和 agentx-fast 必须与主标签一起使用。fork PR 的标签由维护者添加。参见扫描标签与复用。
  • **合并后:**PR 作者必须确保所有 GitHub Actions 任务通过。临时性失败通常可以通过重新运行恢复;参见重新运行失败任务的说明。

@Klaud-Cold

Klaud-Cold commented Oct 15, 2026 •

Copy link
Copy Markdown
Collaborator Author

Initial attempt · Failed · Run 38682595271 / attempt 1 · 2026-10-15 08:26 UTC
lmsysorg/sglang:v0.5.21-cu130@sha256:b1259f3ea3275f66237c498ea388919729018bc9f01c3d638391e06e2cf3f469 · c20b965a27a9 · Mean latency
Change: Pin lmsysorg/sglang:v0.5.21-cu130 by index digest (image labels: commit e00930c5 = v0.5.21 tag; flashinfer 0.6.14 to 0.6.18, sglang-kernel 0.4.5 to 0.4.7, torch 2.11 to 2.13) and replace the CLI aliases v0.5.21 removed from server_args.py, --cuda-graph-max-bs 256 to --cuda-graph-max-bs-decode 256 and --disable-piecewise-cuda-graph to --cuda-graph-backend-prefill disabled; all other recipe flags still parse (https://github.com/sgl-project/sglang/blob/v0.5.16/python/sglang/srt/server_args.py, https://github.com/sgl-project/sglang/blob/v0.5.21/python/sglang/srt/arg_groups/fields/exec_.py); SGLANG_ENABLE_SPEC_V2 was already a no-op in v0.5.16 and is left as is.

Point Output tok/s/GPU ↑ TTFT ms ↓ TPOT ms ↓
8k/1k c1 TP4 EP1 1ecad2 N/A N/A N/A
8k/1k c64 TP4 EP4 6b9bd7 N/A N/A N/A

Note: 8k/1k c1 TP4 EP1 1ecad2: cancelled.
Note: All rows: request errors unavailable.
Note: All rows: Δ N/A: point unavailable.
Note: 8k/1k c64 TP4 EP4 6b9bd7: failed.

Next: Both Slurm jobs (c64 throughput and c128 eval) stopped in srtctl before the container started because node-local weights /scratch/models/DeepSeek-R1-0528-NVFP4-v2 are missing on b200-nscale, so the candidate is readiness-blocked until the checkpoint is restaged; c1 and c32 were cancelled by fail-fast and the image was never exercised.

中文

初次尝试 · 失败 · Run 38682595271 / attempt 1 · 2026-10-15 08:26 UTC
lmsysorg/sglang:v0.5.21-cu130@sha256:b1259f3ea3275f66237c498ea388919729018bc9f01c3d638391e06e2cf3f469 · c20b965a27a9 · 平均延迟
**变更:**按 index digest 固定 lmsysorg/sglang:v0.5.21-cu130(镜像标签:commit e00930c5 即 v0.5.21 tag;flashinfer 0.6.14 改为 0.6.18、sglang-kernel 0.4.5 改为 0.4.7、torch 2.11 改为 2.13),并替换 v0.5.21 从 server_args.py 中移除的 CLI 别名:--cuda-graph-max-bs 256 改为 --cuda-graph-max-bs-decode 256,--disable-piecewise-cuda-graph 改为 --cuda-graph-backend-prefill disabled;其余配方参数仍可解析(https://github.com/sgl-project/sglang/blob/v0.5.16/python/sglang/srt/server_args.py, https://github.com/sgl-project/sglang/blob/v0.5.21/python/sglang/srt/arg_groups/fields/exec_.py);SGLANG_ENABLE_SPEC_V2 在 v0.5.16 中已无作用,保持不变。实测数值及异常说明见上表。
**下一步:**两个 Slurm 任务(c64 吞吐与 c128 评测)均在容器启动前于 srtctl 中止,原因是 b200-nscale 上缺少节点本地权重 /scratch/models/DeepSeek-R1-0528-NVFP4-v2,因此该候选在权重重新部署前处于就绪阻塞状态;c1 与 c32 被 fail-fast 取消,镜像未被实际运行。

@Klaud-Cold

Copy link
Copy Markdown
Collaborator Author

readiness-blocked · Cleanup pending. Stop and confirm owned runs before closing.

中文

readiness-blocked · 清理待完成。先停止并确认自有运行结束,再关闭 PR。

@Klaud-Cold Klaud-Cold closed this Oct 15, 2026
@Klaud-Cold
Klaud-Cold deleted the klaud/auto-72260f70af9bbbbc-555679de269db36f branch October 15, 2026 08:27
@Klaud-Cold

Copy link
Copy Markdown
Collaborator Author

readiness-blocked · Repairs: 0 · Runs: 38682595271
All owned runs ended. PR closed; branch deleted for retry.

中文

readiness-blocked · 修复次数:0 · 运行:38682595271
所有自有运行均已结束。PR 已关闭;分支已删除,可重新尝试。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

1 participant