Skip to content

[Klaud Cold] Update DSR1 FP4 B200 SGLang MTP image to v0.5.21 / [Klaud Cold] 将 DSR1 FP4 B200 SGLang MTP 镜像更新至 v0.5.21 - #3914

Closed
Klaud-Cold wants to merge 1 commit into
mainfrom
klaud/auto-72260f70af9bbbbc-555679de269db36f
Closed

[Klaud Cold] Update DSR1 FP4 B200 SGLang MTP image to v0.5.21 / [Klaud Cold] 将 DSR1 FP4 B200 SGLang MTP 镜像更新至 v0.5.21#3914
Klaud-Cold wants to merge 1 commit into
mainfrom
klaud/auto-72260f70af9bbbbc-555679de269db36f

Conversation

@Klaud-Cold

@Klaud-Cold Klaud-Cold commented Oct 11, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Goal: Update SGLang image from lmsysorg/sglang:v0.5.16-cu130 to lmsysorg/sglang:v0.5.21-cu130@sha256:b1259f3ea3275f66237c498ea388919729018bc9f01c3d638391e06e2cf3f469.
Baseline: 2026-08-06 · lmsysorg/sglang:v0.5.16-cu130
Mean latency · Sources: API 1, API 2, API 3

Point Total tok/s/GPU ↑ Output tok/s/GPU ↑ TTFT ms ↓ TPOT ms ↓
8k/1k c1 TP4 EP1 1ecad2 664.03 74.17 205.09 3.16
8k/1k c2 TP4 EP1 b79296 1,120.85 125.76 250.4 3.59
8k/1k c4 TP4 EP1 d56e03 1,649.89 183.51 296.12 4.98
8k/1k c8 TP4 EP1 298215 2,321.26 260.92 412.66 7
8k/1k c16 TP4 EP1 c2ec41 2,940.82 325.82 826.07 11.1
8k/1k c32 TP4 EP1 fb2b10 3,839.76 429.34 1,288.53 16.88
8k/1k c64 TP4 EP4 6b9bd7 5,786.09 641.84 1,462.11 22.72
8k/1k c128 TP4 EP4 c02deb 7,290.73 807.46 1,991.22 36.79
8k/1k c256 TP4 EP4 98a050 8,783.42 976.13 6,426.93 57.6
Eval Score ↑ Samples
gsm8k/em_strict · c32 95.75% 1,319
gsm8k/em_strict · c128 95.83% 1,319
gsm8k/em_strict · c256 94.92% 1,319
中文

**目标：**将 SGLang 镜像从 lmsysorg/sglang:v0.5.16-cu130 更新为 lmsysorg/sglang:v0.5.21-cu130@sha256:b1259f3ea3275f66237c498ea388919729018bc9f01c3d638391e06e2cf3f469。
**基线：**2026-08-06 · lmsysorg/sglang:v0.5.16-cu130
平均延迟 · 来源： API 1, API 2, API 3；数值及异常说明见上表。

@github-actions @claude
chore(dsr1): update B200 FP4 SGLang MTP image to v0.5.21-cu130
6f2b0a8 
Pin lmsysorg/sglang:v0.5.21-cu130 by digest and replace the removed
--cuda-graph-max-bs and --disable-piecewise-cuda-graph aliases with their
v0.5.16-equivalent --cuda-graph-max-bs-decode and
--cuda-graph-backend-prefill=disabled flags.

将 B200 FP4 SGLang MTP 镜像更新至 v0.5.21-cu130，并按摘要固定；将已移除的
--cuda-graph-max-bs 和 --disable-piecewise-cuda-graph 别名替换为 v0.5.16 中
等价的 --cuda-graph-max-bs-decode 和 --cuda-graph-backend-prefill=disabled。

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Oct 11, 2026

Copy link
Copy Markdown
Contributor

Thanks for the contribution!

  • Review: If this PR changes files owned by someone other than a repository admin or @SemiAnalysisAI/core, ask one eligible CODEOWNER to complete the latest PR_REVIEW_CHECKLIST.md before contacting a core maintainer on Slack. Follow the template exactly, including As a PR reviewer and CODEOWNER, I have reviewed this and have, so sign-off verification triggers.
  • PR verification: Sweeps run only when the PR appends an inferencex-e2e/perf-changelog.yaml entry and carries exactly one primary label: full-sweep-fail-fast (strongly recommended; canary plus per-matrix fail-fast), full-sweep-enabled (canary; matrix jobs continue after a failure), or non-canary-full-sweep-enabled (no canary or fail-fast). The modifiers all-evals, evals-only, and agentx-fast require a primary label. On fork PRs, a maintainer applies the label. See sweep labels and reuse.
  • After merging: PR authors must ensure all GitHub Actions jobs pass. Transient failures often pass on rerun; see how to rerun failed jobs.
中文

感谢你的贡献！

  • **审阅：**如果 PR 修改的文件归属于仓库管理员及 @SemiAnalysisAI/core 之外的 CODEOWNER，请先联系一位有资格的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，再通过 Slack 联系核心维护者。必须严格遵循模板，并保留 As a PR reviewer and CODEOWNER, I have reviewed this and have，才能触发签核验证。
  • **PR 验证：**只有当 PR 在 inferencex-e2e/perf-changelog.yaml 末尾追加了条目，并且恰好带有一个主标签时，才会运行扫描：full-sweep-fail-fast（强烈推荐；canary 加逐矩阵 fail-fast）、full-sweep-enabled（有 canary；矩阵任务在失败后继续运行）或 non-canary-full-sweep-enabled（无 canary，也无 fail-fast）。修饰标签 all-evals、evals-only 和 agentx-fast 必须与主标签一起使用。fork PR 的标签由维护者添加。参见扫描标签与复用。
  • **合并后：**PR 作者必须确保所有 GitHub Actions 任务通过。临时性失败通常可以通过重新运行恢复；参见重新运行失败任务的说明。

@Klaud-Cold

Klaud-Cold commented Oct 11, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator Author

Initial attempt · Failed · Run 38141134282 / attempt 1 · 2026-10-11 13:08 UTC
lmsysorg/sglang:v0.5.21-cu130@sha256:b1259f3ea3275f66237c498ea388919729018bc9f01c3d638391e06e2cf3f469 · c801c9f2dbbd · Mean latency
Change: Pin lmsysorg/sglang:v0.5.21-cu130 by digest and swap aliases removed after v0.5.16 for their v0.5.16 targets (--cuda-graph-max-bs-decode, --cuda-graph-backend-prefill=disabled; measured ref 6f2b0a8b); c64 and the c128 eval failed before engine start with Model path does not exist: /scratch/models/DeepSeek-R1-0528-NVFP4-v2, and fail-fast cancelled c1/c32.

Point Output tok/s/GPU ↑ TTFT ms ↓ TPOT ms ↓
8k/1k c1 TP4 EP1 1ecad2 N/A N/A N/A
8k/1k c64 TP4 EP4 6b9bd7 N/A N/A N/A

Note: 8k/1k c1 TP4 EP1 1ecad2: cancelled.
Note: All rows: request errors unavailable.
Note: All rows: Δ N/A: point unavailable.
Note: 8k/1k c64 TP4 EP4 6b9bd7: failed.

Eval Score ↑ Samples
gsm8k/em_strict · c32 N/A 1,319/N/A (old/new)
gsm8k/em_strict · c128 N/A 1,319/N/A (old/new)

Note: gsm8k/em_strict · c32: unavailable.
Note: All rows: Δ N/A: eval score unavailable.
Note: gsm8k/em_strict · c128: failed.

Next: Stop as readiness-blocked: DeepSeek-R1-0528-NVFP4-v2 weights need staging under /scratch/models on b200-nscale before this family can be measured.

中文

初次尝试 · 失败 · Run 38141134282 / attempt 1 · 2026-10-11 13:08 UTC
lmsysorg/sglang:v0.5.21-cu130@sha256:b1259f3ea3275f66237c498ea388919729018bc9f01c3d638391e06e2cf3f469 · c801c9f2dbbd · 平均延迟
**变更：**按摘要固定 lmsysorg/sglang:v0.5.21-cu130，并将 v0.5.16 之后移除的别名替换为其 v0.5.16 目标（--cuda-graph-max-bs-decode、--cuda-graph-backend-prefill=disabled；测量提交 6f2b0a8b）；c64 与 c128 评测在引擎启动前因 Model path does not exist: /scratch/models/DeepSeek-R1-0528-NVFP4-v2 失败，c1/c32 被 fail-fast 取消。实测数值及异常说明见上表。
**下一步：**以 readiness-blocked 停止：需先在 b200-nscale 的 /scratch/models 下暂存 DeepSeek-R1-0528-NVFP4-v2 权重，才能测量该配置。

@Klaud-Cold

Klaud-Cold commented Oct 11, 2026

Copy link
Copy Markdown
Collaborator Author

readiness-blocked · Cleanup pending. Stop and confirm owned runs before closing.

中文

readiness-blocked · 清理待完成。先停止并确认自有运行结束，再关闭 PR。

@Klaud-Cold Klaud-Cold closed this Oct 11, 2026
@Klaud-Cold
Klaud-Cold deleted the klaud/auto-72260f70af9bbbbc-555679de269db36f branch October 11, 2026 13:09
@Klaud-Cold

Klaud-Cold commented Oct 11, 2026

Copy link
Copy Markdown
Collaborator Author

readiness-blocked · Repairs: 0 · Runs: 38141134282
All owned runs ended. PR closed; branch deleted for retry.

中文

readiness-blocked · 修复次数：0 · 运行：38141134282
所有自有运行均已结束。PR 已关闭；分支已删除，可重新尝试。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Ankur-singh Ankur-singh Awaiting requested review from Ankur-singh Ankur-singh will be requested when the pull request is marked ready for review Ankur-singh is a code owner
@kedarpotdar-nv kedarpotdar-nv Awaiting requested review from kedarpotdar-nv kedarpotdar-nv will be requested when the pull request is marked ready for review kedarpotdar-nv is a code owner
@wzhao18 wzhao18 Awaiting requested review from wzhao18 wzhao18 will be requested when the pull request is marked ready for review wzhao18 is a code owner
@nvpohanh nvpohanh Awaiting requested review from nvpohanh nvpohanh will be requested when the pull request is marked ready for review nvpohanh is a code owner
@RohitNagraj RohitNagraj Awaiting requested review from RohitNagraj RohitNagraj will be requested when the pull request is marked ready for review RohitNagraj is a code owner

Assignees

No one assigned

Labels

None yet

Projects

InferenceX Board
Status: Done

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@Klaud-Cold