Skip to content

[Klaud Cold] Update qwen3.5-fp4-gb300-dynamo-sglang-agentic-pp-pareto SGLang image to v0.5.21-cu130 / [Klaud Cold] 将 qwen3.5-fp4-gb300-dynamo-sglang-agentic-pp-pareto 的 SGLang 镜像更新至 v0.5.21-cu130 - #4082

Closed
Klaud-Cold wants to merge 1 commit into
mainfrom
klaud/auto-1c3468b715193aac-cbc2302e7c3ce1dc
Closed

Klaud-Cold wants to merge 1 commit into
mainfrom
klaud/auto-1c3468b715193aac-cbc2302e7c3ce1dc

Conversation

@Klaud-Cold

@Klaud-Cold Klaud-Cold commented Oct 15, 2026 •

Copy link
Copy Markdown
Collaborator

Goal: Update SGLang image from lmsysorg/sglang:nightly-dev-cu13-20260831-bb5e6198 to lmsysorg/sglang:v0.5.21-cu130@sha256:b1259f3ea3275f66237c498ea388919729018bc9f01c3d638391e06e2cf3f469.
Baseline: 2026-09-09 · lmsysorg/sglang:nightly-dev-cu13-20260831-bb5e6198
Mean latency · Sources: API 1, API 2, API 3

Point Total tok/s/GPU ↑ Output tok/s/GPU ↑ TTFT ms ↓ TPOT ms ↓
AgentX c565 PTP1x3/DTP4x4 5ac19c 87,274.51 836.15 4,201.16 6.16
AgentX c704 PTP1x3/DTP4x2 f8ec59 131,367.59 1,266.3 5,057.49 11.01
Eval Score ↑ Samples
gsm8k/em_strict · c565 97.35% 1,319
gsm8k/em_strict · c704 97.73% 1,319
中文

**目标:**将 SGLang 镜像从 lmsysorg/sglang:nightly-dev-cu13-20260831-bb5e6198 更新为 lmsysorg/sglang:v0.5.21-cu130@sha256:b1259f3ea3275f66237c498ea388919729018bc9f01c3d638391e06e2cf3f469。
**基线:**2026-09-09 · lmsysorg/sglang:nightly-dev-cu13-20260831-bb5e6198
平均延迟 · 来源: API 1, API 2, API 3;数值及异常说明见上表。

Replace the 0831 nightly with the v0.5.21-cu130 release pinned by digest.
SGLang 0.5.21 drops the deprecated --cuda-graph-max-bs and
--mamba-scheduler-strategy aliases and the SGLANG_DISABLE_TP_MEMORY_INBALANCE_CHECK
forwarding, so use their canonical spellings. PP + MTP on disaggregated prefill
passes validation natively, so the PYTHONOPTIMIZE workaround is removed.

将 GB300 FP4 PP-pareto 配置的 SGLang 镜像从 0831 nightly 更新为按 digest 固定的
v0.5.21-cu130 正式版。SGLang 0.5.21 移除了 --cuda-graph-max-bs、
--mamba-scheduler-strategy 弃用别名以及 SGLANG_DISABLE_TP_MEMORY_INBALANCE_CHECK
的转发,因此改用规范名称。分离式 prefill 上的 PP + MTP 已原生通过校验,
故移除 PYTHONOPTIMIZE 变通设置。

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

Thanks for the contribution!

  • Review: If this PR changes files owned by someone other than a repository admin or @SemiAnalysisAI/core, ask one eligible CODEOWNER to complete the latest PR_REVIEW_CHECKLIST.md before contacting a core maintainer on Slack. Follow the template exactly, including As a PR reviewer and CODEOWNER, I have reviewed this and have, so sign-off verification triggers.
  • PR verification: Sweeps run only when the PR appends an inferencex-e2e/perf-changelog.yaml entry and carries exactly one primary label: full-sweep-fail-fast (strongly recommended; canary plus per-matrix fail-fast), full-sweep-enabled (canary; matrix jobs continue after a failure), or non-canary-full-sweep-enabled (no canary or fail-fast). The modifiers all-evals, evals-only, and agentx-fast require a primary label. On fork PRs, a maintainer applies the label. See sweep labels and reuse.
  • After merging: PR authors must ensure all GitHub Actions jobs pass. Transient failures often pass on rerun; see how to rerun failed jobs.
中文

感谢你的贡献!

  • **审阅:**如果 PR 修改的文件归属于仓库管理员及 @SemiAnalysisAI/core 之外的 CODEOWNER,请先联系一位有资格的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md,再通过 Slack 联系核心维护者。必须严格遵循模板,并保留 As a PR reviewer and CODEOWNER, I have reviewed this and have,才能触发签核验证。
  • **PR 验证:**只有当 PR 在 inferencex-e2e/perf-changelog.yaml 末尾追加了条目,并且恰好带有一个主标签时,才会运行扫描:full-sweep-fail-fast(强烈推荐;canary 加逐矩阵 fail-fast)、full-sweep-enabled(有 canary;矩阵任务在失败后继续运行)或 non-canary-full-sweep-enabled(无 canary,也无 fail-fast)。修饰标签 all-evals、evals-only 和 agentx-fast 必须与主标签一起使用。fork PR 的标签由维护者添加。参见扫描标签与复用。
  • **合并后:**PR 作者必须确保所有 GitHub Actions 任务通过。临时性失败通常可以通过重新运行恢复;参见重新运行失败任务的说明。

@Klaud-Cold

Klaud-Cold commented Oct 15, 2026 •

Copy link
Copy Markdown
Collaborator Author

Initial attempt · Queued · Run 38663648843 / attempt 1 · 2026-10-15 06:45 UTC
lmsysorg/sglang:v0.5.21-cu130@sha256:b1259f3ea3275f66237c498ea388919729018bc9f01c3d638391e06e2cf3f469 · c2aa0c36b1af
Change: SGLang bb5e6198 → v0.5.21 (e00930c5, 1591 ahead/0 behind; FlashInfer 0.6.18, sglang-kernel 0.4.7) removed deprecated aliases, so decode uses --cuda-graph-max-bs-decode and --mamba-radix-cache-strategy, both roles use SGLANG_ENABLE_TP_MEMORY_INBALANCE_CHECK=0, and prefill drops PYTHONOPTIMIZE=1 because check_pipeline_parallel_compat now allows PP + MTP on disaggregated prefill; Dynamo 61c37bc is kept though upstream only certified 0.5.21 in 5c8b22b3.

Next: All four smoke jobs remain queued on gb300-nv after more than five hours behind higher-priority work; keep waiting without cancelling, then diagnose the first server error if any job fails.

中文

初次尝试 · 排队中 · Run 38663648843 / attempt 1 · 2026-10-15 06:45 UTC
lmsysorg/sglang:v0.5.21-cu130@sha256:b1259f3ea3275f66237c498ea388919729018bc9f01c3d638391e06e2cf3f469 · c2aa0c36b1af
**变更:**SGLang bb5e6198 → v0.5.21(e00930c5,领先 1591、落后 0;FlashInfer 0.6.18,sglang-kernel 0.4.7)移除了弃用别名,因此 decode 改用 --cuda-graph-max-bs-decode 与 --mamba-radix-cache-strategy,两个角色改用 SGLANG_ENABLE_TP_MEMORY_INBALANCE_CHECK=0;由于 check_pipeline_parallel_compat 已允许分离式 prefill 上的 PP + MTP,prefill 移除 PYTHONOPTIMIZE=1;Dynamo 保持 61c37bc,但上游直到 5c8b22b3 才适配 0.5.21。实测数值及异常说明见上表。
**下一步:**四个冒烟任务在 gb300-nv 上排在更高优先级任务之后已超过五小时;继续等待且不取消,若有任务失败则诊断第一个服务端错误。

@Klaud-Cold

Copy link
Copy Markdown
Collaborator Author

unexpected-error · Cleanup pending. Stop and confirm owned runs before closing.

中文

unexpected-error · 清理待完成。先停止并确认自有运行结束,再关闭 PR。

@Klaud-Cold Klaud-Cold closed this Oct 16, 2026
@Klaud-Cold
Klaud-Cold deleted the klaud/auto-1c3468b715193aac-cbc2302e7c3ce1dc branch October 16, 2026 00:40
@Klaud-Cold

Copy link
Copy Markdown
Collaborator Author

unexpected-error · Repairs: unknown · Runs: 38663648843
All owned runs ended. PR closed; branch deleted for retry.

中文

unexpected-error · 修复次数:unknown · 运行:38663648843
所有自有运行均已结束。PR 已关闭;分支已删除,可重新尝试。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

1 participant