[Klaud Cold] Update DSR1 FP8 H200 TRT-LLM MTP image to 1.4.0rc0 / [Klaud Cold] 将 DSR1 FP8 H200 TRT-LLM MTP 镜像更新至 1.4.0rc0

[Klaud-Cold]

Goal: Update TensorRT-LLM image from nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc14 to nvcr.io/nvidia/tensorrt-llm/release:1.4.0rc0.
Baseline: 2026-05-18 · nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc14
Mean latency · Sources: API 1, API 2

Point	Total tok/s/GPU ↑	Output tok/s/GPU ↑	TTFT ms ↓	TPOT ms ↓
1k/1k c4 TP8 EP8 478e1e	105.32	52.4	222.96	9
1k/1k c8 TP8 EP8 d0bfd3	180.03	90.35	290.74	10.63
1k/1k c16 TP8 EP8 13dc39	277.65	138.09	365.26	13.69
1k/1k c32 TP8 EP8 15f275	436.88	218.79	542.45	17.26
1k/1k c64 TP8 EP8 1a6924	644.8	322.33	849.84	23.28
1k/1k c128 TP8 EP8 1a33ea	1,140.05	569.4	1,999.72	24.65
1k/1k c256 TP8 EP8 d29605	1,779.74	890.3	2,135.67	32.16
8k/1k c4 TP8 EP8 657592	464.14	51.63	472.14	8.96
8k/1k c8 TP8 EP8 589cce	703.77	79.11	575.06	11.63
8k/1k c16 TP8 EP8 98a75a	1,013.01	112.23	742.46	16.55
8k/1k c32 TP8 EP8 393af4	1,415.93	158.32	1,024.19	23.64
8k/1k c64 TP8 EP8 8f79b0	1,835.19	203.57	5,161.38	32.72
8k/1k c128 TP8 EP8 77ee1a	2,426.71	268.76	6,826.01	50.57
8k/1k c256 TP8 EP8 41ac84	2,965.43	329.56	9,890.51	84.71

Note: 1k/1k c4 TP8 EP8 478e1e, 1k/1k c8 TP8 EP8 d0bfd3, 1k/1k c16 TP8 EP8 13dc39, 1k/1k c32 TP8 EP8 15f275, 1k/1k c64 TP8 EP8 1a6924, 1k/1k c128 TP8 EP8 1a33ea, 1k/1k c256 TP8 EP8 d29605: Single-turn 1k1k retired 2026-07-17 in #2263 after this baseline; excluded from coverage.

Eval: N/A

中文

**目标：**将 TensorRT-LLM 镜像从 nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc14 更新至 nvcr.io/nvidia/tensorrt-llm/release:1.4.0rc0。
**基线：**2026-05-18 · nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc14
平均延迟 · 来源： API 1, API 2；数值及异常说明见上表。

[github-actions]

Thanks for the contribution!

• Review: If this PR changes files owned by someone other than a repository admin or @SemiAnalysisAI/core, ask one eligible CODEOWNER to complete the latest PR_REVIEW_CHECKLIST.md before contacting a core maintainer on Slack. Follow the template exactly, including As a PR reviewer and CODEOWNER, I have reviewed this and have, so sign-off verification triggers.
• PR verification: Sweeps run only when the PR appends an inferencex-e2e/perf-changelog.yaml entry and carries exactly one primary label: full-sweep-fail-fast (strongly recommended; canary plus per-matrix fail-fast), full-sweep-enabled (canary; matrix jobs continue after a failure), or non-canary-full-sweep-enabled (no canary or fail-fast). The modifiers all-evals, evals-only, and agentx-fast require a primary label. On fork PRs, a maintainer applies the label. See sweep labels and reuse.
• After merging: PR authors must ensure all GitHub Actions jobs pass. Transient failures often pass on rerun; see how to rerun failed jobs.

中文

感谢你的贡献！

• **审阅：**如果 PR 修改的文件归属于仓库管理员及 @SemiAnalysisAI/core 之外的 CODEOWNER，请先联系一位有资格的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，再通过 Slack 联系核心维护者。必须严格遵循模板，并保留 As a PR reviewer and CODEOWNER, I have reviewed this and have，才能触发签核验证。
• **PR 验证：**只有当 PR 在 inferencex-e2e/perf-changelog.yaml 末尾追加了条目，并且恰好带有一个主标签时，才会运行扫描：full-sweep-fail-fast（强烈推荐；canary 加逐矩阵 fail-fast）、full-sweep-enabled（有 canary；矩阵任务在失败后继续运行）或 non-canary-full-sweep-enabled（无 canary，也无 fail-fast）。修饰标签 all-evals、evals-only 和 agentx-fast 必须与主标签一起使用。fork PR 的标签由维护者添加。参见扫描标签与复用。
• **合并后：**PR 作者必须确保所有 GitHub Actions 任务通过。临时性失败通常可以通过重新运行恢复；参见重新运行失败任务的说明。

[Klaud-Cold]

Initial attempt · Passed · Run 38820255753 / attempt 1 · 2026-10-15 23:24 UTC
nvcr.io/nvidia/tensorrt-llm/release:1.4.0rc0 · 479f104ca1fc · 8k/1k · TP8/EP8 · Mean latency
Change: Bump the master image from 1.3.0rc14 to 1.4.0rc0 (index digest sha256:cab8318458b3dd92cc74c53709a30001d90d38928e5bfeb9fb84280ce00369f5); the recipe is unchanged because every used llm_args field and trtllm-serve flag still exists at v1.4.0rc0.

Concurrency	Output tok/s/GPU ↑	TTFT ms ↓	TPOT ms ↓
4	58.57 (+13.5%)	417.77 (-11.5%)	7.83 (-12.6%)
64	222.81 (+9.4%)	4,375.2 (-15.2%)	30.42 (-7%)

Note: All rows: request errors unavailable.

Eval	Score ↑	Samples
gsm8k/em_strict · c32	95.6% (N/A)	N/A/1,319 (old/new)
gsm8k/em_strict · c128	96.06% (N/A)	N/A/1,319 (old/new)

Note: All rows: Δ N/A: no matched eval baseline.

Next: Append the perf-changelog entry, verify the final matrix and start the full fail-fast sweep.

中文

初次尝试 · 已通过 · Run 38820255753 / attempt 1 · 2026-10-15 23:24 UTC
nvcr.io/nvidia/tensorrt-llm/release:1.4.0rc0 · 479f104ca1fc · 8k/1k · TP8/EP8 · 平均延迟
**变更：**将 master 镜像从 1.3.0rc14 更新至 1.4.0rc0（index digest sha256:cab8318458b3dd92cc74c53709a30001d90d38928e5bfeb9fb84280ce00369f5）；配方未改动，因为所用的全部 llm_args 字段和 trtllm-serve 参数在 v1.4.0rc0 中依然存在。实测数值及异常说明见上表。
**下一步：**追加 perf-changelog 条目，校验最终矩阵并启动完整的 fail-fast sweep。

[Klaud-Cold]

Final full sweep · Running · Run 38823557912 / attempt 1 · 2026-10-16 02:28 UTC
nvcr.io/nvidia/tensorrt-llm/release:1.4.0rc0 · 8008b5dd977c · 8k/1k · TP8/EP8 · Mean latency
Change: Append the perf-changelog entry and run the full fail-fast sweep on the unchanged 1.4.0rc0 recipe.

Concurrency	Output tok/s/GPU ↑	TTFT ms ↓	TPOT ms ↓
4	59.21 (+14.7%)	440.65 (-6.7%)	7.81 (-12.9%)
8	84.25 (+6.5%)	508.32 (-11.6%)	10.97 (-5.7%)
16	119.92 (+6.8%)	642.55 (-13.5%)	15.62 (-5.6%)
32	168.93 (+6.7%)	939.06 (-8.3%)	22.11 (-6.5%)
64	221.6 (+8.9%)	4,447.7 (-13.8%)	30.35 (-7.2%)
128	302.73 (+12.6%)	5,983.3 (-12.3%)	45.26 (-10.5%)
256	372.96 (+13.2%)	8,123.27 (-17.9%)	75 (-11.5%)

Note: All rows: request errors unavailable.

Note: 1k/1k c4 TP8 EP8 478e1e, 1k/1k c8 TP8 EP8 d0bfd3, 1k/1k c16 TP8 EP8 13dc39, 1k/1k c32 TP8 EP8 15f275, 1k/1k c64 TP8 EP8 1a6924, 1k/1k c128 TP8 EP8 1a33ea, 1k/1k c256 TP8 EP8 d29605: Single-turn 1k1k retired 2026-07-17 in #2263 after this baseline; excluded from coverage.

Next: All 7 benchmark points and the c32/c128 evals passed; wait for the queued c256 eval to get an h200 runner.

中文

最终完整 sweep · 运行中 · Run 38823557912 / attempt 1 · 2026-10-16 02:28 UTC
nvcr.io/nvidia/tensorrt-llm/release:1.4.0rc0 · 8008b5dd977c · 8k/1k · TP8/EP8 · 平均延迟
**变更：**追加 perf-changelog 条目，并在未改动的 1.4.0rc0 配方上运行完整 fail-fast sweep。实测数值及异常说明见上表。
**下一步：**全部 7 个基准点以及 c32/c128 评测均已通过；等待排队中的 c256 评测获得 h200 runner。

[github-actions]

View unofficial run (performance): https://inferencex.semianalysis.com/inference?unofficialRun=38847132944

View unofficial run (accuracy): https://inferencex.semianalysis.com/evaluation?unofficialRun=38847132944

[Klaud-Cold]

Infrastructure retry 1 · Passed · Run 38823557912 / attempt 2 · 2026-10-16 03:37 UTC
nvcr.io/nvidia/tensorrt-llm/release:1.4.0rc0 · 8008b5dd977c · 8k/1k · TP8/EP8 · Mean latency
Change: Rerun only the c256 eval, which the scheduler cancelled while it was still queued above the load limit; same head and image.

Concurrency	Output tok/s/GPU ↑	TTFT ms ↓	TPOT ms ↓
4	59.21 (+14.7%)	440.65 (-6.7%)	7.81 (-12.9%)
8	84.25 (+6.5%)	508.32 (-11.6%)	10.97 (-5.7%)
16	119.92 (+6.8%)	642.55 (-13.5%)	15.62 (-5.6%)
32	168.93 (+6.7%)	939.06 (-8.3%)	22.11 (-6.5%)
64	221.6 (+8.9%)	4,447.7 (-13.8%)	30.35 (-7.2%)
128	302.73 (+12.6%)	5,983.3 (-12.3%)	45.26 (-10.5%)
256	372.96 (+13.2%)	8,123.27 (-17.9%)	75 (-11.5%)

Note: All rows: request errors unavailable.

Next: Run finish to verify the complete artifacts and mark the PR ready.

中文

基础设施重试 1 · 已通过 · Run 38823557912 / attempt 2 · 2026-10-16 03:37 UTC
nvcr.io/nvidia/tensorrt-llm/release:1.4.0rc0 · 8008b5dd977c · 8k/1k · TP8/EP8 · 平均延迟
**变更：**仅重跑 c256 评测：该任务在排队期间因调度器负载超限被取消；提交和镜像均不变。实测数值及异常说明见上表。
**下一步：**运行 finish 核验完整产物并将 PR 标记为 ready。

[Klaud-Cold]

Repair 1/5 · Deferred · Run 38847132944 / attempt 1 · 2026-10-16 04:09 UTC
nvcr.io/nvidia/tensorrt-llm/release:1.4.0rc0 · cd531bd9960b · 8k/1k · TP8/EP8 · Mean latency
Change: Spell the master image as nvcr.io/nvidia/tensorrt-llm/release:1.4.0rc0 (same squash file at runtime) because the final report rejects the nvcr.io# spelling, and merge main to clear a perf-changelog conflict.

Concurrency	Output tok/s/GPU ↑	TTFT ms ↓	TPOT ms ↓
4	N/A	N/A	N/A
8	N/A	N/A	N/A
16	N/A	N/A	N/A
32	N/A	N/A	N/A
64	N/A	N/A	N/A
128	N/A	N/A	N/A
256	N/A	N/A	N/A

Note: All rows: cancelled.
Note: All rows: request errors unavailable.
Note: All rows: Δ N/A: point unavailable.

Next: Defer: this sweep was cancelled while queued and h200-dgxc is above the 200% load limit; a later wave may retry the 1.4.0rc0 update.

中文

修复 1/5 · 已延期 · Run 38847132944 / attempt 1 · 2026-10-16 04:09 UTC
nvcr.io/nvidia/tensorrt-llm/release:1.4.0rc0 · cd531bd9960b · 8k/1k · TP8/EP8 · 平均延迟
**变更：**将 master 镜像写为 nvcr.io/nvidia/tensorrt-llm/release:1.4.0rc0（运行时使用同一 squash 文件），因为最终报告不接受 nvcr.io# 写法；同时合并 main 以解决 perf-changelog 冲突。实测数值及异常说明见上表。
**下一步：**延期：该 sweep 在排队时被取消，且 h200-dgxc 负载超过 200% 上限；后续批次可重新尝试 1.4.0rc0 更新。

[Oseltamivir]

@Klaud-Cold pause on this to priortize cluster usage for muse spark open

[Klaud-Cold]

Claude finished @Oseltamivir's task in 43s —— View job

---

Paused for Muse Spark Open cluster priority

• Find active runs on klaud/auto-5a3ab4e2462f0912-1e6fa8d804bf7b88
• Cancel the in-progress sweep (Repair 1/5): Run 38847132944 is now cancelled, which frees the H200 runners.
• Check that nothing else is still queued or running on this branch. No other run is active.

I didn't push any commits, and the branch is still at cd531bd9960b.

What's done: the earlier full sweep (Run 38823557912) passed on the old head 8008b5dd977c. All 7 8k/1k points improved (output tok/s/GPU up 6.5% to 14.7%), and all 3 evals passed after the c256 eval was rerun.

What's left: the repair sweep on cd531bd9960b hasn't produced results. That head changes how the image name is written (nvcr.io# to nvcr.io/) and merges main. To resume when the cluster is free, reopen the PR and re-trigger the sweep. The PR already has the full-sweep-fail-fast label. Then run finish once all 7 points and 3 evals are green.

[Klaud-Cold]

capacity-deferred · Cleanup pending. Stop and confirm owned runs before closing.

中文

capacity-deferred · 清理待完成。先停止并确认自有运行结束，再关闭 PR。

[Klaud-Cold]

capacity-deferred · Repairs: 1 · Runs: 38820255753, 38823518655, 38823557912, 38847076908, 38847132944, 38849125610
All owned runs ended. PR closed; branch deleted for retry.

中文

capacity-deferred · 修复次数：1 · 运行：38820255753, 38823518655, 38823557912, 38847076908, 38847132944, 38849125610
所有自有运行均已结束。PR 已关闭；分支已删除，可重新尝试。