[Klaud Cold] Update Qwen3.5 FP4 MI355X ATOM image to v0.1.7 / [Klaud Cold] 将 Qwen3.5 FP4 MI355X ATOM 镜像更新至 v0.1.7 by Klaud-Cold · Pull Request #3913 · SemiAnalysisAI/InferenceX

Klaud-Cold · 2026-10-11T12:54:11Z

Goal: Update ATOM image from rocm/atom:rocm7.2.2_ubuntu24.04_py3.12_pytorch_release_2.10.0_atom0.1.2.post to rocm/atom-dev:nightly_202610080328.
Baseline: 2026-05-01 · rocm/atom:rocm7.2.2_ubuntu24.04_py3.12_pytorch_release_2.10.0_atom0.1.2.post
Mean latency · Sources: API 1, API 2, API 3

Point	Total tok/s/GPU ↑	Output tok/s/GPU ↑	TTFT ms ↓	TPOT ms ↓
1k/1k c4 TP2 EP1 5c91be	367.05	182.63	120.2	10.51
1k/1k c4 TP4 EP1 6487a8	200.27	99.65	122.59	9.6
1k/1k c8 TP2 EP1 0e2b45	618.69	310.49	173.37	12.46
1k/1k c8 TP4 EP1 bd26ff	363.77	182.56	138.53	10.57
1k/1k c16 TP2 EP1 e1d4bd	1,025.29	509.93	179.42	15.18
1k/1k c16 TP4 EP1 426baa	632.23	314.44	153.67	12.29
1k/1k c32 TP2 EP1 547154	1,555.93	779.2	230.31	19.84
1k/1k c64 TP2 EP1 9bec36	2,284.76	1,142.12	301.87	27.06
1k/1k c128 TP2 EP1 5b8dac	3,160.24	1,578.39	471.34	39.18
1k/1k c256 TP2 EP1 d5617f	4,299.18	2,150.63	759.11	57.59
8k/1k c4 TP2 EP1 cea4fb	1,471.35	163.68	444.59	11.48
8k/1k c4 TP4 EP1 ca8d67	840.38	93.49	277.59	10.15
8k/1k c8 TP2 EP1 2ce007	2,366.53	266.04	561.43	14.19
8k/1k c8 TP4 EP1 3fe619	1,473.53	165.65	354.32	11.47
8k/1k c16 TP2 EP1 bfdd18	3,605.97	399.56	690.07	18.74
8k/1k c16 TP4 EP1 383f0a	2,441.43	270.53	416.06	13.85
8k/1k c32 TP2 EP1 723def	4,932.58	551.6	986.42	27.33
8k/1k c64 TP2 EP1 67a37b	6,280.03	696.71	1,590.94	43.53
8k/1k c128 TP2 EP1 2efa1b	7,566.72	838.13	2,732.19	72.48
8k/1k c256 TP2 EP1 bcab83	8,718.23	968.97	4,950.05	125.44

Note: 1k/1k c4 TP2 EP1 5c91be, 1k/1k c4 TP4 EP1 6487a8, 1k/1k c8 TP2 EP1 0e2b45, 1k/1k c8 TP4 EP1 bd26ff, 1k/1k c16 TP2 EP1 e1d4bd, 1k/1k c16 TP4 EP1 426baa, 1k/1k c32 TP2 EP1 547154, 1k/1k c64 TP2 EP1 9bec36, 1k/1k c128 TP2 EP1 5b8dac, 1k/1k c256 TP2 EP1 d5617f: Single-turn 1k1k retired 2026-07-17 in #2263 after this baseline; excluded from coverage.

Eval	Score ↑	Samples
gsm8k/em_strict · c64	96.97%	1,319
gsm8k/em_strict · c256	96.82%	1,319

中文

**目标：**将 ATOM 镜像从 rocm/atom:rocm7.2.2_ubuntu24.04_py3.12_pytorch_release_2.10.0_atom0.1.2.post 更新为 rocm/atom-dev:nightly_202610080328。
**基线：**2026-05-01 · rocm/atom:rocm7.2.2_ubuntu24.04_py3.12_pytorch_release_2.10.0_atom0.1.2.post
平均延迟 · 来源： API 1, API 2, API 3；数值及异常说明见上表。

Update qwen3.5-fp4-mi355x-atom from atom0.1.2.post to rocm/atom-dev:nightly_202610080328 (ATOM v0.1.7, a526f0d). 将 qwen3.5-fp4-mi355x-atom 的 ATOM 镜像从 atom0.1.2.post 更新为 rocm/atom-dev:nightly_202610080328（ATOM v0.1.7，a526f0d）。 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

github-actions · 2026-10-11T12:54:20Z

Thanks for the contribution!

Review: If this PR changes files owned by someone other than a repository admin or @SemiAnalysisAI/core, ask one eligible CODEOWNER to complete the latest PR_REVIEW_CHECKLIST.md before contacting a core maintainer on Slack. Follow the template exactly, including As a PR reviewer and CODEOWNER, I have reviewed this and have, so sign-off verification triggers.
PR verification: Sweeps run only when the PR appends an inferencex-e2e/perf-changelog.yaml entry and carries exactly one primary label: full-sweep-fail-fast (strongly recommended; canary plus per-matrix fail-fast), full-sweep-enabled (canary; matrix jobs continue after a failure), or non-canary-full-sweep-enabled (no canary or fail-fast). The modifiers all-evals, evals-only, and agentx-fast require a primary label. On fork PRs, a maintainer applies the label. See sweep labels and reuse.
After merging: PR authors must ensure all GitHub Actions jobs pass. Transient failures often pass on rerun; see how to rerun failed jobs.

中文

感谢你的贡献！

**审阅：**如果 PR 修改的文件归属于仓库管理员及 @SemiAnalysisAI/core 之外的 CODEOWNER，请先联系一位有资格的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，再通过 Slack 联系核心维护者。必须严格遵循模板，并保留 As a PR reviewer and CODEOWNER, I have reviewed this and have，才能触发签核验证。
**PR 验证：**只有当 PR 在 inferencex-e2e/perf-changelog.yaml 末尾追加了条目，并且恰好带有一个主标签时，才会运行扫描：full-sweep-fail-fast（强烈推荐；canary 加逐矩阵 fail-fast）、full-sweep-enabled（有 canary；矩阵任务在失败后继续运行）或 non-canary-full-sweep-enabled（无 canary，也无 fail-fast）。修饰标签 all-evals、evals-only 和 agentx-fast 必须与主标签一起使用。fork PR 的标签由维护者添加。参见扫描标签与复用。
**合并后：**PR 作者必须确保所有 GitHub Actions 任务通过。临时性失败通常可以通过重新运行恢复；参见重新运行失败任务的说明。

Klaud-Cold · 2026-10-11T12:55:49Z

Initial attempt · Passed · Run 38141062012 / attempt 1 · 2026-10-11 13:29 UTC
rocm/atom-dev:nightly_202610080328 · c801c9f2dbbd · Mean latency
Change: Measured a5f438b: image-only bump to ATOM v0.1.7 (OCI revision a526f0d, digest sha256:9c3070bb…, https://github.com/ROCm/ATOM/releases/tag/v0.1.7) from an old image built from ATOM main around 2026-04-22 (~f4478c5e, inferred from build time); all srtctl flags are unchanged in arg_utils.py, and new defaults enable prefix caching, chunked prefill and cudagraph sizes up to 512.

Point	Output tok/s/GPU ↑	TTFT ms ↓	TPOT ms ↓
8k/1k c4 TP2 EP1 cea4fb	142.52 (-12.9%)	1,313.7 (+195.5%)	12.16 (+6%)
8k/1k c4 TP4 EP1 ca8d67	78.8 (-15.7%)	1,207.43 (+335%)	10.94 (+7.8%)

Note: All rows: request errors unavailable.

Eval	Score ↑	Samples
gsm8k/em_strict · c64	96.59% (-0.38 pp)	1,319 each

Next: Append the changelog entry and run the final full sweep.

中文

初次尝试 · 已通过 · Run 38141062012 / attempt 1 · 2026-10-11 13:29 UTC
rocm/atom-dev:nightly_202610080328 · c801c9f2dbbd · 平均延迟
**变更：**测量提交 a5f438b：仅将镜像更新至 ATOM v0.1.7（OCI revision a526f0d，digest sha256:9c3070bb…），旧镜像约于 2026-04-22 基于 ATOM main 构建（约 f4478c5e，由构建时间推断）；srtctl 传入的参数在 arg_utils.py 中均未变化，新默认值启用 prefix caching、chunked prefill，并将 cudagraph 尺寸扩展至 512。实测数值及异常说明见上表。
**下一步：**追加 changelog 条目并运行最终完整 sweep。

Append the perf-changelog entry for qwen3.5-fp4-mi355x-atom. 为 qwen3.5-fp4-mi355x-atom 追加 perf-changelog 条目。 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Klaud-Cold · 2026-10-11T13:31:51Z

Final full sweep · Passed · Run 38143456084 / attempt 1 · 2026-10-11 17:33 UTC
rocm/atom-dev:nightly_202610080328 · ac737411f401 · Mean latency
Change: Validate the complete updated-image family at this head.

Point	Output tok/s/GPU ↑	TTFT ms ↓	TPOT ms ↓
8k/1k c4 TP2 EP1 cea4fb	144.5 (-11.7%)	1,231.25 (+176.9%)	12.07 (+5.2%)
8k/1k c4 TP4 EP1 ca8d67	79.35 (-15.1%)	1,192.04 (+329.4%)	10.88 (+7.2%)
8k/1k c8 TP2 EP1 2ce007	244.83 (-8%)	1,002.63 (+78.6%)	14.95 (+5.4%)
8k/1k c8 TP4 EP1 3fe619	127.56 (-23%)	1,076.71 (+203.9%)	14.22 (+24%)
8k/1k c16 TP2 EP1 bfdd18	349.87 (-12.4%)	1,080.27 (+56.5%)	21.25 (+13.4%)
8k/1k c16 TP4 EP1 383f0a	203.35 (-24.8%)	948.79 (+128%)	18.25 (+31.7%)
8k/1k c32 TP2 EP1 723def	479.94 (-13%)	1,198.44 (+21.5%)	31.41 (+14.9%)
8k/1k c64 TP2 EP1 67a37b	679.44 (-2.5%)	1,632.83 (+2.6%)	44.62 (+2.5%)
8k/1k c128 TP2 EP1 2efa1b	843.65 (+0.7%)	2,595.09 (-5%)	72.14 (-0.5%)
8k/1k c256 TP2 EP1 bcab83	1,005.36 (+3.8%)	4,388.92 (-11.3%)	121.31 (-3.3%)

Note: All rows: request errors unavailable.

Eval	Score ↑	Samples
gsm8k/em_strict · c64	97.12% (+0.15 pp)	1,319 each
gsm8k/em_strict · c256	96.97% (+0.15 pp)	1,319 each

Note: 1k/1k c4 TP2 EP1 5c91be, 1k/1k c4 TP4 EP1 6487a8, 1k/1k c8 TP2 EP1 0e2b45, 1k/1k c8 TP4 EP1 bd26ff, 1k/1k c16 TP2 EP1 e1d4bd, 1k/1k c16 TP4 EP1 426baa, 1k/1k c32 TP2 EP1 547154, 1k/1k c64 TP2 EP1 9bec36, 1k/1k c128 TP2 EP1 5b8dac, 1k/1k c256 TP2 EP1 d5617f: Single-turn 1k1k retired 2026-07-17 in #2263 after this baseline; excluded from coverage.

Next: Mark ready for maintainer review.

中文

最终完整 sweep · 已通过 · Run 38143456084 / attempt 1 · 2026-10-11 17:33 UTC
rocm/atom-dev:nightly_202610080328 · ac737411f401 · 平均延迟
**变更：**验证此提交更新镜像后的完整配置族。实测数值及异常说明见上表。
**下一步：**标记为就绪，等待维护者审查。

github-actions · 2026-10-11T17:31:06Z

View unofficial run (performance): https://inferencex.semianalysis.com/inference?unofficialRun=38143456084

View unofficial run (accuracy): https://inferencex.semianalysis.com/evaluation?unofficialRun=38143456084

Klaud-Cold · 2026-10-11T17:33:16Z

validated · Repairs: 0 · Runs: 38141062012, 38143436286, 38143456084
All owned runs ended. Full sweep verified; ready for review.

中文

validated · 修复次数：0 · 运行：38141062012, 38143436286, 38143456084
所有自有运行均已结束。完整 sweep 已验证；已就绪，等待审查。

Klaud-Cold · 2026-10-11T17:33:25Z

/use 38143456084

claude · 2026-10-11T17:33:33Z

Claude finished @Klaud-Cold's task in 25s —— View job

Review

I found no blocking issues. The image tag is identical in amd-master.yaml:387 and the recipe's model.container. The image comes from public rocm/atom-dev. The perf-changelog.yaml entry is added at the end of the file and links to this PR. The mi355x-fp8-mtp recipe still uses the old image. That's expected because it belongs to a different config key.

🟡 WARNING: the changelog entry doesn't mention a performance regression. The final sweep (run 38143456084) is a regression at low and mid concurrency compared with the 2026-05-01 baseline:

c4–c32: output tok/s/GPU drops by 8–25% (TP4 c16: −24.8%) and TTFT rises by 21–329%.
TP4: TPOT also gets worse, by up to +31.7%.
c64–c256: results are flat to slightly better.

The run comments link the change in defaults to v0.1.7, which turns on prefix caching and chunked prefill and allows cudagraph sizes up to 512. That would be consistent with the TTFT increase, but no run has tested it. If this trade-off is intended, say so in the changelog description so the drop on the dashboard has an explanation. If it isn't intended, try turning off the new defaults in the recipe args (for example, disable chunked prefill and prefix caching) before merging.

github-actions · 2026-10-11T17:33:47Z

@Klaud-Cold staged run 38143456084: https://inferencemax-app-git-staging-semianalysisai.vercel.app/inference?i_dates=2026-10-11~r38143456084

This run remains available across future /stage-results requests. Staging the same run ID again updates its staged data. Staging workflow

claude

Code review found no issues

No high-confidence issues detected in this change.

chore(changelog): record Qwen3.5 FP4 MI355X ATOM v0.1.7 bump

ac73741

Append the perf-changelog entry for qwen3.5-fp4-mi355x-atom. 为 qwen3.5-fp4-mi355x-atom 追加 perf-changelog 条目。 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Klaud-Cold added the full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended) label Oct 11, 2026

Klaud-Cold marked this pull request as ready for review October 11, 2026 17:33

Klaud-Cold requested a review from a team October 11, 2026 17:33

Klaud-Cold requested review from 1am9trash, billishyahao, chunfangamd, seungrokj and yctseng0211 as code owners October 11, 2026 17:33

claude Bot reviewed Oct 11, 2026

View reviewed changes

Navigation Menu

Repository navigation

[Klaud Cold] Update Qwen3.5 FP4 MI355X ATOM image to v0.1.7 / [Klaud Cold] 将 Qwen3.5 FP4 MI355X ATOM 镜像更新至 v0.1.7 - #3913

[Klaud Cold] Update Qwen3.5 FP4 MI355X ATOM image to v0.1.7 / [Klaud Cold] 将 Qwen3.5 FP4 MI355X ATOM 镜像更新至 v0.1.7#3913
Klaud-Cold wants to merge 2 commits into
mainfrom
klaud/auto-586320e6b46a61e1-d7d3a8ed94c43625

Klaud-Cold commented Oct 11, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Oct 11, 2026

Uh oh!

Klaud-Cold commented Oct 11, 2026 •

edited

Loading

Uh oh!

Klaud-Cold commented Oct 11, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Oct 11, 2026

Uh oh!

Klaud-Cold commented Oct 11, 2026

Uh oh!

Klaud-Cold commented Oct 11, 2026

Uh oh!

claude Bot commented Oct 11, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Oct 11, 2026 •

edited

Loading

Uh oh!

claude Bot left a comment

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

Klaud-Cold commented Oct 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Oct 11, 2026

Uh oh!

Klaud-Cold commented Oct 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Klaud-Cold commented Oct 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Oct 11, 2026

Uh oh!

Klaud-Cold commented Oct 11, 2026

Uh oh!

Klaud-Cold commented Oct 11, 2026

Uh oh!

claude Bot commented Oct 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review

Uh oh!

github-actions Bot commented Oct 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

claude Bot left a comment

Choose a reason for hiding this comment

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Klaud-Cold commented Oct 11, 2026 •

edited

Loading

Klaud-Cold commented Oct 11, 2026 •

edited

Loading

Klaud-Cold commented Oct 11, 2026 •

edited

Loading

claude Bot commented Oct 11, 2026 •

edited

Loading

github-actions Bot commented Oct 11, 2026 •

edited

Loading