Skip to content

[Klaud Cold] Update DSR1 FP8 MI355X ATOM MTP image to v0.1.7 / [Klaud Cold] 将 DSR1 FP8 MI355X ATOM MTP 镜像更新至 v0.1.7 - #3915

Open
Klaud-Cold wants to merge 2 commits into
mainfrom
klaud/auto-e60aea4ed67834be-f8526e08d2a1a71a
Open

Klaud-Cold wants to merge 2 commits into
mainfrom
klaud/auto-e60aea4ed67834be-f8526e08d2a1a71a

Conversation

@Klaud-Cold

@Klaud-Cold Klaud-Cold commented Oct 11, 2026 •

Copy link
Copy Markdown
Collaborator

Goal: Update ATOM image from rocm/atom:rocm7.2.4_ubuntu24.04_py3.12_pytorch_release_2.10.0_atom0.1.3 to rocm/atom-dev:nightly_202610080328@sha256:9c3070bb7d2600d2e1bcfd03cf5f71e4dc0941f3e1d20d3a5845056aae2334eb.
Baseline: 2026-06-02 · rocm/atom:rocm7.2.4_ubuntu24.04_py3.12_pytorch_release_2.10.0_atom0.1.3
Mean latency · Sources: API 1, API 2, API 3

Point Total tok/s/GPU ↑ Output tok/s/GPU ↑ TTFT ms ↓ TPOT ms ↓
1k/1k c4 TP8 EP1 c24d64 167.25 83.27 102.79 5.72
1k/1k c8 TP8 EP1 f84cc2 248.12 124.58 116.26 7.66
1k/1k c16 TP8 EP1 93b5ce 411.03 204.54 143.33 9.38
1k/1k c32 TP8 EP1 1a341c 647.61 324.52 183.9 11.74
1k/1k c64 TP8 EP1 4df1a8 876.82 438.56 251.73 17.47
1k/1k c128 TP8 EP1 7853b7 1,309.11 654.22 357.57 23.36
1k/1k c256 TP8 EP1 1147f0 1,820.45 911.22 555.67 33.64
1k/1k c512 TP8 EP1 33d209 50.53 25.28 1,038.26 59.18
8k/1k c4 TP8 EP1 ba4138 709.69 79.02 322.79 5.79
8k/1k c8 TP8 EP1 c367ee 991.89 111.63 402.04 8.24
8k/1k c16 TP8 EP1 e01752 1,405.57 155.94 488.73 11.95
8k/1k c32 TP8 EP1 ce6ec3 1,994.1 223.24 716.62 16.68
8k/1k c64 TP8 EP1 45e1ee 2,503.9 278.08 1,121.99 27.05
8k/1k c128 TP8 EP1 2fb877 3,018.75 334.7 1,958.48 44.97
8k/1k c256 TP8 EP1 c8b94c 3,395.38 377.77 3,593.96 79.92

Note: 1k/1k c4 TP8 EP1 c24d64, 1k/1k c8 TP8 EP1 f84cc2, 1k/1k c16 TP8 EP1 93b5ce, 1k/1k c32 TP8 EP1 1a341c, 1k/1k c64 TP8 EP1 4df1a8, 1k/1k c128 TP8 EP1 7853b7, 1k/1k c256 TP8 EP1 1147f0, 1k/1k c512 TP8 EP1 33d209: Single-turn 1k1k retired 2026-07-17 in #2263 after this baseline; excluded from coverage.

Eval Score ↑ Samples
gsm8k/em_strict · c64 95.68% 1,319
gsm8k/em_strict · c256 96.06% 1,319
中文

**目标:**将 ATOM 镜像从 rocm/atom:rocm7.2.4_ubuntu24.04_py3.12_pytorch_release_2.10.0_atom0.1.3 更新为 rocm/atom-dev:nightly_202610080328@sha256:9c3070bb7d2600d2e1bcfd03cf5f71e4dc0941f3e1d20d3a5845056aae2334eb。
**基线:**2026-06-02 · rocm/atom:rocm7.2.4_ubuntu24.04_py3.12_pytorch_release_2.10.0_atom0.1.3
平均延迟 · 来源: API 1, API 2, API 3;数值及异常说明见上表。

Pin rocm/atom-dev:nightly_202610080328 by digest; its OCI labels
identify ATOM 0.1.7 at release commit a526f0d.

将 DSR1 FP8 MI355X ATOM MTP 镜像更新为 v0.1.7 发布构建,并按摘要固定。

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

Thanks for the contribution!

  • Review: If this PR changes files owned by someone other than a repository admin or @SemiAnalysisAI/core, ask one eligible CODEOWNER to complete the latest PR_REVIEW_CHECKLIST.md before contacting a core maintainer on Slack. Follow the template exactly, including As a PR reviewer and CODEOWNER, I have reviewed this and have, so sign-off verification triggers.
  • PR verification: Sweeps run only when the PR appends an inferencex-e2e/perf-changelog.yaml entry and carries exactly one primary label: full-sweep-fail-fast (strongly recommended; canary plus per-matrix fail-fast), full-sweep-enabled (canary; matrix jobs continue after a failure), or non-canary-full-sweep-enabled (no canary or fail-fast). The modifiers all-evals, evals-only, and agentx-fast require a primary label. On fork PRs, a maintainer applies the label. See sweep labels and reuse.
  • After merging: PR authors must ensure all GitHub Actions jobs pass. Transient failures often pass on rerun; see how to rerun failed jobs.
中文

感谢你的贡献!

  • **审阅:**如果 PR 修改的文件归属于仓库管理员及 @SemiAnalysisAI/core 之外的 CODEOWNER,请先联系一位有资格的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md,再通过 Slack 联系核心维护者。必须严格遵循模板,并保留 As a PR reviewer and CODEOWNER, I have reviewed this and have,才能触发签核验证。
  • **PR 验证:**只有当 PR 在 inferencex-e2e/perf-changelog.yaml 末尾追加了条目,并且恰好带有一个主标签时,才会运行扫描:full-sweep-fail-fast(强烈推荐;canary 加逐矩阵 fail-fast)、full-sweep-enabled(有 canary;矩阵任务在失败后继续运行)或 non-canary-full-sweep-enabled(无 canary,也无 fail-fast)。修饰标签 all-evals、evals-only 和 agentx-fast 必须与主标签一起使用。fork PR 的标签由维护者添加。参见扫描标签与复用。
  • **合并后:**PR 作者必须确保所有 GitHub Actions 任务通过。临时性失败通常可以通过重新运行恢复;参见重新运行失败任务的说明。

@Klaud-Cold

Klaud-Cold commented Oct 11, 2026 •

Copy link
Copy Markdown
Collaborator Author

Initial attempt · Passed · Run 38141170149 / attempt 1 · 2026-10-11 13:17 UTC
rocm/atom-dev:nightly_202610080328@sha256:9c3070bb7d2600d2e1bcfd03cf5f71e4dc0941f3e1d20d3a5845056aae2334eb · c801c9f2dbbd · 8k/1k · TP8/EP1 · Mean latency
Change: Measure ref eb9f066: Pin the ATOM v0.1.7 release image (OCI labels: version 0.1.7, revision a526f0d, ROCm 7.2.4, torch 2.10.0); the old atom0.1.3 tag was built from ATOM main near v0.1.3 without a revision label, and the used --method, --num-speculative-tokens, --kv_cache_dtype, --no-enable_prefix_caching, -tp and --server-port parsers and defaults are unchanged in arg_utils.py, so no flag changes.

Concurrency Output tok/s/GPU ↑ TTFT ms ↓ TPOT ms ↓
4 72.21 (-8.6%) 316.92 (-1.8%) 6.26 (+8.1%)
Eval Score ↑ Samples
gsm8k/em_strict · c64 95.98% (+0.30 pp) 1,319 each

Next: Append the changelog entry and run the final full sweep.

中文

初次尝试 · 已通过 · Run 38141170149 / attempt 1 · 2026-10-11 13:17 UTC
rocm/atom-dev:nightly_202610080328@sha256:9c3070bb7d2600d2e1bcfd03cf5f71e4dc0941f3e1d20d3a5845056aae2334eb · c801c9f2dbbd · 8k/1k · TP8/EP1 · 平均延迟
**变更:**测量提交 eb9f066:固定使用 ATOM v0.1.7 发布镜像(OCI 标签:版本 0.1.7、修订 a526f0d、ROCm 7.2.4、torch 2.10.0);旧 atom0.1.3 标签由接近 v0.1.3 的 ATOM main 构建且无修订标签。所用参数解析与默认值在 v0.1.7 中未变,因此无需修改参数。实测数值及异常说明见上表。
**下一步:**追加 changelog 条目并运行最终完整 sweep。

记录 DSR1 FP8 MI355X ATOM MTP 镜像更新的 perf-changelog 条目。

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Klaud-Cold Klaud-Cold added the full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended) label Oct 11, 2026
@Klaud-Cold

Klaud-Cold commented Oct 11, 2026 •

Copy link
Copy Markdown
Collaborator Author

Final full sweep · Passed · Run 38142648266 / attempt 1 · 2026-10-11 14:15 UTC
rocm/atom-dev:nightly_202610080328@sha256:9c3070bb7d2600d2e1bcfd03cf5f71e4dc0941f3e1d20d3a5845056aae2334eb · bdf53825691e · 8k/1k · TP8/EP1 · Mean latency
Change: Validate the complete updated-image family at this head.

Concurrency Output tok/s/GPU ↑ TTFT ms ↓ TPOT ms ↓
4 71.84 (-9.1%) 304.97 (-5.5%) 6.22 (+7.3%)
8 109.92 (-1.5%) 368.92 (-8.2%) 8.41 (+2.1%)
16 165.47 (+6.1%) 476 (-2.6%) 11.18 (-6.4%)
32 229.88 (+3%) 666.82 (-6.9%) 16.28 (-2.4%)
64 288.75 (+3.8%) 1,030.88 (-8.1%) 26.06 (-3.7%)
128 350.11 (+4.6%) 1,753.29 (-10.5%) 42.98 (-4.4%)
256 404.45 (+7.1%) 3,138.9 (-12.7%) 74.87 (-6.3%)

Note: All rows: request errors unavailable.

Eval Score ↑ Samples
gsm8k/em_strict · c64 95.83% (+0.15 pp) 1,319 each
gsm8k/em_strict · c256 95.83% (-0.23 pp) 1,319 each

Note: 1k/1k c4 TP8 EP1 c24d64, 1k/1k c8 TP8 EP1 f84cc2, 1k/1k c16 TP8 EP1 93b5ce, 1k/1k c32 TP8 EP1 1a341c, 1k/1k c64 TP8 EP1 4df1a8, 1k/1k c128 TP8 EP1 7853b7, 1k/1k c256 TP8 EP1 1147f0, 1k/1k c512 TP8 EP1 33d209: Single-turn 1k1k retired 2026-07-17 in #2263 after this baseline; excluded from coverage.

Next: Mark ready for maintainer review.

中文

最终完整 sweep · 已通过 · Run 38142648266 / attempt 1 · 2026-10-11 14:15 UTC
rocm/atom-dev:nightly_202610080328@sha256:9c3070bb7d2600d2e1bcfd03cf5f71e4dc0941f3e1d20d3a5845056aae2334eb · bdf53825691e · 8k/1k · TP8/EP1 · 平均延迟
**变更:**验证此提交更新镜像后的完整配置族。实测数值及异常说明见上表。
**下一步:**标记为就绪,等待维护者审查。

@github-actions

Copy link
Copy Markdown
Contributor

@Klaud-Cold
Klaud-Cold marked this pull request as ready for review October 11, 2026 14:15
@Klaud-Cold
Klaud-Cold requested a review from a team October 11, 2026 14:15
@Klaud-Cold

Copy link
Copy Markdown
Collaborator Author

validated · Repairs: 0 · Runs: 38141170149, 38142625880, 38142648266
All owned runs ended. Full sweep verified; ready for review.

中文

validated · 修复次数:0 · 运行:38141170149, 38142625880, 38142648266
所有自有运行均已结束。完整 sweep 已验证;已就绪,等待审查。

@Klaud-Cold

Copy link
Copy Markdown
Collaborator Author

/use 38142648266

@claude

claude Bot commented Oct 11, 2026 •

Copy link
Copy Markdown
Contributor

Claude finished @Klaud-Cold's task in 17s —— View job


LGTM, no blocking issues found.

@github-actions

github-actions Bot commented Oct 11, 2026 •

Copy link
Copy Markdown
Contributor

@Klaud-Cold staged run 38142648266: https://inferencemax-app-git-staging-semianalysisai.vercel.app/inference?i_dates=2026-10-11~r38142648266

This run remains available across future /stage-results requests. Staging the same run ID again updates its staged data. Staging workflow

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing blocking. The comments below are optional suggestions. There is no need to push a fix for them before merging.

- dsr1-fp8-mi355x-atom-mtp
description:
- "Update ATOM image from v0.1.3 to the v0.1.7 release build (rocm/atom-dev:nightly_202610080328, digest-pinned)."
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/3915

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 (optional) Changelog entry says image was updated "to the v0.1.7 release build" but the actual image pinned in both YAMLs is rocm/atom-dev:nightly_202610080328, a nightly dev build with no v0.1.7 tag anywhere. Anyone reading perf-changelog.yaml later to find which release is running will be misled into believing a tagged v0.1.7 release is deployed when it's actually an untagged nightly/dev digest build. Fix: make the description match the actual reference, e.g. "Update ATOM image from v0.1.3 to nightly build rocm/atom-dev:nightly_202610080328 (digest-pinned)", dropping the unverifiable "v0.1.7 release build" claim.

Why this was flagged

perf-changelog.yaml:9544 adds config-keys: [dsr1-fp8-mi355x-atom-mtp] with description:9546 "Update ATOM image from v0.1.3 to the v0.1.7 release build (rocm/atom-dev:nightly_202610080328, digest-pinned)". The actual image set at 8k1k.yaml:6 and amd-master.yaml:533 is rocm/atom-dev:nightly_202610080328@ sha256:9c3070bb..., which is a nightly/dev tag, not a release tag, and no v0.1.7 string appears anywhere in the diff. A later engineer or on-call reading the changelog to understand provenance of a regression will believe a stable v0.1.7 release is running and rule out nightly-build instability as a cause. No CI check cross-validates changelog description text against the image string, so nothing catches this mismatch before merge.

Verification: nit. The changelog entry at inferencex-e2e/perf-changelog.yaml:9545 reads "Update ATOM image from v0.1.3 to the v0.1.7 release build (rocm/atom-dev:nightly_202610080328, digest-pinned)." but the image actually pinned at inferencex-e2e/benchmarks/single_node/srt-slurm-recipes/dsr1/atom/mi355x-fp8-mtp/8k1k.yaml:6 and inferencex-e2e/configs/amd-master.yaml:535 is rocm/atom-dev:nightly_202610080328@ sha256:9c3070bb..., a nightly dev-channel build, not a "release build".

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended)

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

1 participant