Repository navigation
[AMD] DeepSeek-V4-Pro MI355X vLLM AgentX: add AITER MegaMoE DEP8 band / [AMD] DeepSeek-V4-Pro MI355X vLLM AgentX：新增 AITER MegaMoE DEP8 测试点 - #3968
Conversation
Bump dsv4-fp4-mi355x-vllm-agentic-mtp to vllm/vllm-openai-rocm:nightly-fe4df3df0eb703862c686b256e9fe44fa94c8945, which includes the AITER MegaMoEV2 integration for DeepSeek-V4 (vllm-project/vllm#59685). Add DEP8 points with --moe-backend aiter_mega_moe, expert parallelism and the MoRI high-throughput all2all at concurrency 32-256, beside the unchanged TP8 and EP1 DEP8 bands. MegaMoE shared-expert fusion stays off because it requantizes the FP8 shared experts of the DSpark draft layers to MXFP4. 将 dsv4-fp4-mi355x-vllm-agentic-mtp 的镜像更新为 vllm/vllm-openai-rocm:nightly-fe4df3df0eb703862c686b256e9fe44fa94c8945，该镜像包含 DeepSeek-V4 的 AITER MegaMoEV2 集成（vllm-project/vllm#59685）。新增并发 32-256 的 DEP8 测试点，使用 --moe-backend aiter_mega_moe、专家并行与 MoRI high-throughput all2all，原有 TP8 与 EP1 DEP8 测试点保持不变。MegaMoE 的 shared-expert fusion 保持关闭，因为它会将 DSpark draft 层的 FP8 shared experts 重新量化为 MXFP4。 Co-authored-by: Cursor <cursoragent@cursor.com>
|
Thanks for the contribution!
中文感谢你的贡献！
|
将 perf-changelog 中的 PR 链接设置为 #3968。 Co-authored-by: Cursor <cursoragent@cursor.com>
micah-wil
added
the
full-sweep-enabled
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
| - "Add a DEP8 band with AITER MegaMoE at concurrency 32, 64, 128, 192 and 256: data-parallel-size 8, tensor-parallel-size 1, --enable-expert-parallel, --all2all-backend mori_high_throughput and --moe-backend aiter_mega_moe, behind the same vLLM Router consistent-hash routing, prefill interleaving and max-num-seqs = concurrency as the existing DEP8 band. vLLM sizes the MoRI symmetric heap for MegaMoE (8G)." | ||
| - "The TP8 band (concurrency 1-64) and the EP1 DEP8 band (concurrency 64, 128, 192) keep their settings on the new image, so the frontier can pick the better MoE at each overlapping concurrency." | ||
| - "No weight-precision change to the DSpark draft: VLLM_ROCM_AITER_MEGA_MOE_FUSE_SHARED_EXPERTS stays 0, because the shared-expert fusion requantizes the FP8 shared experts of the draft layers to MXFP4. Without it, the routed experts of the target and the bundled DeepSeek-V4-Pro-0813 draft keep their shipped MXFP4 weights and the FP8 shared experts run on their regular path. Like the DeepGEMM MegaMoE backends of the B200/B300 vLLM AgentX configs, the MegaMoE band runs the routed experts with FP8 activations (a8w4) where the default AITER MoE uses BF16; the TP8 and EP1 DEP8 bands keep the default." | ||
| pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/XXXX |
There was a problem hiding this comment.
🔴 This PR's new changelog entry uses the placeholder pull/XXXX (4 X's), which the repo's own merge/CI tooling does not recognize, so the automated PR-link canonicalization and CI validation will reject this file as submitted. PR_LINK_PLACEHOLDERS in inferencex-e2e/infx/workflows/validate_perf_changelog.py only allows XXX or the literal .../pull/XXX (3 X's), checked by exact in membership, not a regex. validate_added_pr_link (used by CI on PR runs) and canonicalize_appended_links (used by prepare_perf_changelog_merge.py at merge time) both raise ChangelogValidationError for any link not in that set and not equal to the real PR link. Fix: replace pull/XXXX with the exact placeholder pull/XXX (or the real PR number once known) so both checks accept the entry.
Why this was flagged
Trigger: perf-changelog.yaml:9659 is appended by this PR with pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/XXXX. Entry point 1: validate_perf_changelog.py's compare_entries -> validate_added_pr_link(link, pr_number) with pr_number set from CI's actual PR context; link not in PR_LINK_PLACEHOLDERS and link != expected is True since PR_LINK_PLACEHOLDERS only contains XXX/pull/XXX, so it raises new PR entry must use ... or an XXX placeholder; found 'https://github.com/SemiAnalysisAI/InferenceX/pull/XXXX'. Entry point 2: prepare_perf_changelog_merge.py's canonicalize_appended_links, run at merge time to auto-substitute the real PR number, hits the same link not in PR_LINK_PLACEHOLDERS check and raises appended entry N has unexpected pr-link. Base branch entries all use the 3-X placeholder (e.g. line 5598) or a real number, so none hit this; this is a defect introduced by this diff's new entry.
Verification: The new changelog entry (perf-changelog.yaml:9659) uses pr-link https://github.com/SemiAnalysisAI/InferenceX/pull/XXXX (4 X's). The only accepted placeholders are the two exact strings in PR_LINK_PLACEHOLDERS (validate_perf_changelog.py:22-25): XXX and https://github.com/SemiAnalysisAI/InferenceX/pull/XXX (3 X's), matched by exact in membership, not regex (line 147). The 4-X link is neither a placeholder nor the canonical link and raises ChangelogValidationError.
| - { tp: 8, ep: 1, dp-attn: true, kv-offloading: none, spec-decoding: draft_model, conc-list: [64, 128, 192], router: { name: vllm-router, version: "0.1.14" }, srt-recipe: benchmarks/single_node/srt-slurm-recipes/dsv4/vllm/mi355x-fp4-mtp/agentic.yaml } | ||
| # DEP8 with AITER MegaMoE (EP8) beside the EP1 DPA band; the Pareto | ||
| # frontier keeps whichever MoE wins at each concurrency. | ||
| - { tp: 8, ep: 8, dp-attn: true, kv-offloading: none, spec-decoding: draft_model, conc-list: [32, 64, 128, 192, 256], router: { name: vllm-router, version: "0.1.14" }, srt-recipe: benchmarks/single_node/srt-slurm-recipes/dsv4/vllm/mi355x-fp4-mtp/agentic.yaml } |
There was a problem hiding this comment.
🔴 New EP8 row makes test-config --exp-names ambiguous at conc 32/128/192 for dsv4-fp4-mi355x-vllm-agentic-mtp, which previously resolved to one row. _agentic_entries in infx/matrix/generate.py (lines 1196-1200) builds exp-name only from tp/kv-offloading/spec-decoding, ignoring ep and dp-attn, so the new {tp:8, ep:8, dp-attn:true, conc-list:[32,64,128,192,256]} row (amd-master.yaml:944) now shares dsv4_tp8_conc{32,64,128,192}_kvnone_spec-draft_model with the existing tp8/ep1 bands. filter_exp_names (generate.py:1573-1577) then raises "Experiment name(s) matched multiple rows", so a developer targeting one scenario now fails outright at those concurrencies, which worked before this diff. Fix: make exp-name encode ep and dp-attn (as the multinode exp-name helper already does), so the name stays unique per topology at every concurrency.
Why this was flagged
Trigger: anyone invoking python -m infx.matrix.generate test-config --config-keys dsv4-fp4-mi355x-vllm-agentic-mtp --exp-names dsv4_tp8_conc{32,128,192}_kvnone_spec-draft_model (e.g. via the e2e-tests.yml workflow dispatch, per .github/workflows/README.md:106-116) for the single-node agentic config in amd-master.yaml. _agentic_entries (infx/matrix/generate.py:1196-1200) derives exp-name from only tp, kv-offloading and spec-decoding; it never includes ep or dp-attn. The new row at amd-master.yaml:944 (tp:8, ep:8, conc-list [32,64,128,192,256]) duplicates the exp-name of the existing tp:8 ep:1 rows at amd-master.yaml:940/942 for conc 32, 64, 128, 192. filter_exp_names (generate.py:1560-1578) counts matches per name and raises ValueError when a name matches more than one row, so generation fails instead of selecting one scenario. Before this diff only conc=64 collided (pre-existing, between the two ep:1 bands); this diff widens the collision to conc 32, 128 and 192 as well, newly breaking previously-working --exp-names selections for those points.
Verification: nit. The mechanical claim is accurate and reachable, but the identical failure route already exists on the base branch, so merging adds only loud, immediately-actionable errors in a manual selection tool, not a new production incident. _agentic_entries (generate.py:1196-1200) omits ep/dp-attn from the exp-name; the new EP8 row (amd-master.yaml:944) collides with pre-existing bands at conc 32/128/192, and filter_exp_names (generate.py:1573-1577) raises "matched multiple rows."
This branch has not been deployed
1 participant
Bump dsv4-fp4-mi355x-vllm-agentic-mtp to vllm/vllm-openai-rocm:nightly-fe4df3df0eb703862c686b256e9fe44fa94c8945, which includes the AITER MegaMoEV2 integration for DeepSeek-V4 (vllm-project/vllm#59685). Add DEP8 points with --moe-backend aiter_mega_moe, expert parallelism and the MoRI high-throughput all2all at concurrency 32-256, beside the unchanged TP8 and EP1 DEP8 bands. MegaMoE shared-expert fusion stays off because it requantizes the FP8 shared experts of the DSpark draft layers to MXFP4.
将 dsv4-fp4-mi355x-vllm-agentic-mtp 的镜像更新为 vllm/vllm-openai-rocm:nightly-fe4df3df0eb703862c686b256e9fe44fa94c8945，该镜像包含 DeepSeek-V4 的 AITER MegaMoEV2 集成（vllm-project/vllm#59685）。新增并发 32-256 的 DEP8 测试点，使用 --moe-backend aiter_mega_moe、专家并行与 MoRI high-throughput all2all，原有 TP8 与 EP1 DEP8 测试点保持不变。MegaMoE 的 shared-expert fusion 保持关闭，因为它会将 DSpark draft 层的 FP8 shared experts 重新量化为 MXFP4。