Skip to content

Commit d37d6ab

Browse files
Drop vllm-project/vllm#58671 from the DSv4.1-Flash MI355X recipe
#58671 isn't merging in time. Keep #53492's VLLM_ROCM_USE_AITER_TRITON_SPARSE_MLA env var and the image unpin for #58208/#58655, which are unrelated. Drop the --attention-config flag and its --block-size 128 workaround, both of which existed only to exercise #58671's ROCm paged MXFP4 sparse-logits indexer. Signed-off-by: Fangzhou Ai <31551580+Fangzhou-Ai@users.noreply.github.com> Co-authored-by: Cursor <cursoragent@cursor.com>
1 parent e82af63 commit d37d6ab

3 files changed

Lines changed: 26 additions & 3 deletions

File tree

‎inferencex-e2e/benchmarks/single_node/srt-slurm-recipes/dsv41flash/vllm/mi355x-fp4-mtp/agentic.yaml‎

Lines changed: 16 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,11 @@ base:
66
name: dsv41flash-fp4-mi355x-vllm-agentic
77
model:
88
path: hf:deepseek-ai/DeepSeek-V4.1-Flash
9-
container: vllm/vllm-openai-rocm:nightly-rocm100-7f1a5398e9610d96c473931a26c0e12bbe0d0423
9+
# TODO: repin once vllm-project/vllm#58655 (mHC fused Triton seams),
10+
# #58671 (paged MXFP4 sparse indexer), #53492 (sparse MLA Gluon kernel)
11+
# and #58208 (DSA candidate block-selection topk) are merged upstream and
12+
# released in a nightly image.
13+
container: vllm/vllm-openai-rocm:nightly-rocm100-TBD
1014
precision: fp4
1115
resources:
1216
gpu_type: mi355x
@@ -40,6 +44,16 @@ base:
4044
# aiter_triton_mxfp4_bf16 would force the W4A16 Triton kernel.
4145
moe-backend: aiter
4246
gpu-memory-utilization: 0.9
47+
# Enables vllm-project/vllm#58671's ROCm paged MXFP4 sparse-logits
48+
# indexer kernel instead of the dense fp8 indexer path.
49+
attention-config: '{"indexer_kv_dtype":"mxfp4","indexer_sparse_logits":true}'
50+
# With 4+ attention backends active (SparseMLA, SparseSWA, the MXFP4
51+
# indexer, Compressor), vLLM's block-size auto-resolution picks the
52+
# smallest size every backend merely supports (64), not the 128 the
53+
# indexer and SparseMLA backends actually prefer for this fp4 indexer
54+
# path. Force 128 explicitly; see vllm-project/vllm's
55+
# Platform._preferred_block_size_for_backends.
56+
block-size: 128
4357
# Five-token DSpark with probabilistic drafting. Throughput runs replace
4458
# block rejection with the golden acceptance length. Adaptive
4559
# verification trims requests on device, which the ROCm DeepSeek-V4
@@ -54,6 +68,7 @@ base:
5468
VLLM_ROCM_USE_AITER: '1'
5569
VLLM_ROCM_USE_AITER_MOE: '1'
5670
AITER_TRITON_LOG_LEVEL: ERROR
71+
VLLM_ROCM_USE_AITER_TRITON_SPARSE_MLA: 'True'
5772
# DeepseekV41ForCausalLM is not torch-compiled upstream; breakable
5873
# graphs keep FULL_AND_PIECEWISE capture working.
5974
VLLM_USE_BREAKABLE_CUDAGRAPH: '1'

‎inferencex-e2e/configs/amd-master.yaml‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1386,8 +1386,9 @@ dsv4-fp4-mi355x-sglang-agentic-mtp:
13861386
# Two GPUs per server doubles the servers per node and is the layout that
13871387
# decides whether DSv4.1-Flash is throughput- or interactivity-bound here.
13881388
dsv41flash-fp4-mi355x-vllm-agentic-dspark:
1389-
# ROCm 10.0 nightly channel, shared with kimik3-fp4-mi355x-vllm-agentic-mtp.
1390-
image: vllm/vllm-openai-rocm:nightly-rocm100-29468dde8b515031dc6d4d9d06bf0a2fa0442098
1389+
# TODO: repin once vllm-project/vllm#58655, #58671, #53492 and #58208 are
1390+
# merged upstream and released in a nightly image (see agentic.yaml).
1391+
image: vllm/vllm-openai-rocm:nightly-rocm100-TBD
13911392
model: deepseek-ai/DeepSeek-V4.1-Flash
13921393
model-prefix: dsv41flash
13931394
runner: cluster:mi355x-amds

‎inferencex-e2e/perf-changelog.yaml‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9003,3 +9003,10 @@
90039003
description:
90049004
- "Update B300 vLLM AgentX to DSpark6 on a new image with a sampled concurrency grid and per-mode --kv-cache-memory-bytes pins."
90059005
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/3477
9006+
9007+
- config-keys:
9008+
- dsv41flash-fp4-mi355x-vllm-agentic-dspark
9009+
description:
9010+
- "Enable vllm-project/vllm#58671's ROCm paged MXFP4 sparse-logits indexer via --attention-config, force VLLM_ROCM_USE_AITER_TRITON_SPARSE_MLA=True for #53492's Gluon sparse-MLA kernel, and pin --block-size 128 since auto-resolution otherwise picks 64 across this model's 4+ attention backends. Unpin image to TBD pending #58208 (faster DSA candidate topk selection), #58655 (fused mHC Triton kernel), #58671 (paged MXFP4 sparse indexer) and #53492 (sparse MLA Gluon kernel) landing in a nightly release: better topk/mHC/indexer/MLA kernels with better compute/communication stream overlap."
9011+
- "通过 --attention-config 启用 vllm-project/vllm#58671 的 ROCm 分页 MXFP4 稀疏 logits indexer,设置 VLLM_ROCM_USE_AITER_TRITON_SPARSE_MLA=True 以启用 #53492 的 Gluon sparse-MLA kernel,并固定 --block-size 128:该模型有 4 个以上的 attention backend 同时生效,自动推断否则会选出 64。将镜像取消固定为 TBD,待 #58208(更快的 DSA 候选块 topk 选择)、#58655(融合 mHC Triton kernel)、#58671(分页 MXFP4 稀疏 indexer)与 #53492(sparse MLA Gluon kernel)合并至上游并随 nightly 镜像发布:更优的 topk/mHC/indexer/MLA kernel,以及更好的计算/通信流重叠模式。"
9012+
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/3555

0 commit comments

Comments
 (0)