[AMD][MI355X] DSv4.1 Flash: enable the vLLM mono decode kernels / [AMD][MI355X] DSv4.1 Flash：启用 vLLM mono decode 内核（镜像待定）

inferencex-e2e/benchmarks/single_node/srt-slurm-recipes/dsv41flash/vllm/mi355x-fp4-mtp/agentic.yaml

@@ -6,10 +6,13 @@ base:
   name: dsv41flash-fp4-mi355x-vllm-agentic
   model:
     path: hf:deepseek-ai/DeepSeek-V4.1-Flash
-    # Repinned to a nightly build that includes vllm-project/vllm#58671
-    # (paged MXFP4 sparse indexer), #58655 (mHC fused Triton seams) and
-    # #53492 (sparse MLA Gluon kernel).
-    container: vllm/vllm-openai-rocm:nightly-rocm100-ac9126e58aa7bbab1856ba6593ba4d5003fea516
+    # The first ROCm 10.0 nightly carrying vllm-project/vllm#60397 (DSv4.1
+    # mono decode kernels), which the environment below enables. It also
+    # brings #59685 (AITER MegaMoEV2 for DeepSeek V4), #56638 (shared
+    # combine_topk_swa_indices kernel in place of the Torch prefill combine),
+    # #60159 (native merge_attn_states on gfx950) and #60389 (MLA derived
+    # weights kept in place on weight reload).
+    container: vllm/vllm-openai-rocm:nightly-rocm100-7d0b4e57aac4c4323225b0eee9ae960ce75ffffe
     precision: fp4
   resources:
     gpu_type: mi355x
@@ -62,6 +65,13 @@ base:
       env:
         VLLM_ROCM_USE_AITER: '1'
         VLLM_ROCM_USE_AITER_MOE: '1'
+        # Mono decode kernels (vllm-project/vllm#60397): two persistent
+        # launches a layer for decode steps of up to 48 rows (c8 with five
+        # drafts); larger steps take the regular path.
+        VLLM_ROCM_MONO_DECODE: '1'
+        # INT4 quick all-reduce above its size threshold
+        # (vllm-project/recipes#1061).
+        VLLM_ROCM_QUICK_REDUCE_QUANTIZATION: INT4
         AITER_TRITON_LOG_LEVEL: ERROR
         VLLM_ROCM_USE_AITER_TRITON_SPARSE_MLA: 'True'
         # DeepseekV41ForCausalLM is not torch-compiled upstream; breakable
@@ -80,11 +90,10 @@ base:
       WEKA_LOADER_OVERRIDE: semianalysis_cc_traces_weka_062126
       AIPERF_REQUIRED_SERVER_METRIC_PREFIX: 'vllm:'
 
-# One variant per point (c128 excluded for both TP2 and TP4). Engram tables
-# move to pinned host memory at TP2, where resident tables leave too little
-# KV. Graph capture covers 2x CONC x (1 + 5 drafts). The prefill chunk shrinks
-# at TP2 from c64 to buy KV room, and where it drops below 1024 x 6 slots the
-# scheduler is capped at the captured shape.
+# One variant per point, concurrency 1 through 64 on both TP arms (c128 and
+# c192 excluded). Engram tables move to pinned host memory at TP2, where
+# resident tables leave too little KV. Graph capture covers 2x CONC x
+# (1 + 5 drafts). The prefill chunk is the upstream 16384 everywhere.
 override_tp4_c1:
   roles:
     agg:
@@ -262,7 +271,7 @@ override_tp2_c64:
         tensor-parallel-size: 2
         engram-config: '{"cpu_offload":true}'
         max-cudagraph-capture-size: 1024
-        max-num-batched-tokens: 8192
+        max-num-batched-tokens: 16384
   benchmark:
     env:
       CONC: '64'

inferencex-e2e/configs/amd-master.yaml

@@ -1498,9 +1498,9 @@ dsv4-fp4-mi355x-sglang-agentic-mtp:
 # Two GPUs per server doubles the servers per node and is the layout that
 # decides whether DSv4.1-Flash is throughput- or interactivity-bound here.
 dsv41flash-fp4-mi355x-vllm-agentic-dspark:
-  # Repinned to a nightly build that includes vllm-project/vllm#58671,
-  # #58655 and #53492 (see agentic.yaml).
-  image: vllm/vllm-openai-rocm:nightly-rocm100-ac9126e58aa7bbab1856ba6593ba4d5003fea516
+  # The first ROCm 10.0 nightly carrying vllm-project/vllm#60397 (see
+  # agentic.yaml).
+  image: vllm/vllm-openai-rocm:nightly-rocm100-7d0b4e57aac4c4323225b0eee9ae960ce75ffffe
   model: deepseek-ai/DeepSeek-V4.1-Flash
   model-prefix: dsv41flash
   runner: cluster:mi355x-amds
@@ -1511,11 +1511,9 @@ dsv41flash-fp4-mi355x-vllm-agentic-dspark:
     agentic-coding:
     - dram-utilization: 0.60
       search-space:
-      # The recipe only changes the memory split where a point ran short of KV:
-      # the prefill chunk shrinks at TP=2 c64. c128 is excluded for both TP=2
-      # and TP=4. Every point is measured here rather than combined from the
-      # earlier run: the image move leaves concurrency 1-32 measured only on
-      # the superseded nightly-eed1f3d0 pin.
+      # Concurrency 1-64 on both TP arms, every point re-measured on the new
+      # image. c128 and c192 are excluded: c128 did not raise the scored
+      # metric at TP=2 (150,683 against 151,896 tok/s/chip at c64).
       - { tp: 4, kv-offloading: none, spec-decoding: mtp, conc-list: [1, 2, 4, 8, 16, 32, 64], srt-recipe: benchmarks/single_node/srt-slurm-recipes/dsv41flash/vllm/mi355x-fp4-mtp/agentic.yaml }
       - { tp: 2, kv-offloading: none, spec-decoding: mtp, conc-list: [1, 2, 4, 8, 16, 32, 64], srt-recipe: benchmarks/single_node/srt-slurm-recipes/dsv41flash/vllm/mi355x-fp4-mtp/agentic.yaml }
 

inferencex-e2e/perf-changelog.yaml

@@ -9561,3 +9561,15 @@
     - "C1 to C16 are unchanged."
     - "No data-type or precision change to the DSpark draft: the bundled draft of deepseek-ai/DeepSeek-V4-Pro-0813 runs as shipped."
   pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/3819
+
+- config-keys:
+    - dsv41flash-fp4-mi355x-vllm-agentic-dspark
+  scenario-type:
+    - agentic-coding
+  description:
+    - "Update the image from nightly-rocm100-ac9126e58aa7bbab1856ba6593ba4d5003fea516 to nightly-rocm100-7d0b4e57aac4c4323225b0eee9ae960ce75ffffe, the first ROCm 10.0 nightly with the DeepSeek-V4.1 mono decode kernels of vllm-project/vllm#60397, and update model.container to match. The image also brings vllm-project/vllm#59685, #56638, #60159 and #60389."
+    - "Enable the mono decode kernels (VLLM_ROCM_MONO_DECODE=1) at TP2 and TP4: a decode step of up to 48 rows, i.e. concurrency 1-8 with five DSpark drafts, runs each backbone layer as two persistent launches (30 of 40 layers whole, the other 10 the FFN launch only). Larger steps take the regular path, so C16-C64 are expected unchanged."
+    - "Enable VLLM_ROCM_QUICK_REDUCE_QUANTIZATION=INT4 (vllm-project/recipes#1061) for tensor-parallel all-reduces above the quick-reduce size threshold."
+    - "Restore --max-num-batched-tokens 16384 at TP2 c64: vllm-project/vllm#58014 removed the profiling over-reservation the 8192 cap worked around. The prefill chunk is now 16384 at every point and the ladder stays at concurrency 1-64."
+    - "No data-type or precision change to the DSpark draft: vllm-project/vllm#60397 skips the draft layers (layer id >= num_hidden_layers), so the bundled DSpark draft of deepseek-ai/DeepSeek-V4.1-Flash runs on vLLM's regular path as shipped."
+  pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/3690