@@ -412,6 +412,38 @@ else
 fi
 CONTEXT_LEN=1048576
 
+# ROCm RoPE + KV-cache fusion (opt-in, ROCM_ROPE_KVCACHE_FUSION=1).
+#
+# vLLM's O3 defaults enable fuse_rope_kvcache / fuse_rope_kvcache_cat_mla on
+# ROCm (see enable_rope_kvcache_fusion in vllm/config/vllm.py), but both land
+# disabled here. The gate is splitting_ops, not the custom-op list: when
+# splitting_ops is left unset, set_splitting_ops_for_v1 populates it from
+# _attention_ops -- which appends unified_kv_cache_update /
+# unified_mla_kv_cache_update -- and then explicitly disables fuse_rope_kvcache
+# with "splitting_ops is None and Inductor graph partition is not enabled".
+# The profiled run shows exactly that list, so the fusion never ran.
+# ("+rotary_embedding" is appended automatically when the flag survives, so it
+# does not need to be listed by hand.)
+#
+# Passing an explicit empty splitting_ops takes the documented escape hatch.
+# FULL_DECODE_ONLY is unaffected by the empty list: the downgrade branch for
+# empty splitting_ops only rewrites PIECEWISE and FULL_AND_PIECEWISE.
+#
+# Trace evidence for the payoff (194 decode steps, MI355X agentic):
+#   _inverse_rope_gptj_kernel                    78.6 ms  n=12800  6.14 us avg
+#   fusedDeepseekV4QNormRopeKVRopeQuantInsert    80.3 ms  n=12800  6.27 us avg
+#   _finalize_norm_rope_quant_store              31.8 ms  n=6200   5.14 us avg
+# All three sit on the ~4 us MI355X dispatch floor, so most of that time is
+# launch overhead rather than work. rope_kvcache_fusion_max_token_num defaults
+# to 256 and decode steps here verify 16-28 tokens, so decode stays in range.
+#
+# Left off by default: this changes the compiled graph structure and needs its
+# own accuracy gate before it becomes the baseline.
+COMPILATION_CONFIG='{"mode":3,"cudagraph_mode":"FULL_DECODE_ONLY"}'
+if [ "${ROCM_ROPE_KVCACHE_FUSION:-0}" = "1" ]; then
+    COMPILATION_CONFIG='{"mode":3,"cudagraph_mode":"FULL_DECODE_ONLY","splitting_ops":[],"pass_config":{"fuse_rope_kvcache":true,"fuse_rope_kvcache_cat_mla":true}}'
+fi
+
 # MTP: cudagraph capture sizes are in TOKENS. With num_speculative_tokens=N,
 # every uniform decode batch of S seqs verifies S*(1+N) tokens, so capture the
 # explicit multiples (1+N), 2*(1+N), ..., MAX_NUM_SEQS*(1+N) -- one graph per
@@ -442,8 +474,17 @@ fi
 # "available KV cache memory ... larger than ..." on the tighter (eval-only)
 # relaunch. 0.86 (247.7 GiB) adds ~2.6 GiB, nearly all to KV (~26.5 GiB), so
 # the KV check clears with margin while still keeping ~8 GiB free-mem headroom
-# below the ~256 GiB hard-fail ceiling. Additional 0.9 for mooncake headroom
-GPU_MEM_UTIL=0.90
+# below the ~256 GiB hard-fail ceiling.
+#
+# The DRAM-offload arms (mooncake/lmcache) wanted extra headroom and pushed this
+# to 0.90, but those arms are commented out in amd-master.yaml -- the active arms
+# are both kv-offloading:none. 0.90 (259.2 GiB) is above the ~256 GiB hard-fail
+# ceiling the comment above documents, so keep the derived 0.86 for GPU-resident
+# runs and only raise it when a DRAM-offload arm is re-enabled.
+GPU_MEM_UTIL=0.86
+if [ "$KV_OFFLOADING" != "none" ]; then
+    GPU_MEM_UTIL=0.90
+fi
 
 # Long-context forward passes (~370K tokens with fp8 KV + DRAM offload) can exceed
 # vLLM's default 300s worker RPC timeout, killing the engine with
@@ -472,7 +513,7 @@ VLLM_CMD=(
     "${MODE_ARGS[@]}"
     --gpu-memory-utilization "$GPU_MEM_UTIL"
     --moe-backend aiter
-    --compilation-config '{"mode":3,"cudagraph_mode":"FULL_DECODE_ONLY"}'
+    --compilation-config "$COMPILATION_CONFIG"
     --speculative-config "$SPEC_CONFIG"
     --tokenizer-mode deepseek_v4
     --tool-call-parser deepseek_v4
 
@@ -1550,9 +1550,31 @@ dsv4-fp4-mi355x-vllm-agentic-mtp:
       search-space:
       #- { tp: 8, ep: 1, dp-attn: true, kv-offloading: dram, kv-offload-backend: { name: mooncake, version: "v0.3.12" }, spec-decoding: mtp, conc-list: [48], router: { name: vllm-router, version: "0.1.14" } }
       #- { tp: 8, ep: 1, dp-attn: true, kv-offloading: dram, kv-offload-backend: { name: lmcache, version: "v0.5.2" }, spec-decoding: mtp, conc-list: [48], router: { name: vllm-router, version: "0.1.14" } }
-      # DP+TP 160.9 GiB   
-      # TP    134.9 GiB   
+      # DP+TP 160.9 GiB
+      # TP    134.9 GiB
+      # Baseline: pure TP8, no EP. This is the arm the MI355X agentic profile was
+      # captured on (--tensor-parallel-size 8 --data-parallel-size 1, no all2all).
       - { tp: 8, ep: 1, dp-attn: false, kv-offloading: none, spec-decoding: mtp, conc-list: [1, 4, 8, 16, 32, 40, 48] }
+      # DEP8: DP-attention + EP8. The entry comment above ("Sweeps DP-attention
+      # on/off and EP=8") and the script header both describe this arm, but no
+      # search-space row enabled it -- every row was ep:1/dp-attn:false, so
+      # neither --enable-expert-parallel nor the DP path was ever exercised.
+      #
+      # Profile-driven motivation (MI355X dsv4 agentic trace, 200 steps):
+      #   * MoE GEMMs scale super-linearly with per-rank batch -- mfma_moe1 x2.16
+      #     and mfma_moe2 x2.78 going from 16 to 28 decode tokens/step. They are
+      #     latency-bound at small batch, so raising per-rank batch via DP-attn
+      #     is a direct win on 1643 ms (20.6% of all kernel time).
+      #   * cross_device_reduce_2stage costs 577.8 ms (7.2%) as the per-layer MoE
+      #     allreduce that EP replaces with all2all dispatch/combine.
+      # Both effects need measurement, not extrapolation: all2all on MI355X is
+      # many small messages and may not beat the fused allreduce. Hence this arm.
+      #
+      # conc starts at 32: under DP-attn --max-num-seqs is applied PER DP rank,
+      # so conc<8 leaves most of the 8 ranks empty and DEP is strictly worse.
+      # router is required -- it expands the single HTTP backend into one logical
+      # worker per DP rank and stamps X-data-parallel-rank on forwarded requests.
+      - { tp: 8, ep: 8, dp-attn: true, kv-offloading: none, spec-decoding: mtp, conc-list: [32, 48, 64], router: { name: vllm-router, version: "0.1.14" } }
 
 # DSv4-Pro FP4 on MI355X via SGLang. Uses a rocm720 mi35x image built off the
 # amd/deepseek_v4 branch in sgl-project/sglang; the SHA is encoded in the
 
@@ -5364,3 +5364,13 @@
     - "Apply the accuracy-gated Kimi-K2.5 MXFP4 settings: tuned AITER MXFP4 MoE, fused shared experts, FP8 KV cache, block size 16, 16384 batched tokens, 512 sequences, async scheduling, gpu-memory-utilization 0.85 (headroom for CUDA-graph capture on MI355X), and the AITER BF16 GEMM path"
     - "Extend the TP4 and TP8 8k1k concurrency sweep from 64 to 128 (1k1k deprecated per #2263)"
   pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2213
+
+- config-keys:
+    - dsv4-fp4-mi355x-vllm-agentic-mtp
+  description:
+    - "Add the DEP8 arm the entry always described but never ran. Both the amd-master.yaml entry comment ('Sweeps DP-attention on/off and EP=8') and the script header (which defines DEP as DP_ATTENTION=true, EP_SIZE>1) document this configuration, and the script already implements it -- EP_SIZE>1 appends --enable-expert-parallel and --enable-ep-weight-filter, DP_ATTENTION=true swaps to --tensor-parallel-size 1 --data-parallel-size TP plus the DEP8 prefill budget. But every search-space row was ep:1/dp-attn:false, so neither branch was ever taken. New row: { tp: 8, ep: 8, dp-attn: true, kv-offloading: none, conc-list: [32, 48, 64], router: vllm-router 0.1.14 }. The existing pure-TP row is kept as the comparison baseline; it is the configuration the MI355X agentic profile was captured on."
+    - "Profile-driven motivation from the 200-step MI355X agentic trace (TP8/DP1/EP-off). MoE GEMMs are latency-bound at small per-rank batch and scale super-linearly with it: mfma_moe1 x2.16 and mfma_moe2 x2.78 from 16 to 28 decode tokens/step, over 1643 ms of kernel time (20.6%). DP-attention raises per-rank batch and should convert directly. Separately, cross_device_reduce_2stage costs 577.8 ms (7.2%, 368.7 ms of it in decode) as the per-layer MoE allreduce that EP replaces with all2all dispatch/combine. The second effect is the one that needs measuring rather than extrapolating: all2all on MI355X is many small messages and may not beat the fused allreduce, which is precisely why this arm exists."
+    - "conc starts at 32 (not 1/4/8 as in the pure-TP row) because --max-num-seqs is applied PER DP rank under DP-attention: at conc 8 across 8 ranks most ranks sit empty and DEP is strictly worse than pure TP. The router entry is required, not optional -- it expands the single HTTP backend into one logical worker per DP rank and stamps X-data-parallel-rank on forwarded requests; without it requests do not spread across ranks. Both commented-out DPA arms already carried it."
+    - "Fix --gpu-memory-utilization: was a flat 0.90, now 0.86 for kv-offloading:none and 0.90 only when offloading is on. The comment block above the assignment derives 0.86 in detail (0.95 requests 273.6 GiB and every DP worker hard-fails against the ~256 GiB free-at-init ceiling from the MI355X ~32 GiB/GPU carveout; 0.85 leaves ~23.9 GiB KV, below the 24.06 GiB one 1M-context request needs once the MTP draft layer's KV is counted) and then the assignment used 0.90 anyway, for mooncake headroom -- but both mooncake and lmcache arms are commented out, so every active arm is GPU-resident and 0.90 (259.2 GiB) sits above the documented hard-fail ceiling. The DRAM value is preserved behind the KV_OFFLOADING check so re-enabling an offload arm keeps its headroom."
+    - "Add opt-in ROCm RoPE + KV-cache fusion via ROCM_ROPE_KVCACHE_FUSION=1 (default off). vLLM's O3 defaults enable fuse_rope_kvcache / fuse_rope_kvcache_cat_mla on ROCm, but both land disabled in this recipe. The gate is splitting_ops, not the custom-op list: with splitting_ops unset, set_splitting_ops_for_v1 populates it from _attention_ops (appending unified_kv_cache_update / unified_mla_kv_cache_update) and then explicitly disables fuse_rope_kvcache with 'splitting_ops is None and Inductor graph partition is not enabled'. The profiled server log shows exactly that populated list, confirming the fusion never ran. Passing an explicit empty splitting_ops is the escape hatch the warning itself names; FULL_DECODE_ONLY is unaffected because the empty-list downgrade branch only rewrites PIECEWISE and FULL_AND_PIECEWISE. Trace evidence for the payoff: _inverse_rope_gptj_kernel 78.6 ms (n=12800, 6.14 us avg), fusedDeepseekV4QNormRopeKVRopeQuantInsert 80.3 ms (n=12800, 6.27 us avg), _finalize_norm_rope_quant_store 31.8 ms (n=6200) -- all three on the ~4 us MI355X dispatch floor, so most of that time is launch overhead rather than work. rope_kvcache_fusion_max_token_num defaults to 256 and decode steps verify 16-28 tokens, so decode stays in range. Left off by default because it changes the compiled graph structure and needs its own accuracy gate."
+  pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/TBD