config: update B300 DSV41flash vLLM image and retune TP4/DEP2 (#3653)

Move the B300 DeepSeek-V4.1-Flash vLLM AgentX recipe to
nightly-dev-x86_64-cu130-ac9126e58aa7 and enable FlashInfer autotuning.
Run TP4 at CONC 1-16 and replace TP2 with DEP2 (TP1 x DP2 + EP2,
MegaMoE) at CONC 8-192 behind a consistent-hash vLLM Router. Append the
perf-changelog entry and document the B300 settings.

将 B300 DeepSeek-V4.1-Flash vLLM AgentX 配方更新到
nightly-dev-x86_64-cu130-ac9126e58aa7，并开启 FlashInfer autotune。
TP4 覆盖 CONC 1-16；用 DEP2（TP1 x DP2 + EP2，MegaMoE）替代 TP2，覆盖
CONC 8-192，前置一致性哈希 vLLM Router。追加 perf-changelog 条目，并补充
B300 配置文档。

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: functionstackx <47992694+functionstackx@users.noreply.github.com>

Author: 3 people

inferencex-e2e/benchmarks/single_node/srt-slurm-recipes/dsv41flash/vllm/b300-fp4-mtp/agentic.yaml

@@ -6,7 +6,7 @@ base:
   name: dsv41flash-fp4-b300-vllm-agentic
   model:
     path: hf:deepseek-ai/DeepSeek-V4.1-Flash
-    container: vllm/vllm-openai:nightly-ddd6fbca148a867aad1fcab7ec72f582b9977db4
+    container: vllm/vllm-openai:nightly-dev-x86_64-cu130-ac9126e58aa7
     precision: fp4
   resources:
     gpu_type: b300
@@ -37,17 +37,21 @@ base:
         enable-auto-tool-choice: true
         reasoning-parser: deepseek_v41
         engram-config: '{"cpu_offload":true}'
-        # FlashInfer sparse attention with the merged Blackwell sparse indexer
-        # (MXFP4 indexer KV, sparse logits) at both TP sizes; fp8 KV cache.
+        kernel-config: '{"enable_flashinfer_autotune":true}'
         attention-config: '{"backend":"FLASHINFER_MLA_SPARSE_DSV41","indexer_kv_dtype":"mxfp4","indexer_sparse_logits":true}'
         kv-cache-dtype: fp8
         # Five-token DSpark with probabilistic drafting. Throughput runs replace
         # block rejection with the golden acceptance length.
         speculative-config: '{"method":"dspark","num_speculative_tokens":5,"draft_sample_method":"probabilistic","rejection_sample_method":"block","enable_adaptive_verification":false}'
         max-model-len: 1048576
-        max-num-seqs: 256
+        # Piecewise graph capture sizes are multiples of the six-token DSpark
+        # verification block, denser for small batches.
+        compilation-config: '{"mode":"VLLM_COMPILE","cudagraph_mode":"FULL_AND_PIECEWISE","cudagraph_capture_sizes":[6,12,18,24,30,36,48,60,72,96,120,144,192,240,288,384,480,576,768,1020,1536,2046,3072,4092,6144,8190]}'
+        max-cudagraph-capture-size: 8190
+        max-num-batched-tokens: 8192
         disable-uvicorn-access-log: true
       env:
+        VLLM_USE_V2_MODEL_RUNNER: '1'
         VLLM_USE_RUST_FRONTEND: '1'
         PYTHONUNBUFFERED: '1'
         VLLM_ENGINE_READY_TIMEOUT_S: '7200'
@@ -58,203 +62,66 @@ base:
       MODEL: deepseek-ai/DeepSeek-V4.1-Flash
       AIPERF_REQUIRED_SERVER_METRIC_PREFIX: 'vllm:'
 
-# One variant per point. Piecewise graph capture sizes are multiples of the
-# six-token DSpark verification block, denser for small batches, and each
-# batched-token limit matches the largest captured graph: CONC <= 4 and TP2
-# CONC 128 capture up to 2046 tokens (the latter at 0.97 memory utilization),
-# every other point up to 8190.
-override_tp4_c1:
-  roles:
-    agg:
-      gpus: 4
-      args:
-        tensor-parallel-size: 4
-        compilation-config: '{"mode":"VLLM_COMPILE","cudagraph_mode":"FULL_AND_PIECEWISE","cudagraph_capture_sizes":[6,12,18,24,30,36,48,60,72,96,120,144,192,240,288,384,480,576,768,1020,1536,2046]}'
-        max-cudagraph-capture-size: 2046
-        max-num-batched-tokens: 2048
-  benchmark:
-    env:
-      CONC: '1'
-
-override_tp4_c2:
-  roles:
-    agg:
-      gpus: 4
-      args:
-        tensor-parallel-size: 4
-        compilation-config: '{"mode":"VLLM_COMPILE","cudagraph_mode":"FULL_AND_PIECEWISE","cudagraph_capture_sizes":[6,12,18,24,30,36,48,60,72,96,120,144,192,240,288,384,480,576,768,1020,1536,2046]}'
-        max-cudagraph-capture-size: 2046
-        max-num-batched-tokens: 2048
-  benchmark:
-    env:
-      CONC: '2'
-
-override_tp4_c4:
-  roles:
-    agg:
-      gpus: 4
-      args:
-        tensor-parallel-size: 4
-        compilation-config: '{"mode":"VLLM_COMPILE","cudagraph_mode":"FULL_AND_PIECEWISE","cudagraph_capture_sizes":[6,12,18,24,30,36,48,60,72,96,120,144,192,240,288,384,480,576,768,1020,1536,2046]}'
-        max-cudagraph-capture-size: 2046
-        max-num-batched-tokens: 2048
-  benchmark:
-    env:
-      CONC: '4'
-
-override_tp4_c8:
-  roles:
-    agg:
-      gpus: 4
-      args:
-        tensor-parallel-size: 4
-        compilation-config: '{"mode":"VLLM_COMPILE","cudagraph_mode":"FULL_AND_PIECEWISE","cudagraph_capture_sizes":[6,12,18,24,30,36,48,60,72,96,120,144,192,240,288,384,480,576,768,1020,1536,2046,3072,4092,6144,8190]}'
-        max-cudagraph-capture-size: 8190
-        max-num-batched-tokens: 8192
-  benchmark:
-    env:
-      CONC: '8'
-
-override_tp4_c16:
-  roles:
-    agg:
-      gpus: 4
-      args:
-        tensor-parallel-size: 4
-        compilation-config: '{"mode":"VLLM_COMPILE","cudagraph_mode":"FULL_AND_PIECEWISE","cudagraph_capture_sizes":[6,12,18,24,30,36,48,60,72,96,120,144,192,240,288,384,480,576,768,1020,1536,2046,3072,4092,6144,8190]}'
-        max-cudagraph-capture-size: 8190
-        max-num-batched-tokens: 8192
-  benchmark:
-    env:
-      CONC: '16'
-
-override_tp4_c32:
-  roles:
-    agg:
-      gpus: 4
-      args:
-        tensor-parallel-size: 4
-        compilation-config: '{"mode":"VLLM_COMPILE","cudagraph_mode":"FULL_AND_PIECEWISE","cudagraph_capture_sizes":[6,12,18,24,30,36,48,60,72,96,120,144,192,240,288,384,480,576,768,1020,1536,2046,3072,4092,6144,8190]}'
-        max-cudagraph-capture-size: 8190
-        max-num-batched-tokens: 8192
-  benchmark:
-    env:
-      CONC: '32'
-
-override_tp4_c64:
-  roles:
-    agg:
-      gpus: 4
-      args:
-        tensor-parallel-size: 4
-        compilation-config: '{"mode":"VLLM_COMPILE","cudagraph_mode":"FULL_AND_PIECEWISE","cudagraph_capture_sizes":[6,12,18,24,30,36,48,60,72,96,120,144,192,240,288,384,480,576,768,1020,1536,2046,3072,4092,6144,8190]}'
-        max-cudagraph-capture-size: 8190
-        max-num-batched-tokens: 8192
-  benchmark:
-    env:
-      CONC: '64'
+# Native zip expansion creates one variant per concurrency.
 
-override_tp4_c128:
+# TP4 covers low concurrency with THP Engram tables.
+zip_override_tp4:
   roles:
     agg:
       gpus: 4
       args:
         tensor-parallel-size: 4
-        compilation-config: '{"mode":"VLLM_COMPILE","cudagraph_mode":"FULL_AND_PIECEWISE","cudagraph_capture_sizes":[6,12,18,24,30,36,48,60,72,96,120,144,192,240,288,384,480,576,768,1020,1536,2046,3072,4092,6144,8190]}'
-        max-cudagraph-capture-size: 8190
-        max-num-batched-tokens: 8192
-  benchmark:
-    env:
-      CONC: '128'
-
-override_tp2_c2:
-  roles:
-    agg:
-      gpus: 2
-      args:
-        tensor-parallel-size: 2
-        compilation-config: '{"mode":"VLLM_COMPILE","cudagraph_mode":"FULL_AND_PIECEWISE","cudagraph_capture_sizes":[6,12,18,24,30,36,48,60,72,96,120,144,192,240,288,384,480,576,768,1020,1536,2046]}'
-        max-cudagraph-capture-size: 2046
-        max-num-batched-tokens: 2048
-  benchmark:
-    env:
-      CONC: '2'
-
-override_tp2_c4:
-  roles:
-    agg:
-      gpus: 2
-      args:
-        tensor-parallel-size: 2
-        compilation-config: '{"mode":"VLLM_COMPILE","cudagraph_mode":"FULL_AND_PIECEWISE","cudagraph_capture_sizes":[6,12,18,24,30,36,48,60,72,96,120,144,192,240,288,384,480,576,768,1020,1536,2046]}'
-        max-cudagraph-capture-size: 2046
-        max-num-batched-tokens: 2048
-  benchmark:
-    env:
-      CONC: '4'
-
-override_tp2_c8:
-  roles:
-    agg:
-      gpus: 2
-      args:
-        tensor-parallel-size: 2
-        compilation-config: '{"mode":"VLLM_COMPILE","cudagraph_mode":"FULL_AND_PIECEWISE","cudagraph_capture_sizes":[6,12,18,24,30,36,48,60,72,96,120,144,192,240,288,384,480,576,768,1020,1536,2046,3072,4092,6144,8190]}'
-        max-cudagraph-capture-size: 8190
-        max-num-batched-tokens: 8192
-  benchmark:
-    env:
-      CONC: '8'
+        engram-config: '{"cpu_offload":true,"use_thp":true}'
+        max-num-seqs: [8, 8, 16, 32]
+        compilation-config:
+          - '{"mode":"VLLM_COMPILE","cudagraph_mode":"FULL_AND_PIECEWISE","cudagraph_capture_sizes":[6,12,18,24,30,36,48,60,72,96,120,144,192,240,288,384,480,576,768,1020,1536,2046,3072,4092]}'
+          - '{"mode":"VLLM_COMPILE","cudagraph_mode":"FULL_AND_PIECEWISE","cudagraph_capture_sizes":[6,12,18,24,30,36,48,60,72,96,120,144,192,240,288,384,480,576,768,1020,1536,2046,3072,4092]}'
+          - '{"mode":"VLLM_COMPILE","cudagraph_mode":"FULL_AND_PIECEWISE","cudagraph_capture_sizes":[6,12,18,24,30,36,48,60,72,96,120,144,192,240,288,384,480,576,768,1020,1536,2046,3072,4092,6144,8190]}'
+          - '{"mode":"VLLM_COMPILE","cudagraph_mode":"FULL_AND_PIECEWISE","cudagraph_capture_sizes":[6,12,18,24,30,36,48,60,72,96,120,144,192,240,288,384,480,576,768,1020,1536,2046,3072,4092,6144,8190],"decoder_replay_cudagraph_capture_sizes":[6,12,18,24,30,36,48,60,72,96,120,144,192,240,288,384,480,576,768,1020,1536,2048,2304,2560,2816,3072,3328,3584,3840,4096]}'
+        max-cudagraph-capture-size: [4092, 4092, 8190, 8190]
+        max-num-batched-tokens: [4096, 4096, 8192, 8192]
+  benchmark:
+    env:
+      CONC: ['1', '4', '8', '16']
 
-override_tp2_c16:
-  roles:
-    agg:
-      gpus: 2
-      args:
-        tensor-parallel-size: 2
-        compilation-config: '{"mode":"VLLM_COMPILE","cudagraph_mode":"FULL_AND_PIECEWISE","cudagraph_capture_sizes":[6,12,18,24,30,36,48,60,72,96,120,144,192,240,288,384,480,576,768,1020,1536,2046,3072,4092,6144,8190]}'
-        max-cudagraph-capture-size: 8190
-        max-num-batched-tokens: 8192
-  benchmark:
-    env:
-      CONC: '16'
-
-override_tp2_c32:
-  roles:
-    agg:
-      gpus: 2
-      args:
-        tensor-parallel-size: 2
-        compilation-config: '{"mode":"VLLM_COMPILE","cudagraph_mode":"FULL_AND_PIECEWISE","cudagraph_capture_sizes":[6,12,18,24,30,36,48,60,72,96,120,144,192,240,288,384,480,576,768,1020,1536,2046,3072,4092,6144,8190]}'
-        max-cudagraph-capture-size: 8190
-        max-num-batched-tokens: 8192
-  benchmark:
-    env:
-      CONC: '32'
-
-override_tp2_c64:
-  roles:
-    agg:
-      gpus: 2
-      args:
-        tensor-parallel-size: 2
-        compilation-config: '{"mode":"VLLM_COMPILE","cudagraph_mode":"FULL_AND_PIECEWISE","cudagraph_capture_sizes":[6,12,18,24,30,36,48,60,72,96,120,144,192,240,288,384,480,576,768,1020,1536,2046,3072,4092,6144,8190]}'
-        max-cudagraph-capture-size: 8190
-        max-num-batched-tokens: 8192
-  benchmark:
-    env:
-      CONC: '64'
-
-override_tp2_c128:
+# DEP2: TP1 x DP2 + EP2 with MegaMoE behind a consistent-hash router;
+# MegaAttention at CONC 128 and above.
+zip_override_dep2:
+  frontend:
+    type: vllm-router
+    args:
+      policy: consistent_hash
+      prometheus-host: 127.0.0.1
+      prometheus-port: 18000
+      request-timeout-secs: 14400
+      disable-retries: true
+  setup_script: vllm-router-0.1.14.sh
   roles:
     agg:
       gpus: 2
       args:
-        tensor-parallel-size: 2
-        compilation-config: '{"mode":"VLLM_COMPILE","cudagraph_mode":"FULL_AND_PIECEWISE","cudagraph_capture_sizes":[6,12,18,24,30,36,48,60,72,96,120,144,192,240,288,384,480,576,768,1020,1536,2046]}'
-        max-cudagraph-capture-size: 2046
-        max-num-batched-tokens: 2048
-        gpu-memory-utilization: 0.97
-  benchmark:
-    env:
-      CONC: '128'
+        tensor-parallel-size: 1
+        data-parallel-size: 2
+        enable-expert-parallel: true
+        kernel-config: '{"enable_flashinfer_autotune":true,"moe_backend":"deep_gemm_mega_moe"}'
+        engram-config: '{"cpu_offload":true,"embedding_across_dp":true}'
+        max-num-seqs: [16, 16, 32, 64, 128, 192]
+        attention-config:
+          - '{"backend":"FLASHINFER_MLA_SPARSE_DSV41","indexer_kv_dtype":"mxfp4","indexer_sparse_logits":true}'
+          - '{"backend":"FLASHINFER_MLA_SPARSE_DSV41","indexer_kv_dtype":"mxfp4","indexer_sparse_logits":true}'
+          - '{"backend":"FLASHINFER_MLA_SPARSE_DSV41","indexer_kv_dtype":"mxfp4","indexer_sparse_logits":true}'
+          - '{"backend":"FLASHINFER_MLA_SPARSE_DSV41","indexer_kv_dtype":"mxfp4","indexer_sparse_logits":true}'
+          - '{"backend":"FLASHMLA_MEGA_ATTN_DSV41","indexer_kv_dtype":"mxfp4","indexer_sparse_logits":true}'
+          - '{"backend":"FLASHMLA_MEGA_ATTN_DSV41","indexer_kv_dtype":"mxfp4","indexer_sparse_logits":true}'
+        compilation-config:
+          - '{"mode":"VLLM_COMPILE","cudagraph_mode":"FULL_AND_PIECEWISE","cudagraph_capture_sizes":[6,12,18,24,30,36,48,60,72,96,120,144,192,240,288,384,480,576,768,1020,1536,2046,3072,4092,6144,8190]}'
+          - '{"mode":"VLLM_COMPILE","cudagraph_mode":"FULL_AND_PIECEWISE","cudagraph_capture_sizes":[6,12,18,24,30,36,48,60,72,96,120,144,192,240,288,384,480,576,768,1020,1536,2046,3072,4092,6144,8190]}'
+          - '{"mode":"VLLM_COMPILE","cudagraph_mode":"FULL_AND_PIECEWISE","cudagraph_capture_sizes":[6,12,18,24,30,36,48,60,72,96,120,144,192,240,288,384,480,576,768,1020,1536,2046,3072,4092,6144,8190],"decoder_replay_cudagraph_capture_sizes":[6,12,18,24,30,36,48,60,72,96,120,144,192,240,288,384,480,576,768,1020,1536,2048,2304,2560,2816,3072,3328,3584,3840,4096]}'
+          - '{"mode":"VLLM_COMPILE","cudagraph_mode":"FULL_AND_PIECEWISE","cudagraph_capture_sizes":[6,12,18,24,30,36,48,60,72,96,120,144,192,240,288,384,480,576,768,1020,1536,2046,3072,4092,6144,8190],"decoder_replay_cudagraph_capture_sizes":[6,12,18,24,30,36,48,60,72,96,120,144,192,240,288,384,480,576,768,1020,1536,2048,2304,2560,2816,3072,3328,3584,3840,4096]}'
+          - '{"mode":"VLLM_COMPILE","cudagraph_mode":"FULL_AND_PIECEWISE","cudagraph_capture_sizes":[6,12,18,24,30,36,48,60,72,96,120,144,192,240,288,384,480,576,768,1020,1536,2046,3072,4092,6144,8190],"decoder_replay_cudagraph_capture_sizes":[6,12,18,24,30,36,48,60,72,96,120,144,192,240,288,384,480,576,768,1020,1536,2048,2304,2560,2816,3072,3328,3584,3840,4096]}'
+          - '{"mode":"VLLM_COMPILE","cudagraph_mode":"FULL_AND_PIECEWISE","cudagraph_capture_sizes":[6,12,18,24,30,36,48,60,72,96,120,144,192,240,288,384,480,576,768,1020,1536,2046,3072,4092],"decoder_replay_cudagraph_capture_sizes":[6,12,18,24,30,36,48,60,72,96,120,144,192,240,288,384,480,576,768,1020,1536,2048]}'
+        max-cudagraph-capture-size: [8190, 8190, 8190, 8190, 8190, 4092]
+        max-num-batched-tokens: [8192, 8192, 8192, 8192, 16384, 16384]
+  benchmark:
+    env:
+      CONC: ['8', '16', '32', '64', '128', '192']

inferencex-e2e/configs/nvidia-master.yaml

@@ -8475,7 +8475,7 @@ dsv41flash-fp4-gb200-sglang-agentic-dspark:
       - { tp: 2, ep: 1, kv-offloading: none, spec-decoding: mtp, conc-list: [1, 2, 4, 8, 16, 32, 64, 128], srt-recipe: benchmarks/single_node/srt-slurm-recipes/dsv41flash/sglang/gb200-fp4-mtp/agentic.yaml }
 
 dsv41flash-fp4-b300-vllm-agentic-dspark:
-  image: vllm/vllm-openai:nightly-ddd6fbca148a867aad1fcab7ec72f582b9977db4
+  image: vllm/vllm-openai:nightly-dev-x86_64-cu130-ac9126e58aa7
   model: deepseek-ai/DeepSeek-V4.1-Flash
   model-prefix: dsv41flash
   runner: cluster:b300-dsxe
@@ -8486,11 +8486,12 @@ dsv41flash-fp4-b300-vllm-agentic-dspark:
     agentic-coding:
     - dram-utilization: 0.80
       search-space:
-      # Engram weights use UVA DRAM; the KV cache stays GPU-resident. The B300
-      # script uses explicit piecewise CUDA graph capture tiers.
-      - { tp: 4, kv-offloading: none, spec-decoding: mtp, conc-list: [1, 2, 4, 8, 16, 32, 64, 128], srt-recipe: benchmarks/single_node/srt-slurm-recipes/dsv41flash/vllm/b300-fp4-mtp/agentic.yaml }
-      # Run the same B300 recipe on two GPUs.
-      - { tp: 2, kv-offloading: none, spec-decoding: mtp, conc-list: [2, 4, 8, 16, 32, 64, 128], srt-recipe: benchmarks/single_node/srt-slurm-recipes/dsv41flash/vllm/b300-fp4-mtp/agentic.yaml }
+      # Engram weights use UVA DRAM; the KV cache stays GPU-resident.
+      # TP4 covers low concurrency with FlashInfer attention and THP Engram tables.
+      - { tp: 4, kv-offloading: none, spec-decoding: mtp, conc-list: [1, 4, 8, 16], srt-recipe: benchmarks/single_node/srt-slurm-recipes/dsv41flash/vllm/b300-fp4-mtp/agentic.yaml }
+      # DEP2 is native TP1 x DP2 + EP2 with MegaMoE behind a consistent-hash vLLM
+      # Router; FlashInfer attention up to conc 64, MegaAttention above.
+      - { tp: 2, ep: 2, dp-attn: true, kv-offloading: none, spec-decoding: mtp, conc-list: [8, 16, 32, 64, 128, 192], router: { name: vllm-router, version: "0.1.14" }, srt-recipe: benchmarks/single_node/srt-slurm-recipes/dsv41flash/vllm/b300-fp4-mtp/agentic.yaml }
 
 dsv41flash-fp4-b200-vllm-agentic-dspark:
   image: vllm/vllm-openai:nightly-ddd6fbca148a867aad1fcab7ec72f582b9977db4

inferencex-e2e/docs/configuration-procedures.md

@@ -331,10 +331,10 @@ The B200 DSpark recipe uses the same minimum capture size and preserves the same
 The GB300 DSpark recipe uses the same minimum capture size and preserves the same workload settings.
 The H200 DSpark recipe uses the same minimum capture size and preserves the same workload settings.
 
-B300 uses the same minimum capture size at c1/c2/c4. Its c1 CI comparison reduced request ITL P90/P99 from 38.74/41.42 ms to 2.62/3.45 ms; c2/c4 require CI confirmation.
+The B300 DSpark recipe sets explicit capture sizes per point, described below.
 
 The AgentX-only `dsv41flash-fp4-<sku>-vllm-agentic-dspark` recipes use the per-SKU
-`image` pinned in [`nvidia-master.yaml`](../configs/nvidia-master.yaml) (originally `vllm/vllm-openai:deepseekv41-flash-0909`, which B300 still uses) at TP4 on Blackwell SKUs with native five-token DSpark,
+`image` pinned in [`nvidia-master.yaml`](../configs/nvidia-master.yaml) (originally `vllm/vllm-openai:deepseekv41-flash-0909`) at TP4 on Blackwell SKUs with native five-token DSpark,
 probabilistic drafting. Throughput uses the [committed golden AL](../infx/golden_al_distribution/dsv41flash_dspark.yaml) of 3.51 for thinking on and five draft tokens, with synthetic rejection sampling and adaptive verification disabled. Accuracy evals retain real block rejection and adaptive verification. `--engram-config '{"cpu_offload":true}'`
 stores Engram embedding tables in pinned host DRAM accessed through UVA;
 `kv-offloading: none` describes the separate, GPU-resident KV cache. MXFP4 expert
@@ -349,12 +349,11 @@ The recipe probes the serving port on the compute node and selects an available
 port if the preferred one is occupied. Serving, replay, metrics, and eval share
 that endpoint.
 
-The B300 entry also includes a TP2 variant at concurrency 2–128. Its dedicated
-script uses `FULL_AND_PIECEWISE` CUDA graphs with explicit capture-size sets ending
-at 2046 or 8190 tokens. It sets `--max-num-batched-tokens` to 2048 for concurrency
-1–4 and TP2 concurrency 128, and to 8192 otherwise; `--max-num-seqs` is 256. The
-TP2 concurrency-128 variant also sets `--gpu-memory-utilization 0.97`. Other SKUs
-continue to use the shared script.
+The B300 entry uses `vllm/vllm-openai:nightly-dev-x86_64-cu130-ac9126e58aa7` with FlashInfer
+autotuning. TP4 covers concurrency 1–16; DEP2 (TP1 x DP2 + EP2, DeepGEMM MegaMoE) covers 8–192
+behind a consistent-hash vLLM Router, switching to MegaAttention at 128 and above. All points use
+`FULL_AND_PIECEWISE` CUDA graphs sized in multiples of the six-token verification block.
+Other SKUs continue to use the shared script.
 
 The GB300 launcher allows 7200 seconds for engine readiness. In [run 34504969146](https://github.com/SemiAnalysisAI/InferenceX/actions/runs/34504969146), the Rust frontend exhausted its 3600-second deadline while the engine was still capturing graphs; model loading alone took 18–23 minutes. This extends startup time without changing the benchmark duration or decoding settings.