Tune GLM-5.2 FP8 B300 SGLang AgentX recipes

Author: hshrivastava-droid

inferencex-e2e/benchmarks/single_node/srt-slurm-recipes/glm5.2/sglang/b300-fp8-mtp/agentic.yaml

@@ -7,7 +7,7 @@ base:
   name: glm5.2-fp8-b300-sglang-agentic
   model:
     path: hf:zai-org/GLM-5.2-FP8
-    container: lmsysorg/sglang:nightly-dev-cu13-20260908-20ca564b
+    container: lmsysorg/sglang@sha256:2d53500cb72184020c59fe16aab3512d53b1aa8ee6195bfb2aaf5855d58099df
     precision: fp8
   resources:
     gpu_type: b300
@@ -70,14 +70,15 @@ base:
       KV_OFFLOADING: dram
       TOTAL_CPU_DRAM_GB: '2399'
 
-# One variant per point: admission is 2x CONC and the graph batch matches it,
-# capped at 64.
+# Tune admission and MTP depth per point; keep graph maxima at 2x CONC.
 override_c1:
   roles:
     agg:
       args:
         max-running-requests: 2
         cuda-graph-max-bs: 2
+        speculative-num-steps: 6
+        speculative-num-draft-tokens: 7
   benchmark:
     env:
       CONC: '1'
@@ -86,8 +87,10 @@ override_c4:
   roles:
     agg:
       args:
-        max-running-requests: 8
+        max-running-requests: 3
         cuda-graph-max-bs: 8
+        speculative-num-steps: 4
+        speculative-num-draft-tokens: 5
   benchmark:
     env:
       CONC: '4'
@@ -106,8 +109,10 @@ override_c12:
   roles:
     agg:
       args:
-        max-running-requests: 24
+        max-running-requests: 5
         cuda-graph-max-bs: 24
+        speculative-num-steps: 5
+        speculative-num-draft-tokens: 6
   benchmark:
     env:
       CONC: '12'
@@ -116,8 +121,12 @@ override_c16:
   roles:
     agg:
       args:
-        max-running-requests: 32
+        max-running-requests: 7
         cuda-graph-max-bs: 32
+        speculative-num-steps: 4
+        speculative-num-draft-tokens: 5
+        prefill-decode-interval: 8
+        hicache-write-policy: write_through
   benchmark:
     env:
       CONC: '16'

inferencex-e2e/configs/nvidia-master.yaml

@@ -7368,25 +7368,10 @@ glm5.2-fp4-b300-sglang-agentic-mtp:
       search-space:
       - { tp: 8, spec-decoding: mtp, kv-offloading: dram, kv-offload-backend: { name: hicache }, conc-list: [1, 4, 8, 12, 16, 20, 24, 28, 32, 40, 48], srt-recipe: benchmarks/single_node/srt-slurm-recipes/glm5.2/sglang/b300-fp4-mtp/agentic.yaml }
 
-# First GLM-5.2 FP8 AgentX recipe on B300, the FP8 precision sibling of
-# glm5.2-fp4-b300-sglang-agentic-mtp and the B300 counterpart of
-# glm5.2-fp8-b200-sglang-agentic-mtp (#2863). Same spec-decode-only shape per
-# the AgentX policy (docs/MODELS.md): SGLang EAGLE off GLM-5.2's built-in nextn head
-# (num-steps 3, eagle-topk 1, 4 draft tokens = 3 speculative tokens) with
-# acceptance pinned to the golden AL 2.99 (infx/golden_al_distribution/glm5.2_mtp.yaml,
-# thinking_on, K=3), which was measured on glm-5.2-fp8, i.e. on this checkpoint.
-# Pinned to the 2026-09-08 cu13 dev nightly (the first that carries sgl-project/sglang#38318, the guard for the EAGLE DSA fp8 read-door crash seen on the 2026-09-07 build), the same tag the B200/B300 GLM-5.2
-# and Qwen3.5 SGLang AgentX recipes moved to that day. Runs on cluster:b300-dsxe
-# (cluster:b300-nv was retired in #2826).
-#
-# Same single arm and concurrency grid as the NVFP4 B300 sibling: cookbook
-# low-latency TP8 with a 270 GB/rank HiCache host-DRAM target pool, conc
-# [1, 4, 8, 12, 16] (steps of at least 2, hard stop at 16). TP8-only for
-# memory as well as comparability -- the ~756 GB block-FP8 checkpoint needs
-# ~94.5 GB/GPU across 8 B300s; the fp8 KV pool that remains inside
-# --mem-fraction-static 0.85 is ~150 GB/GPU against ~187 GB/GPU on NVFP4.
+# GLM-5.2 FP8 AgentX on B300: TP8, 270 GB/rank HiCache.
+# MTP depth and admission caps are set per concurrency.
 glm5.2-fp8-b300-sglang-agentic-mtp:
-  image: lmsysorg/sglang:nightly-dev-cu13-20260908-20ca564b
+  image: lmsysorg/sglang@sha256:2d53500cb72184020c59fe16aab3512d53b1aa8ee6195bfb2aaf5855d58099df
   model: zai-org/GLM-5.2-FP8
   model-prefix: glm5.2
   runner: cluster:b300-dsxe

inferencex-e2e/perf-changelog.yaml

@@ -9410,3 +9410,12 @@
     - "TP4 concurrency 32 sets ATOM_PREFILL_DECODE_INTERVAL=16: after each prefill, ATOM runs at least 16 decode passes and batches the prefills that arrive in the meantime (ATOM's PrefillDelayer, off by default). Prefill no longer interrupts decode as often, so P90 interactivity rises while TTFT rises. Local 3600 s replay: 108.2 -> 150.9 tok/s/user, ITL p90 9.24 -> 6.63 ms, TTFT p50 0.51 -> 1.14 s, throughput unchanged."
     - "TP2 concurrency 64 sets ATOM_PREFILL_DECODE_INTERVAL=4. At this load each batched prefill forward serves several requests, so throughput rises as well as interactivity. Local 3600 s replay: 30.3 -> 38.8 tok/s/user, 105.4k -> 115.2k tok/s/GPU, 6487 -> 6941 requests, TTFT p90 2.3 -> 6.4 s. All other points are unchanged."
   pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/3603
+
+- config-keys:
+    - glm5.2-fp8-b300-sglang-agentic-mtp
+  scenario-type:
+    - agentic-coding
+  description:
+    - "Tune GLM-5.2 FP8 B300 SGLang MTP depth, admission caps, and HiCache scheduling per concurrency."
+    - "Pin the SGLang image by digest."
+  pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/0