Skip to content

Commit c9e1037

Browse files
committed
refactor(srt): size the DSv4-Pro ATOM AgentX LMCache MP L1 from the DRAM budget
Main's C256 and C384 points of dsv4-fp4-mi355x-atom-agentic-mtp start an LMCache MP server with a literal 1000 GB L1 and set TOTAL_CPU_DRAM_GB 1019 (dram-utilization 0.34). The L1 now takes '@dram.total-gb' and the binder writes TOTAL_CPU_DRAM_GB. dram-utilization 0.3334 makes the budget 1000 GB, so the bound L1 stays 1000 GB. ATOM's lmcache.mp.max_pinned_state_bytes keeps its literal 1000000000000: srtctl needs the extra-config JSON brace-doubled, and the binder resolves no reference inside it.
1 parent 3032f1a commit c9e1037

2 files changed

Lines changed: 8 additions & 8 deletions

File tree

‎inferencex-e2e/benchmarks/single_node/srt-slurm-recipes/dsv4/atom/mi355x-fp4-mtp/agentic.yaml‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -224,7 +224,9 @@ override_tp8_c128_dep:
224224

225225
# DRAM variants: ATOM's lmcache_mp tier backed by srtctl's lmcache-server
226226
# service (one LMCache server per node on port 8750, run from the LMCache build in
227-
# the ATOM image) within the 1019 GB budget (dram-utilization 0.34).
227+
# the ATOM image) whose L1 is the point's DRAM budget (1000 GB at dram-utilization
228+
# 0.3334). max_pinned_state_bytes repeats it in bytes: the binder resolves no
229+
# reference inside the brace-doubled JSON.
228230
override_tp8_c256_dep_lmcache:
229231
services:
230232
- name: lmcache
@@ -238,7 +240,7 @@ override_tp8_c256_dep_lmcache:
238240
- --supported-transfer-mode
239241
- lmcache_driven
240242
- --l1-size-gb
241-
- '1000'
243+
- '@dram.total-gb'
242244
- --l1-use-lazy
243245
- --l1-read-ttl-seconds
244246
- '900'
@@ -280,7 +282,6 @@ override_tp8_c256_dep_lmcache:
280282
env:
281283
CONC: '256'
282284
KV_OFFLOADING: 'dram'
283-
TOTAL_CPU_DRAM_GB: '1019'
284285
AIPERF_HTTP_X_DYNAMO_SESSION_ID_FROM_CORRELATION_ID: '1'
285286
AGENTIC_WARMUP_GRACE_PERIOD: '3600'
286287

@@ -297,7 +298,7 @@ override_tp8_c384_dep_lmcache:
297298
- --supported-transfer-mode
298299
- lmcache_driven
299300
- --l1-size-gb
300-
- '1000'
301+
- '@dram.total-gb'
301302
- --l1-use-lazy
302303
- --l1-read-ttl-seconds
303304
- '900'
@@ -339,6 +340,5 @@ override_tp8_c384_dep_lmcache:
339340
env:
340341
CONC: '384'
341342
KV_OFFLOADING: 'dram'
342-
TOTAL_CPU_DRAM_GB: '1019'
343343
AIPERF_HTTP_X_DYNAMO_SESSION_ID_FROM_CORRELATION_ID: '1'
344344
AGENTIC_WARMUP_GRACE_PERIOD: '3600'

‎inferencex-e2e/configs/amd-master.yaml‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -966,8 +966,8 @@ dsv4-fp4-mi355x-vllm-agentic-mtp:
966966
# DeepSeek-V4-Pro-0813 FP4 AgentX on one MI355X node using ATOM DSpark K6.
967967
# Both throughput bands use thinking_on golden AL 3.77; eval uses real
968968
# acceptance. Keep the historical config key for changelog compatibility.
969-
# C256 and C384 offload KV to an LMCache MP server whose 1000 GB L1 fits the
970-
# 1019 GB budget of dram-utilization 0.34.
969+
# C256 and C384 offload KV to an LMCache MP server whose L1 is the 1000 GB
970+
# budget of dram-utilization 0.3334.
971971
dsv4-fp4-mi355x-atom-agentic-mtp:
972972
image: rocm/atom-dev:nightly_202610071700
973973
model: deepseek-ai/DeepSeek-V4-Pro-0813
@@ -979,7 +979,7 @@ dsv4-fp4-mi355x-atom-agentic-mtp:
979979
srt-recipe-dir: dsv4/atom/mi355x-fp4-mtp
980980
scenarios:
981981
agentic-coding:
982-
- dram-utilization: 0.34
982+
- dram-utilization: 0.3334
983983
search-space:
984984
- { tp: 8, ep: 1, dp-attn: false, kv-offloading: none, spec-decoding: draft_model, conc-list: [1, 2, 4, 8, 16], srt-recipe: agentic.yaml }
985985
- { tp: 8, ep: 8, dp-attn: true, kv-offloading: none, spec-decoding: draft_model, conc-list: [48, 64, 96, 128], srt-recipe: agentic.yaml }

0 commit comments

Comments
 (0)