Repository navigation
Commit c9e1037
committed
refactor(srt): size the DSv4-Pro ATOM AgentX LMCache MP L1 from the DRAM budget
Main's C256 and C384 points of dsv4-fp4-mi355x-atom-agentic-mtp start an
LMCache MP server with a literal 1000 GB L1 and set TOTAL_CPU_DRAM_GB 1019
(dram-utilization 0.34). The L1 now takes '@dram.total-gb' and the binder
writes TOTAL_CPU_DRAM_GB. dram-utilization 0.3334 makes the budget 1000 GB,
so the bound L1 stays 1000 GB.
ATOM's lmcache.mp.max_pinned_state_bytes keeps its literal 1000000000000:
srtctl needs the extra-config JSON brace-doubled, and the binder resolves no
reference inside it.1 parent 3032f1a commit c9e1037
2 files changed
Lines changed: 8 additions & 8 deletions
File tree
- inferencex-e2e
- benchmarks/single_node/srt-slurm-recipes/dsv4/atom/mi355x-fp4-mtp
- configs
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
224 | 224 | | |
225 | 225 | | |
226 | 226 | | |
227 | | - | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
228 | 230 | | |
229 | 231 | | |
230 | 232 | | |
| |||
238 | 240 | | |
239 | 241 | | |
240 | 242 | | |
241 | | - | |
| 243 | + | |
242 | 244 | | |
243 | 245 | | |
244 | 246 | | |
| |||
280 | 282 | | |
281 | 283 | | |
282 | 284 | | |
283 | | - | |
284 | 285 | | |
285 | 286 | | |
286 | 287 | | |
| |||
297 | 298 | | |
298 | 299 | | |
299 | 300 | | |
300 | | - | |
| 301 | + | |
301 | 302 | | |
302 | 303 | | |
303 | 304 | | |
| |||
339 | 340 | | |
340 | 341 | | |
341 | 342 | | |
342 | | - | |
343 | 343 | | |
344 | 344 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
966 | 966 | | |
967 | 967 | | |
968 | 968 | | |
969 | | - | |
970 | | - | |
| 969 | + | |
| 970 | + | |
971 | 971 | | |
972 | 972 | | |
973 | 973 | | |
| |||
979 | 979 | | |
980 | 980 | | |
981 | 981 | | |
982 | | - | |
| 982 | + | |
983 | 983 | | |
984 | 984 | | |
985 | 985 | | |
| |||
0 commit comments