References resolve to integers, or to decimal strings in env values and argument lists (e.g. after `--l1-size-gb`), including whole values inside JSON object strings such as `kv-transfer-config`. Per-rank pools take the `per-gpu` sizes, and Mooncake `global_segment_size` takes bytes because it reads `GB` as GiB. Unknown names, embedded references and references on a point without a DRAM budget fail. SimpleCPU, LMCache CPU and `--l1-size-gb` sizes must be references; measured HiCache, TRT-LLM host-cache and Mooncake sizes may stay literal if they fit the node's share. When a backend allocates several pools from the budget, such as the KV and Mamba HiCache pools of a hybrid model, the master's `dram-utilization` sizes one of them.
0 commit comments