Skip to content

Commit c4ea990

Browse files
committed
Merge remote-tracking branch 'origin/fix/single-node-dram-mem' into validate/refactor-stack
#3804 restores h100-dgxc to 2,063,837 MiB and main's H100 recipe values. Keep the stack's '@dram' H100 recipes and its MiniMax-M3 H100 dram-utilization 0.4725 (the ~119 GiB Mooncake segment), so every H100 bound recipe matches refactor/recipe-bundles again.
2 parents a6ab0ec + ccc8b87 commit c4ea990

3 files changed

Lines changed: 6 additions & 6 deletions

File tree

‎inferencex-e2e/configs/CONFIGS.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -195,9 +195,9 @@ schema; unknown keys fail.
195195
master config's `cluster:<id>` label names; broad SKU labels use the GPU family when its
196196
clusters agree. Agentic master configs must use a `cluster:<id>` label; their DRAM
197197
KV-offload matrices combine both fields with `dram-utilization` into `total-cpu-dram-gb`.
198-
- `available-cpu-dram-mib` is the host DRAM, in MiB, a benchmark job can use on a node:
199-
the lower of Slurm's `RealMemory - MemSpecLimit` and the node's `MemTotal` less 16 GiB for
200-
the OS, over the cluster's GPU nodes. Omit it where unmeasured. When
198+
- `available-cpu-dram-mib` is the host DRAM, in MiB, a benchmark job can use on a node.
199+
Measure it as the lower of Slurm's `RealMemory - MemSpecLimit` and the node's `MemTotal`
200+
less 16 GiB for the OS, over the cluster's GPU nodes; omit it where unmeasured. When
201201
`slurm.srt-slurm.single-node-exclusive` is true (the default), a single-node srt-slurm job
202202
requests `--mem=0`, all the memory Slurm has on the node, and its recipe's host KV pools
203203
must fit the node. Otherwise the job requests, and its srun step is capped at, the serving

‎inferencex-e2e/configs/nvidia-master.yaml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -5381,8 +5381,8 @@ minimaxm3-fp8-h100-vllm-agentic-mtp:
53815381
scenarios:
53825382
agentic-coding:
53835383
# The fast sweep places the resident HBM cliff between c5 and c6.
5384-
# Sizes the 105 GiB Mooncake segment per GPU; weights and page cache use the rest.
5385-
- dram-utilization: 0.449
5384+
# Sizes the ~119 GiB Mooncake segment per GPU; weights and page cache use the rest.
5385+
- dram-utilization: 0.4725
53865386
search-space:
53875387
- { tp: 8, spec-decoding: mtp, kv-offloading: none, conc-list: [1, 2, 3, 4, 5], srt-recipe: agentic.yaml }
53885388
- { tp: 8, spec-decoding: mtp, kv-offloading: dram, kv-offload-backend: { name: mooncake, version: "0.3.11.post1" }, conc-list: [6, 8], srt-recipe: agentic.yaml }

‎inferencex-e2e/configs/runners.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -401,7 +401,7 @@ clusters:
401401
single-node-exclusive: false
402402
h100-dgxc:
403403
gpus-per-node: 8
404-
available-cpu-dram-mib: 1_915_904
404+
available-cpu-dram-mib: 2_063_837
405405
arch: x86_64
406406
models:
407407
entries:

0 commit comments

Comments
 (0)