Skip to content

Commit b09359b

Browse files
authored
refactor(matrix): select srt-slurm recipes with srt-recipe-dir and srt-recipe (#3805)
* refactor(matrix): select srt-slurm recipes with srt-recipe-dir and srt-recipe Master configs name each row's recipe with srt-recipe (and eval-srt-recipe) relative to a per-entry srt-recipe-dir instead of CONFIG_FILE and EVAL_CONFIG_FILE additional-settings. Validation requires them on every srt-slurm row, checks the file and selected variant exist at load time, and rejects the old settings; llmd-vllm entries keep CONFIG_FILE. The generator emits project-relative paths, counts nodes from the row's recipe, and the launcher reads SRT_RECIPE/EVAL_SRT_RECIPE, staging the same recipes/<path>:<selector> argument for srtctl. Also drops the unread SYNTHETIC_ACCEPTANCE* and SLURM_PARTITION settings. * chore(recipes): drop stale CONFIG_FILE header comments Master rows now select bundle variants with srt-recipe, so the recipe headers no longer describe how to pick one. * fix(matrix): resolve srt recipes under the loaded config's project Recipe existence, selector checks and recipe node counts read files from the project that holds the loaded configs/ (or the root a caller passes) instead of the tooling checkout's repository_root(). The generator CLI and planner pass config_root(config files), Klaud live families pass the checkout root, and Klaud's canonical matrix fetches the candidate's recipes from the candidate head. * test(operatorx): give the generator fixture a project-layout srt recipe Master configs now name an srt-recipe resolved under the project that holds configs/, so the fixture places its master in configs/ beside a recipe. * fix(configs): select the DSv4.1-Flash ATOM AgentX recipe with srt-recipe-dir srt-recipe paths, which validation rejects now that every srt-slurm entry needs srt-recipe-dir. The entry sets srt-recipe-dir and its rows select agentic.yaml; the generated rows are unchanged. * chore(recipes): drop stale CONFIG_FILE headers from the DSv4.1-Flash GB300 AgentX recipes #3598 added the GB300 Dynamo+SGLang recipes with headers that say to select a variant with CONFIG_FILE. Master rows select them with srt-recipe now, so the headers stop at the variant layout like the other AgentX bundles. * refactor(matrix): deslop srt-recipe selection Set an AgentX row's srt-recipe fields once after the topology branch, and fetch each recipe file a Klaud family references once rather than once per row. * test: deslop srt-recipe selection tests The installed-package test no longer checks recipe node counts, which take an explicit root since srt-recipe and so no longer exercise caller-repository resolution. Drop unused monkeypatch fixtures and a history comment. * docs: deslop srt-recipe selection guidance Show srt-recipe in the CONFIGS.md search-space example, shorten its selection note, name srt-slurm recipes in the revision snapshot description, and shorten the debug-runs recipe path. * fix(configs): select each GB200 DSR1 FP8 low-latency variant explicitly The low-latency row named disagg-tp8-stp-variants.yaml without a selector, so one job submitted all three zip variants under a 4-node, 1-decode-worker label. Each variant now runs only the concurrencies it is tuned for, with its selector and its 7 TP8 decode workers on 16 nodes: zip_override_lowlat[0] (c8) at 4 and 8, zip_override_lowlat[1] (c16-32) at 16. zip_override_lowlat[2] is tuned for concurrency 64, which the entry never ran, so it has no row. * fix(configs): select the Qwen3.8 and Qwen3.5 GB300 AgentX recipes with srt-recipe-dir #3816 and #3840 added qwen3.8next-fp4-gb300-sglang-agentic-mtp and the Qwen3.5 FP8 GB300 midcurve entries with a project-relative srt-recipe path and CONFIG_FILE settings, which validation rejects now that every srt-slurm entry needs srt-recipe-dir. The entries set srt-recipe-dir and their rows select the same recipes and DEP16 overrides; the generated rows are unchanged. * fix(matrix): report a malformed srt-recipe as a validation error A recipe that fails to parse raised a raw yaml.YAMLError out of validate_master_config, so Klaud's canonical_matrix crashed on a malformed candidate recipe instead of rejecting it like other bad srt-recipe references. It now raises the same ValueError, naming the recipe file. * fix(configs): select the Qwen3.5 GB200 FP4 V2 MTP recipes with srt-recipe-dir #2594 added qwen3.5-fp4-gb200-dynamo-sglang-mtp with CONFIG_FILE settings, which validation rejects now that every srt-slurm entry needs srt-recipe-dir. The entry sets srt-recipe-dir and each row selects the same recipe; the generated rows are unchanged.
1 parent ca7b39e commit b09359b

67 files changed

Lines changed: 1843 additions & 1743 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.agents/skills/debug-runs/SKILL.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -109,7 +109,7 @@ Steps:
109109
the canvas, then SSH in with `ssh -A` when a jumpbox or agent forwarding is involved.
110110
2. Reproduce the exact benchmark the launcher runs. Single-node jobs go through the srt driver
111111
of `python -m infx.launch run` (`inferencex-e2e/infx/launch/drivers/srt/`): it binds the
112-
master row's `srt-recipe:` (`inferencex-e2e/benchmarks/single_node/srt-slurm-recipes/<model>/<engine>/<sku>-<precision>[-mtp]/<scenario>.yaml`)
112+
matrix row's `srt-recipe` (`inferencex-e2e/benchmarks/single_node/srt-slurm-recipes/<srt-recipe-dir>/<srt-recipe>`)
113113
with `python -m infx.srt_slurm.single_node prepare`, renders the job-local `srtslurm.yaml`
114114
from the cluster's record, then submits it through srtctl. Read the recipe for the image
115115
(`model.container`), server args and env, and the cluster record for mounts and the job env

‎.claude/commands/add-model-hardware.md‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -75,7 +75,8 @@ This research directly feeds Step 2 (recipe args/env) and Step 3 (search space).
7575
1. `inferencex-e2e/benchmarks/single_node/srt-slurm-recipes/<model-prefix>/<engine>/<sku>-<precision>[-mtp]/8k1k.yaml`
7676
(an `agentic.yaml` beside it for AgentX)
7777
2. an entry in either master config, **`inferencex-e2e/configs/nvidia-master.yaml`** (b*/h*/gb* SKUs) or
78-
**`inferencex-e2e/configs/amd-master.yaml`** (mi* SKUs), with `srt-recipe:` on every search-space row
78+
**`inferencex-e2e/configs/amd-master.yaml`** (mi* SKUs), with `srt-recipe-dir:` (the directory above,
79+
relative to `srt-slurm-recipes/`) on the entry and `srt-recipe:` (the file name) on every search-space row
7980
3. a `inferencex-e2e/perf-changelog.yaml` entry (this diff vs main is what selects the sweep)
8081

8182
## Step 1 — branch + find the sibling to copy

‎.claude/commands/nuke.md‎

Lines changed: 10 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -113,18 +113,22 @@ recipe each master search-space row names in `srt-recipe:`, and
113113
# Usage: edit_recipe_container.py <master_yaml> <new_image> <key1> [key2 ...]
114114
import os, re, sys
115115
f, new_image, keys = sys.argv[1], sys.argv[2], sys.argv[3:]
116-
# srt-recipe paths are relative to inferencex-e2e/, the parent of configs/
117-
root = os.path.dirname(os.path.dirname(os.path.abspath(f)))
116+
# srt-recipe names a file under the key's srt-recipe-dir, below this root
117+
root = os.path.join(os.path.dirname(os.path.dirname(os.path.abspath(f))), "benchmarks/single_node/srt-slurm-recipes")
118118
master = open(f).read().split('\n')
119119
recipes = set()
120120
for key in keys:
121121
kre = re.compile(r'^' + re.escape(key) + r':\s*$')
122122
start = next((i for i,l in enumerate(master) if kre.match(l)), None)
123123
if start is None: sys.exit(f"ERROR: key not found: {key}")
124+
directory, names = None, set()
124125
for j in range(start+1, len(master)):
125126
if re.match(r'^[A-Za-z0-9._-]+:\s*$', master[j]): break # next top-level key
126-
recipes.update(re.findall(r'srt-recipe:\s*([^\s,}]+)', master[j]))
127-
if not recipes: sys.exit(f"ERROR: no srt-recipe for keys {keys}")
127+
d = re.match(r'^\s+srt-recipe-dir:\s*(\S+)\s*$', master[j])
128+
if d: directory = d.group(1)
129+
names.update(re.findall(r'(?<![\w-])srt-recipe:\s*([^\s,}:]+)', master[j]))
130+
if directory is None or not names: sys.exit(f"ERROR: no srt-recipe-dir/srt-recipe for key {key}")
131+
recipes.update(os.path.join(directory, n) for n in names)
128132
for r in sorted(os.path.join(root, x) for x in recipes):
129133
lines = open(r).read().split('\n')
130134
in_model, hit = False, False
@@ -139,9 +143,8 @@ for r in sorted(os.path.join(root, x) for x in recipes):
139143
open(r, 'w').write('\n'.join(lines))
140144
```
141145

142-
If `grep -rn '<recipe path>' inferencex-e2e/configs/*-master.yaml` shows a recipe is also
143-
referenced by a key outside this family, stop and ask the user: bumping it would
144-
break that other key's `model.container == image` check.
146+
If another key's `srt-recipe-dir` plus `srt-recipe` resolves to the same recipe file, stop and ask the user:
147+
bumping it would break that other key's `model.container == image` check.
145148

146149
For each family, run strictly sequentially because git checkouts can't be parallel:
147150

‎.github/workflows/benchmark-multinode-tmpl.yml‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -123,6 +123,8 @@ env:
123123
HF_TOKEN: ${{ secrets.INFERENCEX_OFFICIAL_RO_HF_TOKEN }}
124124
EXP_NAME: ${{ fromJSON(inputs.config).exp-name }}
125125
RECIPE_FINGERPRINT: ${{ fromJSON(inputs.config)['recipe-fingerprint'] || '' }}
126+
SRT_RECIPE: ${{ fromJSON(inputs.config)['srt-recipe'] || '' }}
127+
EVAL_SRT_RECIPE: ${{ fromJSON(inputs.config)['eval-srt-recipe'] || '' }}
126128
IMAGE: ${{ fromJSON(inputs.config).image }}
127129
THINKING_MODE: thinking_on
128130
MODEL_PREFIX: ${{ fromJSON(inputs.config).model-prefix }}

‎inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/RECIPES.md‎

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -20,8 +20,8 @@ qwen3.5/trtllm/gb300-fp4/agentx/disagg-variants.yaml
2020
- Workloads are `1k1k`, `8k1k`, or `agentx`. Existing bundles spanning several fixed sequence lengths use `fixed-seq-len`; keep their override selectors intact.
2121
- Use lowercase, hyphen-separated filenames beginning with `agg` or `disagg`. Include topology and the settings that distinguish sibling recipes, such as parallelism, batch size, concurrency, MTP, offload, or cache configuration. Avoid dates, numbered latency/throughput labels, and repeating the model or hardware already in the path.
2222
- In topology names, `1p4d` denotes prefill/decode worker counts, not necessarily physical nodes. Role-qualified `p-tp4` and `d-tp8` identify prefill/decode TP; `b` denotes batch size and `c` concurrency. The YAML is authoritative for runtime settings.
23-
- Name override bundles `*-variants.yaml`. Multi-node AgentX recipes that differ only per configuration share one bundle per master-config entry, usually `agg-variants.yaml` or `disagg-variants.yaml`: `base` holds the shared settings and each former recipe becomes a named `override_<name>` block holding only its differences (plain overrides, not `zip_override_*`). Master entries select one with `CONFIG_FILE=recipes/<dir>/<bundle>.yaml:override_<name>`. Recipes read as text by a launcher, such as power recipes with top-level `telemetry:`, stay standalone. Keep distinct sweep entry files separate even when their contents match: recipe paths participate in eval grouping. The Qwen3.5 `*-stp-sweep.yaml` and `*-mtp-sweep.yaml` pair preserves that existing distinction.
24-
- Update `CONFIG_FILE` and `EVAL_CONFIG_FILE` references in active and deprecated master configs, launcher path rules, workflow filters, and local documentation together when moving a file. Preserve upstream source URLs as provenance and leave historical performance-changelog entries unchanged. No aliases for the old layout are provided.
23+
- Name override bundles `*-variants.yaml`. Multi-node AgentX recipes that differ only per configuration share one bundle per master-config entry, usually `agg-variants.yaml` or `disagg-variants.yaml`: `base` holds the shared settings and each former recipe becomes a named `override_<name>` block holding only its differences (plain overrides, not `zip_override_*`). Master rows select one with `srt-recipe: <bundle>.yaml:override_<name>`, relative to the entry's `srt-recipe-dir`. Recipes read as text by a launcher, such as power recipes with top-level `telemetry:`, stay standalone. Keep distinct sweep entry files separate even when their contents match: recipe paths participate in eval grouping. The Qwen3.5 `*-stp-sweep.yaml` and `*-mtp-sweep.yaml` pair preserves that existing distinction.
24+
- Update `srt-recipe-dir`, `srt-recipe`, and `eval-srt-recipe` references in master configs, launcher path rules, workflow filters, and local documentation together when moving a file. Preserve upstream source URLs as provenance and leave historical performance-changelog entries unchanged. No aliases for the old layout are provided.
2525

2626
Shared runtime assets stay under `configs/` beside the model directories; they are not standalone recipes. The four files in `configs/dsv4-moe-load-balancer-configs/` are copied verbatim from NVIDIA/srt-slurm commit `deb1dfd9934398664f92d194169c183e009da83b`, preserving the EPLB initial expert assignments formerly used by the DSV4 TRT recipes; no checked-in recipe currently references them. The srt driver ([`infx/launch/drivers/srt/checkout.py`](../../../infx/launch/drivers/srt/checkout.py)) stages them into the job checkout's `configs/` directory for the recipes' bind mounts. Keeping a recipe in this tree does not activate it; the master configs determine the benchmark matrix.
2727

@@ -41,7 +41,8 @@ Recipes use `schema: 2`, `engine`, and `roles`. Each worker role owns its node c
4141
| `roles.prefill.args.ep-size` (SGLang) | `prefill.ep` |
4242
| `roles.prefill.args.enable-dp-attention` | `prefill.dp-attn` |
4343
| `benchmark.concurrencies` | `conc-list` |
44-
| Recipe path, optionally with an override selector | `additional-settings: CONFIG_FILE=recipes/...yaml` |
44+
| Recipe directory, relative to this tree | entry-level `srt-recipe-dir` |
45+
| Recipe file, optionally with a `base`, `override_<name>`, or `zip_override_<name>[<index>]` selector | search-space `srt-recipe`; `eval-srt-recipe` for eval-only real verification |
4546

4647
Keep the recipe and master configuration synchronized. The launcher executes the recipe; the master configuration supplies result labels and scheduling metadata. For aggregate recipes use `roles.agg`; `roles.decode.nodes: colocate` shares prefill nodes and contributes no additional worker nodes to scheduling.
4748

‎inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/dsv4/sglang/b200-fp4/agentx/agg-variants.yaml‎

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,5 @@
11
# AgentX dsv4 sglang b200-fp4 recipes: shared settings in base, one
2-
# override per benchmark configuration. Select one with
3-
# CONFIG_FILE=recipes/dsv4/sglang/b200-fp4/agentx/agg-variants.yaml:override_<name>.
2+
# override per benchmark configuration.
43

54
schema: 2
65

‎inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/dsv4/sglang/b200-fp4/agentx/disagg-variants.yaml‎

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,5 @@
11
# AgentX dsv4 sglang b200-fp4 recipes: shared settings in base, one
2-
# override per benchmark configuration. Select one with
3-
# CONFIG_FILE=recipes/dsv4/sglang/b200-fp4/agentx/disagg-variants.yaml:override_<name>.
2+
# override per benchmark configuration.
43

54
schema: 2
65

‎inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/dsv4/sglang/gb300-fp4/agentx/agg-variants.yaml‎

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,5 @@
11
# AgentX dsv4 sglang gb300-fp4 recipes: shared settings in base, one
2-
# override per benchmark configuration. Select one with
3-
# CONFIG_FILE=recipes/dsv4/sglang/gb300-fp4/agentx/agg-variants.yaml:override_<name>.
2+
# override per benchmark configuration.
43

54
schema: 2
65

‎inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/dsv4/sglang/gb300-fp4/agentx/disagg-variants.yaml‎

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,5 @@
11
# AgentX dsv4 sglang gb300-fp4 recipes: shared settings in base, one
2-
# override per benchmark configuration. Select one with
3-
# CONFIG_FILE=recipes/dsv4/sglang/gb300-fp4/agentx/disagg-variants.yaml:override_<name>.
2+
# override per benchmark configuration.
43

54
schema: 2
65

‎inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/dsv4/trtllm/gb300-fp4/agentx/disagg-variants.yaml‎

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,5 @@
11
# AgentX dsv4 trtllm gb300-fp4 recipes: shared settings in base, one
2-
# override per benchmark configuration. Select one with
3-
# CONFIG_FILE=recipes/dsv4/trtllm/gb300-fp4/agentx/disagg-variants.yaml:override_<name>.
2+
# override per benchmark configuration.
43

54
schema: 2
65

0 commit comments

Comments
 (0)