Skip to content

Commit efe2f87

Browse files
committed
refactor(recipes): select the Qwen3.5 GB300 FP8 AgentX midcurve points from the pareto bundles
of the pareto entries by naming those points' standalone recipes, which the bundles replace. Their rows select the matching pareto overrides, so each still runs exactly the pareto point's recipe rather than a copy that could drift; the DEP16 points keep disagg-dep16-mtp6-variants.yaml. RECIPES.md notes that an entry rerunning another entry's points selects its overrides.
1 parent 66e9cfa commit efe2f87

2 files changed

Lines changed: 6 additions & 6 deletions

File tree

‎inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/RECIPES.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@ qwen3.5/trtllm/gb300-fp4/agentx/disagg-variants.yaml
2020
- Workloads are `1k1k`, `8k1k`, or `agentx`. Existing bundles spanning several fixed sequence lengths use `fixed-seq-len`; keep their override selectors intact.
2121
- Use lowercase, hyphen-separated filenames beginning with `agg` or `disagg`. Include topology and the settings that distinguish sibling recipes, such as parallelism, batch size, concurrency, MTP, offload, or cache configuration. Avoid dates, numbered latency/throughput labels, and repeating the model or hardware already in the path.
2222
- In topology names, `1p4d` denotes prefill/decode worker counts, not necessarily physical nodes. Role-qualified `p-tp4` and `d-tp8` identify prefill/decode TP; `b` denotes batch size and `c` concurrency. The YAML is authoritative for runtime settings.
23-
- A master-config entry with two or more rows of one recipe design keeps them in one `<agg|disagg>[-<axis>]-variants.yaml`; the axis only tells bundles in one directory apart. `base` holds everything the points share, with `schema: 2` beside it, and each row gets one plain `override_<topology>_c<conc>` block (or a similarly point-identifying name, without dates or hardware) holding only its deltas. An override's `null` deletes the base key, so a null that a point needs, such as TensorRT-LLM's `cuda_graph_config:`, stays in `base`. Rows select it with `srt-recipe: <bundle>.yaml:override_<name>`, relative to `srt-recipe-dir`. Do not add `zip_override_*` blocks or share a bundle across entries, and keep one override per row: recipe references participate in eval grouping. Standalone files remain for single-row entries, adapted upstream recipes whose `# Source:` base materially differs, and points too different to share a base, such as a TP low-latency recipe beside wide-EP ones.
23+
- A master-config entry with two or more rows of one recipe design keeps them in one `<agg|disagg>[-<axis>]-variants.yaml`; the axis only tells bundles in one directory apart. `base` holds everything the points share, with `schema: 2` beside it, and each row gets one plain `override_<topology>_c<conc>` block (or a similarly point-identifying name, without dates or hardware) holding only its deltas. An override's `null` deletes the base key, so a null that a point needs, such as TensorRT-LLM's `cuda_graph_config:`, stays in `base`. Rows select it with `srt-recipe: <bundle>.yaml:override_<name>`, relative to `srt-recipe-dir`. Do not add `zip_override_*` blocks or share a bundle across entries, and keep one override per row: recipe references participate in eval grouping. An entry that reruns another entry's points selects that entry's overrides instead of copying them. Standalone files remain for single-row entries, adapted upstream recipes whose `# Source:` base materially differs, and points too different to share a base, such as a TP low-latency recipe beside wide-EP ones.
2424
- Update `srt-recipe-dir`, `srt-recipe`, and `eval-srt-recipe` references in master configs, launcher path rules, workflow filters, and local documentation together when moving a file. Preserve upstream source URLs as provenance and leave historical performance-changelog entries unchanged. No aliases for the old layout are provided.
2525

2626
Shared runtime assets stay under `configs/` beside the model directories; they are not standalone recipes. The four files in `configs/dsv4-moe-load-balancer-configs/` are copied verbatim from NVIDIA/srt-slurm commit `deb1dfd9934398664f92d194169c183e009da83b`, preserving the EPLB initial expert assignments formerly used by the DSV4 TRT recipes; no checked-in recipe currently references them. The srt driver ([`infx/launch/drivers/srt/checkout.py`](../../../infx/launch/drivers/srt/checkout.py)) stages them into the job checkout's `configs/` directory for the recipes' bind mounts. Keeping a recipe in this tree does not activate it; the master configs determine the benchmark matrix.

‎inferencex-e2e/configs/nvidia-master.yaml‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -9210,7 +9210,7 @@ qwen3.5-fp8-gb300-dynamo-sglang-agentic-pp-midcurve:
92109210
name: hicache
92119211
conc-list:
92129212
- 1000
9213-
srt-recipe: disagg-gb300-5p3d-pp4-dep4-c1000-mtp-hicache-nightly-20260913.yaml
9213+
srt-recipe: disagg-variants.yaml:override_5p3d_pp4_dep4_c1000
92149214
prefill:
92159215
num-worker: 5
92169216
tp: 1
@@ -9229,7 +9229,7 @@ qwen3.5-fp8-gb300-dynamo-sglang-agentic-pp-midcurve:
92299229
name: hicache
92309230
conc-list:
92319231
- 200
9232-
srt-recipe: disagg-gb300-1p1d-pp4-dep4-c200-mtp-hicache-nightly-20260913.yaml
9232+
srt-recipe: disagg-variants.yaml:override_1p1d_pp4_dep4_c200
92339233
prefill:
92349234
num-worker: 1
92359235
tp: 1
@@ -9320,7 +9320,7 @@ qwen3.5-fp8-gb300-dynamo-sglang-agentic-agg-midcurve:
93209320
conc-list:
93219321
- 44
93229322
num-nodes: 1
9323-
srt-recipe: agg-gb300-tp4-c44-b1-mtp-hicache-nightly-20260913.yaml
9323+
srt-recipe: agg-variants.yaml:override_tp4_c44_b1
93249324
worker:
93259325
num-worker: 1
93269326
tp: 4
@@ -9334,7 +9334,7 @@ qwen3.5-fp8-gb300-dynamo-sglang-agentic-agg-midcurve:
93349334
conc-list:
93359335
- 1
93369336
num-nodes: 1
9337-
srt-recipe: agg-gb300-tp4-c1-b1-mtp-hicache-nightly-20260913.yaml
9337+
srt-recipe: agg-variants.yaml:override_tp4_c1_b1
93389338
worker:
93399339
num-worker: 1
93409340
tp: 4
@@ -9348,7 +9348,7 @@ qwen3.5-fp8-gb300-dynamo-sglang-agentic-agg-midcurve:
93489348
conc-list:
93499349
- 1
93509350
num-nodes: 2
9351-
srt-recipe: agg-gb300-tp8-c1-b1-mtp-hicache-nightly-20260913.yaml
9351+
srt-recipe: agg-variants.yaml:override_tp8_c1_b1
93529352
worker:
93539353
num-worker: 1
93549354
tp: 8

0 commit comments

Comments
 (0)