Skip to content

refactor(recipes): fold per-point recipe families into one bundle per config key - #3824

Draft
adibarra wants to merge 7 commits into
refactor/srt-bound-versionsfrom
refactor/recipe-bundles
Draft

adibarra wants to merge 7 commits into
refactor/srt-bound-versionsfrom
refactor/recipe-bundles

Conversation

@adibarra

@adibarra adibarra commented Oct 8, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

  • Fix (6f82f4e62): the launcher's job edits (job name, health floor, dist-timeout) now run on the bound concrete recipe. Before, they edited the fragment as text, so bundle variants silently kept the recipe's own name, not inferencex-<runner>, and cleanup could not find them. Existing bundle rows in 39 config keys now submit as inferencex-<runner>. Health floor and dist-timeout are unchanged for them.
  • Consolidation (76e678bed): a master config key with 2+ rows now uses one *-variants.yaml bundle. base holds what the points share, and each row gets one override_<topology>_c<conc> with only its deltas and its old job name. The convention is written in RECIPES.md.
Before After
AgentX (all 10 keys) 50 files, 7294 lines 10 bundles, 2141 lines
Fixed-sequence (17 of 26 multi-file keys) 184 files, 23370 lines 17 bundles, 7911 lines
Multi-node recipe files 369 162

9 fixed-sequence keys stay standalone because they mix recipe designs, e.g. a TP low-latency recipe next to wide-EP ones: 15-30% of keys differ per point, versus 9% or less in every converted key.

Verification

  • Same recipes: for each of the 234 moved rows, with power off and on, the bundle variant chosen by the composer equals the old standalone recipe as a full dict.
  • Matrix: full-sweep (1767 rows) and --all-evals (1797) differ from the base only in srt-recipe strings. Concrete recipes are equal for all 513 moved matrix rows. Their fingerprints change because srt-recipe is part of the row.
  • infx generate: same output for the 17 fixed-sequence keys.
  • Launcher harness: the 500 launches of moved rows load identical srtctl-resolved recipes. The 421 launches of existing bundle rows differ only in name, from the fix.
  • Suite: the full CI pytest command passes 2319 tests, and ruff is clean.

The pre-existing dsr1-fp8-gb200-dynamo-sglang row that selects a whole 3-variant zip bundle is left as is.

@github-actions

github-actions Bot commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

Thanks for the contribution!

  • Review: If this PR changes files owned by someone other than a repository admin or @SemiAnalysisAI/core, ask one eligible CODEOWNER to complete the latest PR_REVIEW_CHECKLIST.md before contacting a core maintainer on Slack. Follow the template exactly, including As a PR reviewer and CODEOWNER, I have reviewed this and have, so sign-off verification triggers.
  • PR verification: Sweeps run only when the PR appends an inferencex-e2e/perf-changelog.yaml entry and carries exactly one primary label: full-sweep-fail-fast (strongly recommended; canary plus per-matrix fail-fast), full-sweep-enabled (canary; matrix jobs continue after a failure), or non-canary-full-sweep-enabled (no canary or fail-fast). The modifiers all-evals, evals-only, and agentx-fast require a primary label. On fork PRs, a maintainer applies the label. See sweep labels and reuse.
  • After merging: PR authors must ensure all GitHub Actions jobs pass. Transient failures often pass on rerun; see how to rerun failed jobs.
中文

感谢你的贡献!

  • **审阅:**如果 PR 修改的文件归属于仓库管理员及 @SemiAnalysisAI/core 之外的 CODEOWNER,请先联系一位有资格的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md,再通过 Slack 联系核心维护者。必须严格遵循模板,并保留 As a PR reviewer and CODEOWNER, I have reviewed this and have,才能触发签核验证。
  • **PR 验证:**只有当 PR 在 inferencex-e2e/perf-changelog.yaml 末尾追加了条目,并且恰好带有一个主标签时,才会运行扫描:full-sweep-fail-fast(强烈推荐;canary 加逐矩阵 fail-fast)、full-sweep-enabled(有 canary;矩阵任务在失败后继续运行)或 non-canary-full-sweep-enabled(无 canary,也无 fail-fast)。修饰标签 all-evals、evals-only 和 agentx-fast 必须与主标签一起使用。fork PR 的标签由维护者添加。参见扫描标签与复用。
  • **合并后:**PR 作者必须确保所有 GitHub Actions 任务通过。临时性失败通常可以通过重新运行恢复;参见重新运行失败任务的说明。

@adibarra
adibarra added this pull request to stack #3808 October 8, 2026 14:35
@adibarra
adibarra force-pushed the refactor/recipe-bundles branch from b8a0abc to 9f1346d Compare October 8, 2026 16:03
@adibarra
adibarra force-pushed the refactor/recipe-bundles branch from 9f1346d to fcd8458 Compare October 8, 2026 16:25
@adibarra
adibarra force-pushed the refactor/recipe-bundles branch from fcd8458 to 3bb06f4 Compare October 8, 2026 17:21
@adibarra
adibarra force-pushed the refactor/recipe-bundles branch from 3bb06f4 to c68a0ad Compare October 8, 2026 21:28
@adibarra
adibarra force-pushed the refactor/recipe-bundles branch 2 times, most recently from 030da06 to f0d9622 Compare October 9, 2026 11:39
@adibarra
adibarra force-pushed the refactor/recipe-bundles branch 2 times, most recently from ff09ac0 to 34c24b0 Compare October 9, 2026 18:20
@adibarra
adibarra force-pushed the refactor/recipe-bundles branch from 34c24b0 to 9c61ead Compare October 9, 2026 18:48
@adibarra
adibarra force-pushed the refactor/recipe-bundles branch from 9c61ead to 6a5e0b4 Compare October 9, 2026 23:43
@adibarra
adibarra force-pushed the refactor/recipe-bundles branch 2 times, most recently from 7bb4259 to 40643b0 Compare October 10, 2026 13:38
@adibarra
adibarra force-pushed the refactor/recipe-bundles branch from 40643b0 to c261feb Compare October 12, 2026 08:02
prepare_recipe edited the staged fragment before the binder ran. Its text
edits only find a flat recipe's top-level name and six-space
watchdog-timeout, so bundle variants kept their recipe names as srtctl's
run name and never got the cluster's dist-timeout. The edits now apply to
the recipe the binder writes, which is flat for every variant. Slurm job
names were unaffected: srtctl names the job from RUNNER_NAME.
… entry

Master-config entries whose rows each named their own standalone recipe now
select overrides of one <agg|disagg>[-axis]-variants.yaml: base holds what
the points share and each override_<topology>_c<conc> holds only its point's
deltas, including the job name. Ten AgentX entries (50 recipes) and
seventeen fixed-sequence entries (184 recipes) move into 27 bundles. Every
moved row's selected variant equals its old recipe after composition, with
and without the power telemetry block, and the full-sweep matrices differ
only in those rows' srt-recipe strings.

Entries whose points mix recipe designs, such as a TP low-latency recipe
beside wide-EP ones, keep their standalone files. Section-header comments,
commented-out settings and per-point labels that restate the topology are
not carried into the bundles. RECIPES.md describes the convention.
… into bundles

#3659 split the GB300 GLM-5.2 SGLang AgentX recipes into one file per point. Like
the other AgentX families, the three aggregate points select overrides of
agg-variants.yaml and the three low-latency 1PxD points overrides of
disagg-variants.yaml; each override selects exactly its old recipe. The
DEP8/DEP16 2P1D point is a different design and keeps its own file.
The health-check floor and model path of a bound bundle variant are already covered by test_srt_config and the multi-node lane test, and the bundle-shaped health budget test exercises inputs that only reach raise_health_attempts as the binder's flat output.
Its other caller went away when the lane edits moved to the bound recipe.
Every launch renames the bound recipe and fingerprints exclude name, so bundles need not set one per override. Drop a misindented commented-out flag and move the NATS comment above request_plane.
…s from the pareto bundles

of the pareto entries by naming those points' standalone recipes, which the
bundles replace. Their rows select the matching pareto overrides, so each
still runs exactly the pareto point's recipe rather than a copy that could
drift; the DEP16 points keep disagg-dep16-mtp6-variants.yaml. RECIPES.md
notes that an entry rerunning another entry's points selects its overrides.
@adibarra
adibarra force-pushed the refactor/recipe-bundles branch from c261feb to efe2f87 Compare October 12, 2026 10:22

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

1 participant