Skip to content

refactor(srt): compose fixed-sequence recipes from fragments and bind the matrix point - #3809

Draft
adibarra wants to merge 8 commits into
refactor/srt-recipe-keyfrom
refactor/srt-composer
Draft

adibarra wants to merge 8 commits into
refactor/srt-recipe-keyfrom
refactor/srt-composer

Conversation

@adibarra

@adibarra adibarra commented Oct 8, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Builds on the approach in Cam's draft #3612, without its behaviour changes.

Active fixed-sequence recipes become fragments: they keep only recipe-specific settings in native srt-slurm YAML. At submit time InferenceX:

  1. Composes the fragment over a shared workload block (configs/srt-recipes/fixed-sequence-{single,multi}.yaml), picked automatically for single- or multi-node. Maps merge, lists are replaced, and the fragment wins. Bundles get the block under base.
  2. Selects the variant, so zip groups keep concurrency paired with their tuning.
  3. Binds master-owned values into the selected recipe: image, model, precision, ISL/OSL and concurrency. If a fragment sets a bound key, submission fails with an error naming the key and file.

Other changes:

  • Recipe conversion: all 282 active fixed-sequence recipes are converted: 50 single-node, 232 multi-node. Retired and unreferenced recipes (*/1k1k/, and the bundles referenced only by commented-out H100 rows) are untouched.
  • Staged checkpoints: hf:<MODEL> maps to the cluster's staged checkpoint. mi355x-amds gets a models.entries row for DeepSeek-R1-0528-MXFP4-v2 instead of a hard-coded /it-share/data path.
  • infx generate --config-key <key|glob> --output-dir <dir>: writes complete bound recipes plus a manifest for review. Install with uv sync --extra recipes.
  • Docs: RECIPES.md documents fragments, composition and infx generate.

No performance change

Unlike #3612, this keeps:

  • Chat template: today's behaviour. Single-node uses the chat template only for speculative runs; multi-node always does.
  • RANDOM_RANGE_RATIO: still comes from the workflow env.
  • Recipe settings: tokenizer-path and served-model-name are left as recipes set them.

Verification

  • Equivalence harness (not committed): ran the real python -m infx.launch through fake Slurm for every srt row on every cluster it can reach, base vs head. A fake srtctl recorded the recipe srtctl would load, using srtctl's own override, variant and alias resolution.
    • AgentX (2176 runs) and fixed-sequence single-node (906): no differences.
    • Fixed-sequence multi-node (500): only expected differences. srtslurm model_paths keys became hf:<MODEL>, with the same resolved path. model.precision changed on 6 b300-fp8 TRT recipes, which were stale (they said fp4). On the mi355x dsr1 rows, the checkpoint is now reached through the staged path.
  • Matrix: full-sweep output is byte-identical to the base, with and without --all-evals.
  • Tests: new behavioural tests cover merge precedence, the bound-key conflict error, variant selection before binding, and single-node chat-template parity. The full CI pytest command passes 2277 tests with 1 GPU-only skip, and ruff is clean.

Known issue

The dsr1-fp8-gb200-dynamo-sglang row that references disagg-tp8-stp-variants.yaml with no selector would submit 3 zip jobs on base and then fail. It now stops before submitting with "selects 3 variants, not one". The row needs a selector or should be removed.

  • TOKENIZER: /model lives in the shared fixed-sequence-multi.yaml block for every multi-node fixed-seq recipe, which adds it to the 3 mi355x recipes.

@github-actions

github-actions Bot commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

Thanks for the contribution!

  • Review: If this PR changes files owned by someone other than a repository admin or @SemiAnalysisAI/core, ask one eligible CODEOWNER to complete the latest PR_REVIEW_CHECKLIST.md before contacting a core maintainer on Slack. Follow the template exactly, including As a PR reviewer and CODEOWNER, I have reviewed this and have, so sign-off verification triggers.
  • PR verification: Sweeps run only when the PR appends an inferencex-e2e/perf-changelog.yaml entry and carries exactly one primary label: full-sweep-fail-fast (strongly recommended; canary plus per-matrix fail-fast), full-sweep-enabled (canary; matrix jobs continue after a failure), or non-canary-full-sweep-enabled (no canary or fail-fast). The modifiers all-evals, evals-only, and agentx-fast require a primary label. On fork PRs, a maintainer applies the label. See sweep labels and reuse.
  • After merging: PR authors must ensure all GitHub Actions jobs pass. Transient failures often pass on rerun; see how to rerun failed jobs.
中文

感谢你的贡献!

  • **审阅:**如果 PR 修改的文件归属于仓库管理员及 @SemiAnalysisAI/core 之外的 CODEOWNER,请先联系一位有资格的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md,再通过 Slack 联系核心维护者。必须严格遵循模板,并保留 As a PR reviewer and CODEOWNER, I have reviewed this and have,才能触发签核验证。
  • **PR 验证:**只有当 PR 在 inferencex-e2e/perf-changelog.yaml 末尾追加了条目,并且恰好带有一个主标签时,才会运行扫描:full-sweep-fail-fast(强烈推荐;canary 加逐矩阵 fail-fast)、full-sweep-enabled(有 canary;矩阵任务在失败后继续运行)或 non-canary-full-sweep-enabled(无 canary,也无 fail-fast)。修饰标签 all-evals、evals-only 和 agentx-fast 必须与主标签一起使用。fork PR 的标签由维护者添加。参见扫描标签与复用。
  • **合并后:**PR 作者必须确保所有 GitHub Actions 任务通过。临时性失败通常可以通过重新运行恢复;参见重新运行失败任务的说明。

@adibarra
adibarra added this pull request to stack #3808 October 8, 2026 14:35
@adibarra
adibarra force-pushed the refactor/srt-composer branch 3 times, most recently from 697e339 to fc3b40a Compare October 9, 2026 11:39
@adibarra
adibarra force-pushed the refactor/srt-composer branch from fc3b40a to f08fa0c Compare October 9, 2026 17:47
@adibarra
adibarra force-pushed the refactor/srt-composer branch 2 times, most recently from c209995 to e3a37da Compare October 9, 2026 23:43
@adibarra
adibarra force-pushed the refactor/srt-composer branch from e3a37da to 1ac6704 Compare October 10, 2026 07:51
… the point

Active fixed-sequence srt-slurm recipes keep only recipe-specific settings.
At launch the lane's shared block (configs/srt-recipes/) is merged under the
fragment, the row's variant is selected, and the binder writes the matrix
point: model path, image, precision, ISL/OSL, single-node client values
(MODEL, CONC, RANDOM_RANGE_RATIO, USE_CHAT_TEMPLATE) and telemetry
concurrencies. A fragment that sets a bound key fails before submission.

The hf:<MODEL> model path maps to the cluster's staged checkpoint and the
master image to its staged container, so recipes stop naming aliases. The
mi355x-amds DeepSeek-R1 MXFP4 checkpoint moves into models.entries. Six
b300-fp8 trtllm recipes declared fp4 precision; it is now bound to fp8.
`infx generate --config-key <key|glob> --output-dir <dir>` writes the bound
recipe of every fixed-sequence point and variant, validated by the pinned
srtctl, with a manifest mapping files to matrix points. srtctl's import
dependencies sit behind the new `recipes` extra. RECIPES.md documents
fragments, composition and the command.
…red block

Every active multi-node fixed-sequence fragment repeated
benchmark.env.TOKENIZER: /model. The shared block
configs/srt-recipes/fixed-sequence-multi.yaml now sets it and the fragments
drop it, along with the benchmark blocks it was the only entry of.

The three mi355x fragments that lacked it (DeepSeek-R1 FP8 disagg variants,
Qwen3.5 FP4 and FP8 1p1d) now tokenize with the mounted checkpoint too
instead of the served model id. Every other bound recipe is unchanged.
infx generate bound every variant a multi-node selector matched, while the launcher's binder refuses anything but one, so it wrote recipes for points that launch rejects, such as dsr1-fp8-gb200-dynamo-sglang. Both now share bind_multinode, and generate fails such a point as launch does.
The binder parses CONC_LIST with the launcher's parse_concurrencies, now in workload.py, so telemetry concurrencies get its canonical and unique checks. Define the bound recipe file name once, drop generate's MODEL_PREFIX and RUN_EVAL inputs that nothing on its path reads, name what _validated does, and drop a model guard the binder makes redundant. Fragment checks drop two cases that repeat a covered branch.
The content checks that follow already prove srtctl ran the bound variant.
Drop restatements of the shared blocks and alias history from RECIPES.md, and correct which keys add-model-hardware says fragments omit.
CI and the testing guide install every extra with the test group.
@adibarra
adibarra force-pushed the refactor/srt-composer branch from 1ac6704 to d86e522 Compare October 10, 2026 13:38

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

1 participant