Repository navigation
Conversation
Contributor
|
Thanks for the contribution!
中文感谢你的贡献!
|
This was referenced Oct 8, 2026
Draft
adibarra
added this pull request to stack #3808
October 8, 2026 14:35
adibarra
force-pushed
the
refactor/srt-composer
branch
3 times, most recently
from
October 9, 2026 11:39
697e339 to
fc3b40a
Compare
adibarra
force-pushed
the
refactor/srt-composer
branch
from
October 9, 2026 17:47
fc3b40a to
f08fa0c
Compare
adibarra
force-pushed
the
refactor/srt-composer
branch
2 times, most recently
from
October 9, 2026 23:43
c209995 to
e3a37da
Compare
adibarra
force-pushed
the
refactor/srt-composer
branch
from
October 10, 2026 07:51
e3a37da to
1ac6704
Compare
… the point Active fixed-sequence srt-slurm recipes keep only recipe-specific settings. At launch the lane's shared block (configs/srt-recipes/) is merged under the fragment, the row's variant is selected, and the binder writes the matrix point: model path, image, precision, ISL/OSL, single-node client values (MODEL, CONC, RANDOM_RANGE_RATIO, USE_CHAT_TEMPLATE) and telemetry concurrencies. A fragment that sets a bound key fails before submission. The hf:<MODEL> model path maps to the cluster's staged checkpoint and the master image to its staged container, so recipes stop naming aliases. The mi355x-amds DeepSeek-R1 MXFP4 checkpoint moves into models.entries. Six b300-fp8 trtllm recipes declared fp4 precision; it is now bound to fp8.
`infx generate --config-key <key|glob> --output-dir <dir>` writes the bound recipe of every fixed-sequence point and variant, validated by the pinned srtctl, with a manifest mapping files to matrix points. srtctl's import dependencies sit behind the new `recipes` extra. RECIPES.md documents fragments, composition and the command.
…red block Every active multi-node fixed-sequence fragment repeated benchmark.env.TOKENIZER: /model. The shared block configs/srt-recipes/fixed-sequence-multi.yaml now sets it and the fragments drop it, along with the benchmark blocks it was the only entry of. The three mi355x fragments that lacked it (DeepSeek-R1 FP8 disagg variants, Qwen3.5 FP4 and FP8 1p1d) now tokenize with the mounted checkpoint too instead of the served model id. Every other bound recipe is unchanged.
infx generate bound every variant a multi-node selector matched, while the launcher's binder refuses anything but one, so it wrote recipes for points that launch rejects, such as dsr1-fp8-gb200-dynamo-sglang. Both now share bind_multinode, and generate fails such a point as launch does.
The binder parses CONC_LIST with the launcher's parse_concurrencies, now in workload.py, so telemetry concurrencies get its canonical and unique checks. Define the bound recipe file name once, drop generate's MODEL_PREFIX and RUN_EVAL inputs that nothing on its path reads, name what _validated does, and drop a model guard the binder makes redundant. Fragment checks drop two cases that repeat a covered branch.
The content checks that follow already prove srtctl ran the bound variant.
Drop restatements of the shared blocks and alias history from RECIPES.md, and correct which keys add-model-hardware says fragments omit.
CI and the testing guide install every extra with the test group.
adibarra
force-pushed
the
refactor/srt-composer
branch
from
October 10, 2026 13:38
1ac6704 to
d86e522
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Builds on the approach in Cam's draft #3612, without its behaviour changes.
Active fixed-sequence recipes become fragments: they keep only recipe-specific settings in native srt-slurm YAML. At submit time InferenceX:
configs/srt-recipes/fixed-sequence-{single,multi}.yaml), picked automatically for single- or multi-node. Maps merge, lists are replaced, and the fragment wins. Bundles get the block underbase.Other changes:
*/1k1k/, and the bundles referenced only by commented-out H100 rows) are untouched.hf:<MODEL>maps to the cluster's staged checkpoint. mi355x-amds gets amodels.entriesrow for DeepSeek-R1-0528-MXFP4-v2 instead of a hard-coded/it-share/datapath.infx generate --config-key <key|glob> --output-dir <dir>: writes complete bound recipes plus a manifest for review. Install withuv sync --extra recipes.RECIPES.mddocuments fragments, composition andinfx generate.No performance change
Unlike #3612, this keeps:
RANDOM_RANGE_RATIO: still comes from the workflow env.tokenizer-pathandserved-model-nameare left as recipes set them.Verification
python -m infx.launchthrough fake Slurm for every srt row on every cluster it can reach, base vs head. A fake srtctl recorded the recipe srtctl would load, using srtctl's own override, variant and alias resolution.model_pathskeys becamehf:<MODEL>, with the same resolved path.model.precisionchanged on 6 b300-fp8 TRT recipes, which were stale (they saidfp4). On the mi355x dsr1 rows, the checkpoint is now reached through the staged path.full-sweepoutput is byte-identical to the base, with and without--all-evals.Known issue
The
dsr1-fp8-gb200-dynamo-sglangrow that referencesdisagg-tp8-stp-variants.yamlwith no selector would submit 3 zip jobs on base and then fail. It now stops before submitting with "selects 3 variants, not one". The row needs a selector or should be removed.TOKENIZER: /modellives in the sharedfixed-sequence-multi.yamlblock for every multi-node fixed-seq recipe, which adds it to the 3 mi355x recipes.