Repository navigation
Conversation
Contributor
|
Thanks for the contribution!
中文感谢你的贡献！
|
This was referenced Oct 8, 2026
Draft
adibarra
added this pull request to stack #3808
October 8, 2026 14:35
adibarra
force-pushed
the
refactor/srt-agentx-compose
branch
from
October 8, 2026 16:03
367e3fb to
dc4671c
Compare
adibarra
force-pushed
the
refactor/srt-agentx-compose
branch
from
October 8, 2026 16:25
dc4671c to
752cb98
Compare
adibarra
force-pushed
the
refactor/srt-agentx-compose
branch
2 times, most recently
from
October 9, 2026 17:47
5f3fd28 to
d2ce3a3
Compare
adibarra
force-pushed
the
refactor/srt-agentx-compose
branch
2 times, most recently
from
October 9, 2026 23:43
a3c209a to
8c01db9
Compare
adibarra
force-pushed
the
refactor/srt-agentx-compose
branch
from
October 10, 2026 07:51
8c01db9 to
d0a5aac
Compare
adibarra
force-pushed
the
refactor/srt-agentx-compose
branch
from
October 10, 2026 13:38
d0a5aac to
524fa70
Compare
adibarra
force-pushed
the
refactor/srt-agentx-compose
branch
from
October 12, 2026 08:02
524fa70 to
c9e1037
Compare
… master rows
AgentX recipes, single- and multi-node, become fragments like the
fixed-sequence ones. configs/srt-recipes/agentic-{single,multi}.yaml supply
the client command; the binder writes the master model, image, precision and
identity, and the launcher hands multi-node clients RESULT_DIR,
AIPERF_DATASET_MMAP_CACHE_DIR and HF_HUB_CACHE from the volumes it mounts
(agentic-volume-mounts replaces lanes._AGENTIC_CACHES). Dead keys and copies
of the workflow environment leave the fragments; a fragment that sets a bound
key fails before submission.
A multinode search-space row with power: true replaces recipe telemetry
blocks: the shared telemetry-dcgm.yaml is merged with the cluster's
power-exporter-port and benchmark.concurrencies is bound after variant
selection, so bundles can measure power. POWER_LANES now judges the row's
field instead of reading recipe text, and inject_concurrencies is gone.
Selected zip variants drop null-valued keys as srtctl apply does, and
h200-dgxc GLM-5.2 FP8 AgentX keeps serving the Hub snapshot.
infx generate now binds every srt-slurm point of the selected keys, AgentX as well as fixed-sequence, single- and multi-node, through the binder the launcher runs. The workflow environment it reproduces gains the AgentX inputs (IS_AGENTIC, KV_OFFLOADING, TOTAL_CPU_DRAM_GB, THINKING_MODE) and the multi-node request fields. A multi-node point takes the binder inputs the launcher derives from the one cluster its runner label schedules on: launch_path, the lane's POWER_LANES decision and the cluster's power-exporter-port give the DCGM telemetry block and bound concurrencies, and config.agentic_client_env, which no longer needs an SrtRun, gives the AgentX client paths. Such a point fails when its label spans several clusters or the lane refuses its power. The manifest records the cluster.
…gment so binding it fails on its model block and benchmark MODEL. Like the other AgentX fragments it drops them, the client command its shared block supplies, and WEKA_LOADER_OVERRIDE, which names the corpus dsv41flash replays by default. Every point binds to main's selected variant without that override.
Recipes copied the matrix total-cpu-dram-gb into TOTAL_CPU_DRAM_GB and hardcoded the host KV pools derived from it, so the single-node launcher had to check the copies against the matrix. The binder now writes KV_OFFLOADING and, for DRAM points, TOTAL_CPU_DRAM_GB from the point, and replaces whole '@dram.<name>' values (total-gb, total-bytes, per-gpu-gb, per-gpu-bytes) with the budget, in total or over the GPUs it covers on a node; multi-node points take the cluster's gpus-per-node from the launcher. Recipes drop their budget copies and size vLLM SimpleCPU, LMCache, Mooncake, TRT-LLM host cache and formula-derived HiCache pools with references; the binder rejects literal SimpleCPU and LMCache sizes, and the recipe==matrix budget check is gone. Configs whose pools followed model-specific formulas, or whose dram-utilization stood in for one rank's pool, get a utilization that keeps those pools. Mooncake segments become bytes of the decimal budget, since vLLM reads "GB" as GiB. The Qwen3.5 MI355X FP8 CONC 64-80 HiCache pools (4 x 400 GB) and the MiniMax-M3 B300 TRT-LLM TP2/TP4 host caches (~388.6 GB per rank) exceeded the 3 TB policy share of their GPUs; they now take the per-GPU budget at dram-utilization 1.0 (374 GB and 374.5/374.75 GB per rank).
…ll pools from the DRAM budget #3659 and #3187 hardcode host pools that follow the point's DRAM budget: the GLM-5.2 GB300 SGLang AgentX HiCache pool (135 GB per rank) and the DSv4 GB300 prefill Mooncake segment (140gb, which Mooncake reads as GiB). They now take '@dram.per-gpu-gb' and '@dram.per-gpu-bytes', and dram-utilization moves to 0.6 (135 GB per GPU, unchanged) and 0.6664 (150.25 GB per GPU, 74 MB under 140 GiB). The DSv4 decode stores sit on decode nodes the budget does not cover and keep their 180gb.
The multi-node binder reads IS_AGENTIC instead of defaulting a missing value to fixed-sequence, model_paths drops a guard the binder makes redundant, the fragment check names the single-node variant case once, and RECIPES.md points at the bound-key tables instead of copying them and drops restated history.
select_recipe binds each variant before validate_recipe sees it, and the binder writes model.path, model.container, model.precision and benchmark.env.MODEL from the same environment, so those four comparisons can never fail. infx generate set KV_OFFLOADING and TOTAL_CPU_DRAM_GB to empty or zero placeholders for fixed-sequence points that nothing reads; it now sets them only for AgentX points.
The launcher and infx generate each derived the DCGM exporter port and the AgentX client paths from the cluster, lane, request and power decision, and infx generate must match the launcher. Both now call config.binder_inputs. infx generate imports the launcher modules at the top: they import nothing that imports it back.
… docstrings
The DRAM literal check and resolve_dram parsed '{'-prefixed strings the
same way; _json_object does it for both. resolve_dram's prefix check was
redundant: a value without the '@dram.' prefix still contains it and so
names no budget field. bind_workload and generate_recipes no longer list
what their callees write.
…r tests The bound AgentX client test asserted values it passed in; the CLI test's DRAM, power and client-env tail repeated the srt driver tests that run the same module with those flags; the TRT-LLM host cache case took the same path as the Mooncake one; a conditional assertion re-checked a non-DRAM point; infx generate's DRAM test re-asserted the matrix budget. The fragment-rejection tests no longer pin the explanatory suffix, and an unused constant goes.
RECIPES.md states the DRAM reference rules and the infx generate binder inputs in fewer words and drops a sentence the power-row table already makes. CONFIGS.md puts mounts back beside volume-mounts, which it describes. The DSv4 ATOM LMCache comment explained a literal the DRAM reference replaced, and the GLM-5.2 GB200 power comment moves onto the power field.
Recipes may no longer set model.container, identity.container.image or telemetry.enabled, yet the Klaud prompt and docs still asked for matching images, nuke.md still ran a script that rewrites model.container (it now fails on every recipe), and the power docs still gated energy collection on recipe telemetry. They now say the binder writes the master image and a power: true master row enables DCGM telemetry.
…fragments #3816, #3840 and #3744 added full recipes: the Qwen3.8 GB300 and Qwen3.5 GB300 DEP16 recipes carry model blocks, the client command, benchmark MODEL, TOTAL_CPU_DRAM_GB, KV_OFFLOADING and the client paths the launcher hands multi-node points, and both new recipes and the DSv4.1-Flash B300 one set the identity repo and image the binder writes. Like the other AgentX fragments they drop them, and WEKA_LOADER_OVERRIDE, which names both models' default corpus. The Qwen3.8 HiCache pool was the 225 GB matrix allowance less 1 GB of alignment headroom; it now takes '@dram.per-gpu-gb' and the entry's dram-utilization drops from 1.0 to 0.995, so the pool stays 224 GB and the bound TOTAL_CPU_DRAM_GB is 224. Each point otherwise binds to main's selected variant, with the Qwen3.5 model alias bound as its Hub id like the other GB300 Qwen3.5 AgentX points.
…RAM budget Main's C256 and C384 points of dsv4-fp4-mi355x-atom-agentic-mtp start an LMCache MP server with a literal 1000 GB L1 and set TOTAL_CPU_DRAM_GB 1019 (dram-utilization 0.34). The L1 now takes '@dram.total-gb' and the binder writes TOTAL_CPU_DRAM_GB. dram-utilization 0.3334 makes the budget 1000 GB, so the bound L1 stays 1000 GB. ATOM's lmcache.mp.max_pinned_state_bytes keeps its literal 1000000000000: srtctl needs the extra-config JSON brace-doubled, and the binder resolves no reference inside it.
adibarra
force-pushed
the
refactor/srt-agentx-compose
branch
from
October 12, 2026 10:22
c9e1037 to
b967a07
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
configs/srt-recipes/agentic-{single,multi}.yaml) and bound like fixed-sequence. The binder writes image, model, precision and identity, andidentity.container.imagestays byte-identical. A fragment that sets a bound key to a different value fails before submission.RESULT_DIR,AIPERF_DATASET_MMAP_CACHE_DIRandHF_HUB_CACHEfrom the volumes the launcher mounts.runners.yamlagentic-volume-mountsreplaceslanes._AGENTIC_CACHES. mi355x keeps its workspace result directory through a lane field.HF_HOMEkeeps its own cache.PORT,INFMAX_CONTAINER_WORKSPACE, retiredSRT_MEASUREMENT_WINDOW_*, defaultWEKA_LOADER_OVERRIDE), copies of workflow env, andbenchmark.concurrencies.power: trueon search-space rows:telemetry:blocks in 175 recipes (153 AgentX, 22 fixed-sequence) and is set on 68 rows.telemetry-dcgm.yamlwith the cluster's newsrt-slurm.power-exporter-portand bindsbenchmark.concurrenciesafter variant selection, so bundles can measure power now.recipe_globrules and theinject_concurrenciestext rewrite. The multinode workflow exportsPOWER.RECIPES.mdandCONFIGS.mdupdated.Verification
full-sweepand--all-evalsare identical to the base, apart from the newpower: trueon exactly the rows whose recipe had telemetry (140 and 125 rows).python -m infx.launchwith a fake srtctl.srtslurm.yaml.Plan.from_envresult as the base.Intentional differences
srtslurm.yamlmodel_pathskeys change from recipe aliases tohf:<MODEL>, with the same resolved path.benchmark.concurrenciesis dropped on 9 qwen3.5 b300 variants without power. It had no effect there.glm5.2-fp8-mi325x-sglang-agentic-mtp,dsv4-fp4-b200-sglang-agentic-hicache-mtp).--set CONC.infx generatebinds every srt-slurm point, AgentX included (power telemetry and client paths come from the launcher's own code). The manifest records the cluster.TOTAL_CPU_DRAM_GBandKV_OFFLOADING, and resolves@dram.total-gb|total-bytes|per-gpu-gb|per-gpu-bytesfor pool sizes (SimpleCPU, Mooncake, LMCache, TRT host cache, HiCache). Fragments that hardcode the budget or a derived pool size are rejected. Configs whose pool was a fraction of the budget havedram-utilizationretuned so the budget equals the pool; Mooncake sizes are exact bytes instead of GiB strings. qwen3.5 mi355x and minimaxm3 b300 TRT pools are cut to the 3 TB policy share.