Repository navigation
[TRTLLM-16460][feat] agent-flow: unified trtllm_modeling_v2 workflow for gated ModelingV2 onboarding and perf tuning - #20907
Draft
Wanli-Jiang wants to merge 17 commits into
Conversation
…the organization accepts
The agent SDK launches its bundled CLI binary, which only moves when the
SDK is upgraded; an organization that requires a newer CLI then refuses
every in-process turn at start-up ("older than the minimum version
required by your organization"). A layer now launches the CLI named by
the backend's CLI-path environment variable, or else the CLI on PATH when
it is newer than the bundled one, and otherwise keeps the SDK's choice.
Signed-off-by: Wanli Jiang <35160485+Wanli-Jiang@users.noreply.github.com>
…s and merges, cancel a local job's process group Agent-flow changes the trtllm_modeling_v2 workflow builds on, the first three ported from NVIDIA#20792: - A session can start as a fork of an earlier one (SessionConfig.fork_from): the Claude Code backend resumes the earlier session with fork_session, and the Codex backend refuses with NotImplementedError. A turn's result carries the backend's session id, and the session layer keeps the last one. - Node commits and merges in git worktrees are signed off, for repositories that require DCO. - Cancelling a local job sends SIGTERM to its whole process group, so the commands it started stop with it; a pid that leads no group gets the signal alone. - The Codex backend launches $CODEX_BIN, else the codex on PATH when it is newer than the pinned runtime (a login offers a new model only to a client that knows it), and a session given no model runs the one Codex is configured with. Signed-off-by: Wanli Jiang <35160485+Wanli-Jiang@users.noreply.github.com>
…les and configuration variants
The spec a run starts from, validated before any agent is constructed:
- spec/task_schema.py: the task.yaml schema with closed keys and batched
errors: checkpoint, units and features, parallel members, the serving
envelope, accuracy workloads and floors, runtime blocks, sessions, the
launch environment, source embargo, agents and the perf block.
- spec/intake/: `trtllm-modeling-v2 intake check|confirm|compile`. A
user-facing task.yaml is checked against its checkpoint and device,
confirmed (and the run refuses one changed since), and compiled with a
per-cluster site.yaml into the resolved spec.
- spec/variants.py: configuration variants, a unit's target served under
another configuration, as members of their own.
- task.example.yaml documents every key.
- spec/intake/derive.py: the fast path. `--checkpoint <hf_dir> --site site.yaml
--goal "..."` derives the request from config.json, the site's device and
defaults blocks and a one-line goal, compiles it with per-unit options,
discriminators and per-member perf configs, and starts the run; its intake
checks are the first part of phase 0, and confirmation is a sidecar file.
- Task validation refuses a TMPDIR too long for the LLM API's unix socket.
- `coverage.accept` names requested cells the run accepts leaving uncovered;
a storage variant boots at every member of its unit.
- accuracy.anchor: pending (per unit or variant): the benchmark is no check, the
logit gate is the oracle, and delivery stays incomplete until a written anchor
is recorded.
- `profile: bringup | standard | certified` (standard by default, also an
intake and site default) sets the switches the spec leaves unsaid:
certified turns on the envelope boot, the serve gate and the witnesses and
benchmarks every member; bringup turns the envelope boot and the serve
gate off. The witnesses are owed only when `accuracy.witness.enabled`
says so.
- `budget: {wall_hours, gpu_hours}` bounds a run; the intake compiles a
request's `constraints.budget` into it.
- A feature names its family (FEATURE_FAMILIES) and a speculator its kind,
not a skill to load.
Signed-off-by: Wanli Jiang <35160485+Wanli-Jiang@users.noreply.github.com>
…scheduler and command execution Where GPU work runs. A compute session is a Slurm allocation that runs a spool scheduler; jobs are files in its spool: - infra/substrate/scheduler.py: a priority queue over device slots with wall and memory limits and a scrubbed MPI environment; a device is quarantined until its memory is released; a job lost with its allocation runs again or is closed out node_lost; a native rebuild is held until the jobs that map the libraries are done (rebuild.py). - spoolctl.py: submit, launch, wait, status and cancel, also runnable by path from a shell without the CLI. placement.py routes a job to a session that can run it now and lets it borrow idle devices. - sessions.py and cli.py: sessions declared in task.yaml, their entry scripts and their hand-over; watchdog.py returns when one needs a human. - managed.py: with --manage-sessions on a host that reaches Slurm, the run submits each declared session that has no live watcher and no job in the queue, submits the successor a session near its end asks for, and cancels what it submitted when it closes. A host without Slurm keeps the broker and the manual hand-over. - provenance.py and the resolve hook record which checkout every process of a job imported, so a job with mixed checkouts is refused. - infra/execution/: one command or one agent turn as a request and an outcome, run here or through a spool, and how many checks and tasks the sessions' free devices admit. - infra/execution/priority.py: a job's class, carried in its spool header: accuracy, then feature, then perf work, which the scheduler starts in that order. - Within a priority, pending jobs run in the order they were queued (mtime, then a queue stamp), and a native rebuild also holds jobs whose native build is linked into the rebuilt checkout. - infra/execution/agent_gate.py: one cap on the agents working at once, slots granted by priority (planner, critical, required, side, background). Every role turn takes a slot, one stays free for the planner, and background turns take only idle slots. - The planner's reserved slot takes nothing from the tasks; side turns (primers, nice_to_have tasks) use only the slots the foreground work leaves idle (or the next one free after 20 minutes). - infra/execution/turns.py: a turn an earlier process answered is adopted, whichever transport ran it, and only for the work it was for (TurnRequest.work), never by another task asked the same words. - A local command leads its own process group; on a timeout or an interrupt the whole group gets SIGTERM, then SIGKILL, so no rank is left holding a device. - A queued gate moves to a session that would start it now (an atomic rename, so it never runs twice), and a dev session lends its free devices while nothing of the campaign's own runs there. - A session near its end judges a job by how long such jobs ran (twice the 90th-percentile of its spool's history), not by its worst-case wall limit. - incremental_build.sh builds from a venv of its own that sees the run's packages, fails if the run's venv changed under it, and refuses a python3 older than 3.7. Signed-off-by: Wanli Jiang <35160485+Wanli-Jiang@users.noreply.github.com>
…mits and the external transport - infra/environment.py: the launch environment, resolved once into the command prefix every role's prompt carries. - infra/vcs.py: questions to git about a checkout, asked with an argv and no optional locks, and the host git's version. - infra/committer.py: commit.sh for concurrent agents: it commits and signs off as the repository's user and refuses symlinks, placeholders and out-of-policy paths. - infra/dispatch.py: the run's dispatch/ directory, which every transport and the operator loop share: the orchestrator keeps a heartbeat there and leaves an exit marker, so a driver tells a busy orchestrator from one that has exited. - infra/external/: the external agent transport, for hosts where turns cannot run in process (in process is the default). A role turn is written as files and answered by a driver session (`trtllm-modeling-v2 driver wait|claim|prompt|answer|status|reload`, infra/driver/); a reload drains the orchestrator and re-executes it on the current flow code. - A one-rank command runs its worker in process (TLLM_WORKER_USE_SINGLE_PROCESS), derived by the environment so every command and the prompt's prefix agree. Signed-off-by: Wanli Jiang <35160485+Wanli-Jiang@users.noreply.github.com>
…the runtime map and delivery checks What onboarding a checkpoint knows before and after the agents run: - onboard/survey.py: what the checkpoint demands and what this build already supplies, read off disk without importing the engine, including the installed FlashInfer's public callables. - onboard/vocabulary.py: vocabulary.yaml, the survey's machine-readable product: each computation step mapped to a catalog entry, a queue row to onboard, or a gap. - onboard/runtime_map.py with runtime_index.yaml and kernel_sources.yaml: which engine area a checkpoint needs, where it lives, and which kernel source each catalog entry draws on; a run keeps the map narrowed to its checkpoint in its workspace (runtime_map.md). - onboard/delivery.py: the agents' claims checked against disk: the catalog's executable contract (CELLS, ARCHS), target selection over PrefillTarget/DecodeTarget, the W-row weight table and the routing. - onboard/embargo.py and logit_reference.py: source_embargo entries, and the Hugging Face reference's per-position predictions for the logit gate. - onboard/coverage.py: a certification whose cells the entry's CELLS already hold closes without a kernel writer turn. - onboard/promotion.py: kernel rows on a shared catalog entry land as target-local variants so they run in parallel; one promote task merges them into the catalog, and delivery checks that none is left (`ops:promoted`). - Promotion is per entry: an entry's variants are promoted once they landed and the target imports them, beside the feature work. - A promote declares only the files it rewrites (the target's tests that import a variant included), cells-only variants beside a catalog call are due at once, and its dispatch names the variants' kernel writer reports. - The vocabulary holds one row per shared entry, every call site's shapes as its cells; delivery fails accuracy:reference while the anchor is pending; the logit reference trusts a checkpoint that ships its own code unless told not to. - A queue row is dispatched by its mode (onboard, verify or promote); the worker's tags, not the row, say what it is briefed with. Signed-off-by: Wanli Jiang <35160485+Wanli-Jiang@users.noreply.github.com>
…ses and their harness Checks are commands with exit codes, never agent judgements. Every one runs with modeling_v2=require and attests the engine it ran: - checks/gates.py: boot (the declared workload, matched on generated text; a multimodal target gets an image), accuracy (trtllm-eval against the anchor with an anchor - k*SE floor and a cross-member finding) and acceptance (acceptance length on natural-text prompts, paired against the previous verdict). Each verdict records the engine's attested configuration, its pools, free memory and block-reuse hits. - checks/witness.py: features boot and accuracy cannot see (disagg equivalence, kv_quant, host offload, bounded replay) as two legs that differ in one variable. The equivalence witness uses natural prose, proves itself sensitive by an ablation leg, and pins attention-DP rank pairs. - envelope_boot.py, logit_agreement.py and serve_gate.py: the target at its declared limits, per-position log-probs against the HF reference, and trtllm-serve answering over HTTP. The envelope boot, the serve gate and the witnesses are owed only when the spec turns them on, as the certified profile does. - checks/harness/: the programs the gates run under the measured tree's interpreter (engine driver, engine facts, reuse probe, eval and quickstart shims). - gate_cli.py: `trtllm-modeling-v2 gate <name>`, one gate run exactly as the orchestrator runs it, on a snapshot of a commit or on a tree; its jobs queue at the feature class, behind the run's accuracy checks. - Boot also compares greedy tokens against the HF reference when one exists. - The disaggregation witness runs the leg B the spec declares (contrast_command and contrast_env), held to rank pairs and an ablation leg. - checks/testcache.py: a test-result cache keyed by the files each test read; `trtllm-modeling-v2 tests affected|compare` runs only what a change affects. - Every gate and job gets a per-run TRITON_CACHE_DIR unless the task sets one, and the generated affected-tests command carries a timeout. - checks/observables.py: per feature, the evidence a check must observe and the oracle it is judged by. A variant is about the engine fields its overlay sets; one it did not observe fails its boot. Every rank's worker prints its own engine record (harness/rankhook, first on PYTHONPATH), so a multi-rank member is never attested from llm.args. A chunked variant must serve a prompt it splits; without a stock speculator each draft depth must beat the one below; serve answers are held to the boot keywords. - The acceptance gate records a stock path that cannot load the checkpoint and skips that leg after; the test cache judges a process by its program; testcache.head_results says whether a file's tests pass at one commit. - A setting both launchers take (KV fraction and dtypes, tokens per block, max_num_tokens, trust_remote_code) is stated once and handed to the launcher that did not state it; trust_remote_code follows the checkpoint's auto_map. - Every process of a check writes the tree's Python modules it imported (the rank hook), and the gate result carries them. Signed-off-by: Wanli Jiang <35160485+Wanli-Jiang@users.noreply.github.com>
… and its tags - agents/roles.py: the roles (planner, surveyor, kernel_writer, assembler, tuner) and which backend and model each runs on: the agent that started the run (Codex inside a Codex session, else Claude Code), routed per role through the spec's agents block. A Codex role with no model runs the one the operator's Codex is set to. - agents/prompts/worker.py: one worker prompt every role shares; a role differs only in its ending, its decisions. Every prompt carries the resolved launch prefix, the embargo and the task's notes, and points at the code (the modeling_v2 package and its op contract, the runtime map) rather than restating it. - agents/prompts/tags.py: a task is briefed by its tags, its kind (survey, kernel, assemble, feature, integrate, investigate, tune) and an optional `tags` list, each adding its section to the dispatch prompt; `feature` brings `assemble`'s section with it. The tune section is the perf guidance a tuner and any perf work read. - agents/prompts/planner.py: the planner's contract and its scheduling rules. It plans only the work the goal needs: a defect off the goal is appended to issues.md in the workspace and gets no task, and an unexplained failure is an `investigate` task, never a bare retry. Messages from the operator are read first, and a red-flags table closes the prompt. - agents/prompts/survey.py: the surveyor's survey (the vocabulary schema and the capture lists). The planner links the core to kernel rows by soft edges, starts a width as a check task, answers FAILED/BLOCKED events first in a small revision and keeps its first turn to what can start now; a kernel writer hands back a FAILED_RECEIPT as soon as the defect is in shared code. - agents/decisions.py: one decision table, per role every decision, when to write it and what follows; each prompt's "Ending your turn" section, the enum and the orchestrator's finishing set are read from it. - The surveyor writes one row per shared entry, its cells spanning the served envelope; kernel writers and assemblers list what they leave uncertified in their decision file, which reaches the planner. - No role reviews another's work: a verdict is a check's, and a change to shared code is followed by a regression check with no agent turn. Signed-off-by: Wanli Jiang <35160485+Wanli-Jiang@users.noreply.github.com>
One skill, trtllm-modeling-v2-entry, loaded before writing or changing a ModelingV2 catalog entry: where the files go, what the entry and its GPU test must do, a test-cost budget per cell, integer-width boundaries, how a run certifies, a macro, promoting a target's local variants and the report. It says only what the code does not: the op contract (catalog/_op.py) and the claims gates that read it are the reference, and an agent reads them and the neighbouring entries rather than being handed a model's specifics. Everything else a worker needs it finds in the tree; its task's tags say what it is doing. Signed-off-by: Wanli Jiang <35160485+Wanli-Jiang@users.noreply.github.com>
…, integration and run records - run/contract.py: every check a run owes, compiled from the resolved spec. A verdict is recorded under the fingerprint of the tree its check read; a pass is shared between checks that measure one engine, and a stale verdict runs again, riskiest first. - run/graph.py: the task graph, the planner's only output: task kinds and the worker tags a task is briefed with, dependencies, the files a task touches, exclusive and nice_to_have tasks, attempts, and which ready tasks start. - run/integration.py: one sparse worktree per task, rebased onto the checkout and fast-forwarded in, so the checkout moves only by whole tasks; a conflict goes back to the task's worker, and a retry starts from the integration head. - run/snapshots.py: read-only snapshots of the checkout's commits, the trees contract checks measure, leased while a check runs. - run/receipts.py, runtime_facts.py: what a finished task hands the tasks that depend on it, and what the engines of passed gates ran. - run/state.py, journal.py, progress.py, drain.py: checkpointed state, the journal, typed decision files (the only channel an agent's verdict reaches the orchestrator through) and what a reload waits for. - The graph gains `check` tasks (checks with no agent turn) and soft dependencies (`depends_on_soft`: start early, integrate after the dependency lands). A change to shared code is followed by a regression check the orchestrator adds and wires itself, so no graph is refused for lacking one; a receipt records the commit the last one found clean. - With the logit gate on, the accuracy benchmark runs at tp1, the widest member and attention-DP members; logits prove the others. - Every check carries an inputs digest (the spec as it applies to its unit, file contents outside the checkout); a verdict on other inputs is stale. Options are compared by content, and acceptance checks share verdicts. - One full-support predicate; a check that raises leaves its task landed and runs again; the decision enums are the decision table's. - A verdict records what its check read of the tree (its Python files) and the marker narrowed to them; it holds on a later state while that is unchanged. - A milestone names the commit its checks measured (contract.measured_commit), however the checkout moved since; the run's clock starts when it goes past preflight, not at a dry run. - A cell is a unit or variant at a member; it holds once all its checks pass, and cells.json keeps, per cell, the latest commit it held on. Under the bringup profile a unit is owed at its cheapest member and no variant. A verdict records how long its check ran. Signed-off-by: Wanli Jiang <35160485+Wanli-Jiang@users.noreply.github.com>
…loop and the workflow - run/preflight.py and run/stages/preflight.py: everything that must hold before an agent is worth starting: the checkpoint, the host, the launch substrate, the KV-cache and capture-list configuration, a probe that every in-process agent can run Bash, and the surface survey. The substrate gate runs beside the first turns and stops them if it fails. - run/orchestrator.py: the graph-driven loop. Tasks start as capacity and their files allow; the planner runs on its own thread and is woken by events it has not already answered; the checks a finished task claims run on a shared pool; the survey's queue rows become kernel tasks with no planner turn. - run/stages/: the planned run's services (worktrees, snapshots, checks, capacity), the kernel, assembly and delivery stages. A worker's dispatch names its task's tags and carries their sections. - run/workflow.py: the run's lifecycle, the target marker, the correctness milestone (`<branch>-correct`, CORRECTNESS DELIVERED) and the closing verdict. - Phase 0 runs intake checks beside the survey; `BRINGUP DELIVERED` marks the core at tp1 before the correctness milestone; a promote task is added before delivery when local variants exist. - At most --max-parallel-tasks agents (default 8) work at once. Tasks on equal critical paths start in natural id order; a kernel row of the core unit is wired as a soft dependency of the core's assembly so it starts first; a FAILED_RECEIPT whose entry landed on the target meets its dependents' hard edges. - A report already answered by the graph, and a green landing that satisfies no check and whose dependents are wired, wake no planner turn. - TMPDIR is checked before sessions are submitted; a multimodal task's processor dependencies are imported in preflight; an unregistered `large` pytest marker becomes a host fact. - Full support (every required check, the delivery files, every shared change regression-checked, no local variant waiting) delivers the run and stops a running campaign; a DONE that leaves it out of reach counts toward the stall cap; tune is background work on idle agent slots. - Fault containment: a planner turn that raises is asked again on the same events, an unreadable proposal is refused and sent back, nothing ends on a lost result. Continuous verification: a member's delivery tier is rehearsed once its other checks hold, a landing is re-checked beside the work in flight, and the orchestrator adds the delivery-files task. - run/requirements.py: the request as a feature x member matrix, written to coverage.yaml; an uncovered cell stops preflight unless task.yaml accepts it, and what the bringup profile leaves out is accepted with the reason. - A kernel writer's engine_defect receipt adds the task that fixes it. - A blocked assembler and a MACRO tell the planner why and what to add; certifications close only on a recorded pass, and delivery requires the delivered tests to pass. - An orchestrator note names what answers it (the run it follows from, or the tasks it reports), so a revision that already holds them wakes no planner turn, and a passing check task wakes none; a ready task's wait ages its rank. - A due promote holds the check tasks that have not started; a task the orchestrator adds goes before the regression check it needs; an engine-defect fix task carries the receipt's whole proposal and its test. - Preflight refuses a placeholder anchor and two values for one launch setting; a reload sets a proposal it did not read aside, or merges it, instead of deleting it. - run/regression.py: the regression check. Once a change to shared code lands, the modeling_v2 package tests it can affect run through the test cache, with no agent turn, and a test that regressed fails the check. - run/inbox.py: the operator's inbox (`trtllm-modeling-v2 feedback`). A message reaches the planner's next turn, urgent, ahead of every other event; it steers the work and relaxes no check. - run/metrics.py: metrics.json, written when the run closes and by `trtllm-modeling-v2 metrics`: milestone times, the cells that hold, check runs and GPU hours by gate kind, turns per role, planner refusals, operator messages and restarts. - Each evaluation records the cells that hold, and `<branch>-best` follows the commit on which the most hold. A run budget, once spent, stops dispatch and ends the run as budget-spent. - run/primers.py (on unless --no-primers): the planner, kernel writer and assembler each get a briefing session, built on agent slots the run leaves idle; once one is ready, the role's turns fork from it. A turn that cannot fork, as on the external transport or on Codex, reads the primer's summary instead and is told the commit it was read at, to diff a file against. A turn never waits for a primer, and a fork that fails drops it. - With --manage-sessions the run submits, renews and releases its own sessions. A check's jobs queue at the accuracy class for the first unit's inner checks and at the feature class otherwise, the regression check's included. - Preflight says when the host's git (before 2.40) or python3 (before 3.7) changes how the run works, and writes the runtime map narrowed to the checkpoint into the workspace. Signed-off-by: Wanli Jiang <35160485+Wanli-Jiang@users.noreply.github.com>
A task of the planned run once the core is correct, or `trtllm-modeling-v2 tune` on a delivered target; it never gates delivery: - tune/stage.py: perf smoke, ground truth (baseline, in-tree reference, identity repeats as the noise floor, spread over the perf nodes), the candidate campaign and its wrap-up, bounded by perf.campaign.max_hours. - tune/campaign.py, worktree.py: one hypothesis per tuner turn and per worktree, measured against a same-host control, with stacked evaluation of compatible winners. - tune/keep_gate.py, inert.py, split.py: a keep needs the target's suite and the contract's accuracy gate; a loss is judged against the tree's own noise and never at a member the candidate does not change; a loss split by member asks for a member-gated variant. - bench.py, measure.py, sol.py, serving_config.py: the serving sweep, its jobs on the perf sessions, speed-of-light ceilings and the serving config derived from the workload. - A campaign told to stop (full support holds) dispatches nothing new, keeps what is measuring, and ends; its turns run at background priority. - A derived serving config trusts a checkpoint that ships its own code. - The keep rule holds the first token too: a candidate may not make the mean TTFT at concurrency 1 slower than perf.campaign.ttft_tolerance (10% by default). - Without git 2.40's `merge-tree --merge-base`, a winner stacks as one change. - UNMERGED_OPTIMIZATIONS.md, in the workspace, lists what the campaign found that the delivered target does not carry. - A tuner turn is briefed by the `tune` tag; it profiles through tune/bench.py --nsys. Signed-off-by: Wanli Jiang <35160485+Wanli-Jiang@users.noreply.github.com>
…nd package entry point `trtllm-modeling-v2 --task task.yaml --workspace W` runs (and resumes) a run; subcommands are tune, gate, spool, intake, session, driver, feedback and metrics. pyproject.toml adds the entry point and package data (the external transport's templates), and an E501 per-file ignore for prompt tables. examples/ holds one model's worked values for operators writing a task.yaml; a run never materializes it. The fast path (`--checkpoint`, `--site`, `--goal`, `--confirm`) and the `tests` subcommand are part of the CLI. What a run does beyond the contract is asked for: --manage-sessions turns on managed sessions. Primer sessions are on unless --no-primers. A run turns Claude Code's auto-memory off, since every worktree of a repository shares it. Signed-off-by: Wanli Jiang <35160485+Wanli-Jiang@users.noreply.github.com>
…r guides, examples, results and reference docs - README.md: what the package is, a quick start, the document map, and every flag and subcommand. - docs/: design.md (goals, principles, architecture, lifecycle, checks, task graph, substrate, tuning, decisions, limitations), user-guide.md, developer-guide.md, examples.md (an end-to-end worked run) and results.md (measured results and what is not measured yet). - docs/mechanisms.md: every mechanism with where it came from, its switch, and whether it has run on a GPU. - INTAKE.md: the user-facing task.yaml, site.yaml and the fast path. - TROUBLESHOOTING.md: keyed on the verbatim error, including the environment failures that read as model failures. - BUILD_AND_TEST.md: building and testing at each scale. - LESSONS.md: what real runs got wrong, and the mechanism that now prevents each. - AGENTS.md gains a short table of environment failures that read as model failures; agent-flow/README.md lists the workflow. - The docs cover full support, the capacity model, fault containment, continuous verification, the requirements matrix, observables, inputs digests, the engine-defect fix task and the decision table; the worker tags, the regression check, the operator inbox, primers and managed sessions. - design.md: staleness by what each check read of the tree. Signed-off-by: Wanli Jiang <35160485+Wanli-Jiang@users.noreply.github.com>
Wanli-Jiang
force-pushed
the
user/williamj/trtllm-modeling-v2-unified
branch
2 times, most recently
from
October 24, 2026 07:56
989fa12 to
352b030
Compare
Use the existing task graph and runtime gates, with worker instructions to close ordinary build/import/launch repairs inside their goal and ownership. Require the planner to explain changed prerequisites, new evidence or a new diagnostic hypothesis before retrying. Keep full event evidence and leave repair strategy to the worker instead of adding a native repair runner, blocker fingerprints or an automatic exclusivity rule. Retain completed-turn recovery with task-attempt identity, restart-safe regression coverage, native Codex defaults, and Codex primer forks with session IDs and routing-aware cache invalidation. Record resolved session configuration without adding activity timing or per-turn usage telemetry. Validation results are included in the accompanying review artifacts. No GPU onboarding or real-backend speed comparison is claimed. Signed-off-by: Wanli Jiang <35160485+Wanli-Jiang@users.noreply.github.com>
1 task done
…ast escalations and full filesystems Codex roles: - A role session asks for the most open sandbox and approval policy the site's Codex requirements allow (configRequirements/read). Asking for danger-full-access/never where a site forbids them made Codex fall back, silently, to a read-only sandbox. Under workspace-write the checkout and the run's spools are writable roots besides the workspace. - Each session records the sandbox and approval policy it actually runs under (SessionInitEvent, kept in the turn's .done). The agent probe shows them, and fails a Codex session that would run read-only. - The app server is started again when it exits as it starts: several starting at once collide on Codex's shared state database. - commit.sh says so when it cannot open its lock, instead of reporting another agent committing. Runs: - Operator feedback is read while only long checks run: the loop polls for messages instead of waiting for a future to end. - An attempt whose worktree could not be prepared (TaskNotStarted: no worker ran) keeps its number but does not count toward MAX_ATTEMPTS. - ESCALATE no longer ends the run while other work can move: the escalation is logged for the operator, the work that does not wait on it goes on, and the run ends escalated once nothing runs and nothing new came in. - A state write refused for room (EDQUOT, ENOSPC) no longer ends the loop: the state stays in memory and is written again on each pass, the work in flight goes on, and nothing new starts until the write lands. Docs: the default effort statements now say Codex follows its own configuration; the user guide's Codex section describes the sandbox each role asks for. Signed-off-by: Wanli Jiang <35160485+Wanli-Jiang@users.noreply.github.com>
WeiHaocheng
reviewed
Oct 26, 2026
| # SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
|
|
||
| """What onboarding a checkpoint knows: the survey, the vocabulary, the runtime map, delivery.""" |
Collaborator
There was a problem hiding this comment.
The scripts under onboard/, run/, spec/, and tune/ look like agent-generated tools from bringup. It might be hard for agents to actually reuse them later under this layout. I suggest moving them into a unified tools/ directory with a README documenting their capabilities. Some also seem quite bespoke to specific edge cases—we should review them to see what's genuinely worth keeping.
Every role turn now runs in process: one agent CLI per turn, proved by the agent probe before anything is spent. The external transport, which wrote each turn to dispatch/ for another authenticated session to answer, is gone: - infra/external/ (the transport, the driver's turn hand-out, TURN_INSTRUCTIONS.md and turn_prompt.md) and its package data; - the driver's pending, claim, release, prompt, answer and instructions commands, the NEW event, --max-inflight and --sessions-only; - --agent-transport; - the branches only a turn answered in another session needed: primers that could not fork, reloads that did not wait for a turn, restart accounting of turns left with a driver, answers no process read carried to the next turn, and the receipt's fallback to a dispatch/ result. Kept: the orchestrator's heartbeat and exit note, driver wait, status and reload, adoption of a completed in-process turn on restart, and the primer summary for a fork that fails. Each turn's request record now names the agent its role resolves to, routed or not. The docs say to launch the orchestrator from a shell whose agent CLI is logged in and can write its credential store, which the agent probe checks. Signed-off-by: Wanli Jiang <35160485+Wanli-Jiang@users.noreply.github.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
@coderabbitai summary
Description
trtllm-modeling-v2is an agent-flow workflow that takes a HuggingFace checkpoint, a clustersite.yamland a one-line goal, builds the model as a ModelingV2 target in TensorRT-LLM, proves it correct at every requested parallel configuration, and can then tune its serving performance. Agents write the code; whether something is done is decided only by Python checks (boot, accuracy, acceptance, and opt-in witnesses) on the measured tree.This PR unifies #20145 (the gated onboarding workflow, used as the base) with the parts of #20792 (the modeling-v2-onboard planner/worker workflow) that the merge plan kept.
What a run does
--checkpoint,--siteand--goalderive and compile the task spec; its checks are the first part of phase 0.<branch>-correctonce every required check holds on one commit,<branch>-best,cells.json,metrics.json.UNMERGED_OPTIMIZATIONS.md, never merged into the delivered target.Running it from Claude Code or Codex
Started from a Codex session (an interactive one or a
/goal), the orchestrator runs every role on Codex, on the model that Codex is set to and with the newer of itscodexand the pinned runtime; the run copies its skills where Codex finds them (<workspace>/.agents/skills). In-process Codex roles now fork primers through native app-serverthread/fork, with a saved-summary fallback when a fork fails. Unspecified Codex reasoning effort and context-window settings retain native defaults. Each role session asks for the most open sandbox and approval policy the site's Codex requirements allow (full access where allowed, elseworkspace-writewith the checkout and spools writable, andon-requestapprovals the orchestrator grants), records what it actually ran under, and the agent probe stops the run if Codex would run the roles read-only. The Codex session starts the orchestrator detached, outside its sandbox (user guide §8). Verified on a ChatGPT-login Codex: the workflow's agent probe passes from inside a Codex session with noagentsblock, and a Codex role loads the entry skill.Taken from #20792
The operator inbox (
trtllm-modeling-v2 feedback), managed sessions (--manage-sessions), job priority classes (accuracy, then feature, then perf), primer sessions (on by default,--no-primers), the unmerged-optimizations list, and agent-flow fixes (session fork, DCO sign-off on node commits and merges, cancelling a local job's whole process group).Changes since #20145 worth a reviewer's attention
contrast_commandandcontrast_env), the manualperfsubcommand,nsys_summary.pyandmicrobench.py, near-miss stacking, the spool's memory-pressure kill and the external driver-heartbeat check, and (commit 17) the external agent transport (--agent-transport externaland the driver's turn hand-out): every role turn runs in process, and the agent probe proves each agent can take a turn there. A kernel writer'sengine_defectreport still adds the in-run fix task.progress.yamlis written atomically, which fixes a race where a worker's decision could be read as missing.Backend recovery and task policy
depends_ongraph; unresolved blockers with no runnable remedy use the existing escalation path.AgentLayer; changed agent routing invalidates primers. Terminal records retain reported model/effort, elapsed time and outcome.Codex roles under site policies, and runs that keep going
From a real Codex-driven onboarding run (25 h to bring-up, stopped by the operator), the general fixes:
danger-full-access/neverunconditionally. On a site whose/etc/codex/requirements.tomlforbids them, Codex silently ran every role read-only (105Read-only file systemfailures in 91 of 483 sessions). Sessions now ask for what the requirements allow, record the effective sandbox and approval policy in each turn's.done, and the agent probe fails a session that would run read-only.commit.shsays so when it cannot open its lock instead of reporting another agent committing.MAX_ATTEMPTS.ESCALATEno longer ends the run while other work can move: the work that does not wait on the answer goes on, and the run endsescalatedonce nothing runs and nothing new came in.EDQUOT,ENOSPC) no longer kills the loop: the state stays in memory and is written again each pass, and nothing new starts until it lands.Run-specific fixes from that run (sparse worktrees for an inode quota, per-unit check sessions for a two-node topology, fixed-length perf validation) are not included.
Commits
The change is split into 17 commits so each layer can be reviewed on its own:
Known gaps
Test Coverage
agent-flow/testsCPU suite: 2,031 passed, 6 skipped, in 99.93 seconds. The wrapper blocks unmocked Claude SDK clients and mocks legacy CLI help's import-time skill discovery; live backend checks are skipped.agent-flow/testssuite passes here: 2,029 passed, 3 skipped (commit 17 removes the external transport's 14 tests); both pre-commit configs are clean on the changed files. A real Codex app server (0.162) honors theworkspace-writerequest with the writable roots andon-requestapprovals, and the session records them; the site-policy read is covered by CPU tests, not yet exercised on the cluster whose policy motivated it.setup.py; full repository CI is not claimed.PR Checklist
Please review the following before submitting your PR:
PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.
PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.
Test cases are provided for new code paths (see test instructions)
If PR introduces API changes, an appropriate PR label is added - either
api-compatibleorapi-breaking. Forapi-breaking, includeBREAKINGin the PR title.Any new dependencies have been scanned for license and vulnerabilities
CODEOWNERS updated if ownership changes
Documentation updated as needed
Update tava architecture diagram if there is a significant design change in PR.
The reviewers assigned automatically/manually are appropriate for the PR.
Please check this after reviewing the above items as appropriate for this PR.
GitHub Bot Help
To see a list of available CI bot commands, please comment
/bot help.🤖 Generated with Claude Code