Skip to content

Worker reasoning effort follows the dispatch tier - #566

Merged
gmickel merged 9 commits into
mainfrom
fn-299-worker-reasoning-effort-follows-the
Oct 17, 2026
Merged

gmickel merged 9 commits into
mainfrom
fn-299-worker-reasoning-effort-follows-the

Conversation

@gmickel

@gmickel gmickel commented Oct 17, 2026

Copy link
Copy Markdown
Owner

Why

The tier call that already runs before each worker spawn now sets the worker's reasoning effort as well as its model: one level below the host default for a confident mechanical answer, one above for a confident intelligent one, the default otherwise and without a judge. No new judge question or call; requested, routing-block and role-pinned efforts still win.

What changes for a user or operator

Workers on routine tasks spend less thinking time and workers on hard tasks more, at the same model. The Tier: line in each done summary names the effort applied, or why it was not applied. Runs without a TypeSafe key behave exactly as before.

Scope

23 files changed; +452/-35 lines; 8 generated, 3 mechanical.

1. Effort never followed the tier

Check the spec's precedence and fallback rules: the tier answer already existed but only chose the model.

2. flowctl computes the worker effort

Check judge_tier_effort's ladder step, bounds, floors and precedence, and that scouts (no --task) are unaffected.

3. The conductor carries the effort to the spawn

Check that spawn_effort reaches the host's effort parameter, the bridge gets it on the IMPLEMENTER line, and a rejected level retries once at the default.

4. Docs and reach pages

Check the judge and flowctl docs describe the new flags, and the reach pages state which host versions take a spawn effort.

8 generated files

5. Flow records

Spec, task and receipt bookkeeping, plus one bug memory entry on dispatch settings reaching bridged children.

3 mechanical files

6. R6 timed evidence

Check the open item: pass rate held, the mechanical wall-time saving is directional but not confirmed, and the end-to-end control never exercised the feature.

Coverage: R1 → group 2; R2 → group 2; R3 → group 2; R4 → groups 2, 3, 4; R5 → groups 2, 3; R6 → group 6

Blast radius

Read judge_tier_effort in flowctl.py first (the whole decision), then judge-tier.md for how the conductor passes spawn_effort. Scouts and reviewers are untouched. Unproven: the wall-time saving on mechanical fixtures (R6, below), and effort on hosts other than Claude Code 2.1.296+ and Codex 0.146.0+.

Verification

  • Implementation review: codex, 3 rounds: NEEDS_WORK, NEEDS_WORK (R6 only), SHIP
  • Full suite: 218 files, 4055 ran, 0 failures
  • Ruff, Codex sync check, doc anchors: clean
  • Live tier call with judge key: Tier: mechanical (jev 0.99) -> claude-sonnet-5-5 at low effort (host default medium)
  • Tier call without key: at default effort
  • R6 replay pass rate: 15/15 at low vs 15/15 at medium (opus-5.5, 3 mechanical fixtures x 5 reps)
  • unverified: R6 mechanical wall time: medians 55/43/131 s vs 76/67/138 s; pre-registered verdict INCONCLUSIVE (gap 10.0 s vs 9.7 s floor)
  • unverified: R6 end-to-end control: review rounds 2 vs 2, wall 1,886 vs 1,992 s; no draw spawned a worker, so null by construction

Tradeoffs

Decisions: (1) the 0.8 confidence floor that gates the model tier gates effort in both directions, so an intelligent answer below 0.8 keeps the default. (2) --explicit-effort carries the invocation's effort, else the routing block's implementer line; naming only a model keeps the tier effort (R2, for callers that always pin the implementer model). (3) A role-pinned effort applies only to an in-host spawn, never to a bridged implementer. (4) A rejected effort level is retried once without effort; that rule lives in conductor prose because flowctl cannot observe the rejection. (5) Review round 1 found two paths where a bridged implementer never received the effort; fixed in 3fca663.

Open items

R6 is not met as written. Worker replay (claude-opus-5-5, 3 judge-confident mechanical fixtures x 5 reps per arm): pass rate 15/15 at low vs 15/15 at medium, medians 55.0/43.0/130.9 s vs 76.1/67.2/138.0 s, but the pre-registered paired verdict is INCONCLUSIVE (pooled gap 10.0 s against a 9.7 s floor, within-cell spread 29-88 s). The end-to-end large-feature control (median 2 vs 2 review rounds, 1,886 vs 1,992 s) is null by construction: no draw spawned a worker or made a tier call. About 1% of historical tasks get a confident mechanical answer and 2% a confident intelligent one. Accepting this, or running a multi-task --auto study on Claude Code 2.1.296+, is the maintainer's call. The fn271 harness pins Claude Code 2.1.283 (no spawn effort); an opt-in FN271_CLAUDE_DIR override sits uncommitted in agent-evals. flow-next.dev docs for this change ship with the 9.0.0 downstream walk.

https://claude.ai/code/session_01LbHwMnkD6ovBw6n8cnD8hQ

- flowctl judge --preset tier --task takes the host's effort levels and
  default (--effort-levels, --default-effort) plus requested/role efforts,
  and returns spawn_effort: one level below the default for a confident
  mechanical answer, one above for a confident intelligent one, the
  default otherwise and without a judge answer
- requested or role-pinned effort wins; an explicit model without an
  effort keeps the tier effort; a bridged implementer carries it as
  "<model> at <effort>"
- the Tier: line names the effort applied or why it was not; scouts
  (no --task) are unchanged
- judge-tier recipe, multi-task 3c, worker-bridge, judge/flowctl docs and
  the Claude Code and Codex reach pages state the effort facts

Task: fn-299-worker-reasoning-effort-follows-the.1

Claude-Session: https://claude.ai/code/session_01LbHwMnkD6ovBw6n8cnD8hQ
- a role-pinned effort configures the native spawn only; a bridged
  implementer keeps the requested or tier effort in "<model> at <effort>"
- a routing-block implementer the worker bridges takes the level its
  TIER_LINE applied; a bridge that rejects it runs once at its default

Task: fn-299-worker-reasoning-effort-follows-the.1

Claude-Session: https://claude.ai/code/session_01LbHwMnkD6ovBw6n8cnD8hQ
The Agent spawn's effort parameter was observed on Claude Code 2.1.296
and is absent on 2.1.283; the conductor checks the spawn's parameters
before passing effort facts, else the Tier: line says not applied.

R6 study (maintainer eval, claude-opus-5-5, only effort varied):
- worker replay, 3 judge-confident mechanical fixtures x 5 reps x 2 arms:
  pass 15/15 at low vs 15/15 at medium; median wall 55.0/43.0/130.9 s at
  low vs 76.1/67.2/138.0 s at medium, pooled mean gap 10.0 s against a
  9.7 s floor, paired verdict INCONCLUSIVE (within-cell spread 29-88 s)
- end-to-end large-feature case, main vs branch, 3 draws each, same time:
  review rounds median 2 vs 2, wall median 1,992 vs 1,886 s, all hidden
  checks pass; null by construction: no draw spawned a worker or made a
  tier call (single-task specs run inline)

Declined #3: open item for the person - R6's "shorter wall time" clause
is inconclusive at n=5 and the end-to-end comparison never reaches the
worker route; deciding whether to accept this evidence or fund a larger
multi-task --auto study is the maintainer's call, not a code change.

Task: fn-299-worker-reasoning-effort-follows-the.1

Claude-Session: https://claude.ai/code/session_01LbHwMnkD6ovBw6n8cnD8hQ

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 93e6d63. Configure here.

Comment thread plugins/flow-next/scripts/flowctl.py
gmickel and others added 2 commits October 17, 2026 03:40
… implementer (#566)

- Valid narrow leak (custom role effort + bridged routing-block implementer); fixed by not passing --role-effort on that path in judge-tier.md; no flowctl change.

Claude-Session: https://claude.ai/code/session_01LbHwMnkD6ovBw6n8cnD8hQ
@gmickel
gmickel marked this pull request as ready for review October 17, 2026 07:47
@gmickel
gmickel merged commit 4f84433 into main Oct 17, 2026
13 checks passed
@gmickel
gmickel deleted the fn-299-worker-reasoning-effort-follows-the branch October 17, 2026 07:48
gmickel added a commit that referenced this pull request Oct 17, 2026
* origin/main:
  Worker reasoning effort follows the dispatch tier (#566)
  Measured numbers you can trust: limiter, errors, work done (#564)

# Conflicts:
#	CHANGELOG.md
#	plugins/flow-next/scripts/flowctl_tracker/MANIFEST.json
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant