Verify calls `adapter.complete` with `max_tokens=4096`, bypassing the `simple_text` default raised in #242

[gadievron]

Evidence provenance (added 2026-08-22). The run-derived figures in this issue come from a
private scan that a reader cannot reproduce (raptor-e2e-20260818, a 9.85-hour run against
a Python target). They are reported for completeness, not as independently checkable evidence.
Every code claim below is pinned to public source at b5019628 and is checkable; where a
conclusion rests on the private numbers rather than on the code, treat it as my report of what
that run showed. This label was missing when the issue was filed.

Summary

PR #242 raised simple_text's default max_tokens from 8192 to 20000, with a comment naming the
exact failure mode it was fixing: a thinking-era model consumes the output budget on hidden reasoning
and returns a reasoning-only (empty) completion, which the adapter rejects as "no usable content".
Stage 2 verification does not go through simple_text. It calls binding.adapter.complete directly
with a module constant of 4096, so the raised default never reached verify on any route.

Evidence

libs/openant-core/utilities/finding_verifier.py:74 (HEAD b501962):

MAX_TOKENS_PER_RESPONSE = 4096

utilities/finding_verifier.py:399-411 — the agent loop, which bypasses simple_text entirely:

        while iterations < MAX_ITERATIONS:
            iterations += 1

            self._log("debug", f"Iteration {iterations}", iterations=iterations)

            # Adapter handles the rate-limiter wait/report dance internally.
            response = self.binding.adapter.complete(
                model=self.binding.model,
                max_tokens=MAX_TOKENS_PER_RESPONSE,
                system=system_prompt,
                tools=self._tool_defs,
                messages=messages,
            )

libs/openant-core/utilities/llm/helpers.py:41-49 — what PR #242 (a991416, merged 2026-08-16,
two days before this run) put in place:

    # Thinking-era default. Claude-5 / Gemini-2.5+ / OpenAI o-series spend
    # output budget on hidden reasoning; 8192 can be fully consumed by
    # reasoning on a large unit, yielding a reasoning-only (empty) completion
    # that the adapter drops -> hard "no usable content" error. 20000 is under
    # the Anthropic non-streaming 10-min ceiling (32000 is rejected with a
    # "Streaming is required" ValueError; 20000 is accepted) and is a CAP, not
    # a floor on generation -- models still stop at end_turn on small prompts,
    # so this does not raise cost for short answers.
    max_tokens: int = 20000,

git show --stat a991416 --format='%h %ad %s' --date=short | head -12
#   a991416 2026-08-16 fix(analyzer): recover reasoning-only empty-completion analyze responses (#242)
#     core/analysis_core.py | utilities/llm/helpers.py | tests...

The PR touched analyze (core/analysis_core.py:279 calls simple_text with no max_tokens, so it
inherits 20000). It did not touch finding_verifier.py.

Census of explicit budgets

cd libs/openant-core
grep -rn 'max_tokens=' --include='*.py' . --exclude-dir=tests \
  | grep -v 'max_tokens=max_tokens' | grep -vE 'def |the provider supports' | sort

19 hits: one is a comment (context/threat_model_agent.py:288) and two are adapter connectivity
probes with max_tokens=1 (utilities/llm/providers/anthropic.py:240,
utilities/llm/providers/bedrock.py:233). That leaves 16 production override sites.

Of those 16, 10 pin an effective 4096:

core/llm_reachability.py:392            max_tokens=4096
openant/cli.py:1097                     max_tokens=4096
report/generator.py:200                 max_tokens=4096
report/generator.py:342                 max_tokens=4096
report/html_report.py:224               max_tokens=MAX_TOKENS            (html_report.py:46  = 4096)
utilities/context_enhancer.py:333       max_tokens=4096
utilities/stage1_consistency.py:328     max_tokens=MAX_TOKENS            (stage1_consistency.py:29 = 4096)
utilities/agentic_enhancer/agent.py:255 max_tokens=MAX_TOKENS_PER_RESPONSE (agent.py:37 = 4096)
utilities/finding_verifier.py:407       max_tokens=MAX_TOKENS_PER_RESPONSE (finding_verifier.py:74 = 4096)
utilities/finding_verifier.py:1035      max_tokens=MAX_TOKENS_PER_RESPONSE

5 bypass simple_text by calling binding.adapter.complete(...) with an explicit budget —
context/repo_explorer.py:286, report/generator.py:200, report/generator.py:342,
utilities/agentic_enhancer/agent.py:255, utilities/finding_verifier.py:407.

Only 7 call sites inherit the 20000 default (a simple_text(...) call with no max_tokens):

python3 - <<'EOF'
import re, os
hits = []
for dp, dn, fn in os.walk("."):
    if "/tests" in dp or dp.endswith("/tests"): continue
    for f in fn:
        if not f.endswith(".py"): continue
        p = os.path.join(dp, f); src = open(p, encoding="utf-8", errors="replace").read()
        for m in re.finditer(r'simple_text\(', src):
            if "def simple_text(" in src[max(0, m.start()-4):m.end()]: continue
            i, depth = m.end(), 1
            while i < len(src) and depth:
                depth += (src[i] == "(") - (src[i] == ")"); i += 1
            hits.append((p.lstrip("./"), src[:m.start()].count("\n")+1, "max_tokens=" in src[m.start():i]))
print("simple_text call sites:", len(hits), " inheriting 20000:", sum(1 for h in hits if not h[2]))
for h in hits:
    if not h[2]: print("  %s:%d" % h[:2])
EOF

simple_text call sites: 18  inheriting 20000: 7
  core/analysis_core.py:279
  utilities/context_corrector.py:152
  utilities/context_corrector.py:316
  utilities/context_corrector.py:509
  utilities/ground_truth_challenger.py:270
  utilities/ground_truth_challenger.py:343
  utilities/context_reviewer.py:194

Runtime evidence that the budget is reached in verify

The verifier already detects the truncation itself, at utilities/finding_verifier.py:487-504 (excerpt — 13 interior lines elided, 4 of them comments):

            # A finish call on a turn the model TRUNCATED (stop_reason == "max_tokens")
            # is not a trustworthy completed verdict: a well-formed
            # finish(agree=False, "safe") from a cut-off turn would silently downgrade a
            # Stage-1 vulnerable. Treat it as verification-incomplete ...
            if finish_result and stop_reason == "max_tokens":
                ...
                    explanation="Verification incomplete (finish call truncated at max_tokens)",

Over the run's 223 verify checkpoints, 182 are incomplete. Their self-declared reasons:

RUN=~/.openant/projects/gadievron/raptor-e2e-20260818/scans/7dbf9c691d7/python
python3 - <<'EOF'
import json, glob, os, collections
D = os.path.expanduser("$RUN/verify_checkpoints"); h = collections.Counter()
for f in glob.glob(D + "/*.json"):
    if os.path.basename(f) in ("_fingerprint.json", "_summary.json"): continue
    v = json.load(open(f)).get("verification", {})
    if v.get("incomplete") and v.get("explanation"):
        h[v["explanation"][:60]] += 1
for k, c in h.most_common(3): print(c, "|", k)
EOF

92 | Max iterations reached
31 | Verification incomplete (finish call truncated at max_tokens)
11 | Verification incomplete (no tool calls)

31 of the 182 incomplete verifications self-declare stop_reason == "max_tokens". The other
151 do not: 92 record "Max iterations reached", 11 record "no tool calls", and 48 are errored records
that carry no explanation at all. Those 151 are not attributed to the 4096 cap here.

Sibling with the same shape: utilities/agentic_enhancer/agent.py:253-259 calls
self.binding.adapter.complete(..., max_tokens=MAX_TOKENS_PER_RESPONSE) with the same 4096 constant
(agent.py:37), on a stage that is on by default (core/scanner.py:128,
enhance_mode: str = "agentic").

Why it matters

PR #242's comment is an explicit statement that 8192 is too small for a thinking-era model on a large
unit. verify runs a multi-turn tool-using agent on the largest, most complex units in the scan, on
the same model class, at half that figure — and by a code path the fix could not reach, because
the fix lives in simple_text's signature and verify never calls it.

The verifier is careful about the consequence: a truncated finish is deliberately downgraded to
incomplete rather than trusted (:487-508), which is the right call. But an incomplete
verification is an un-adjudicated candidate, and on this run 182 of 223 candidates were
un-adjudicated. At least the 31 self-declared truncations are directly attributable to the budget.

Related: #212 reports a large Stage-2 coverage loss on a different route. That issue's own analysis
identifies a different proximate cause; this issue is about the budget path, which is route-agnostic.

Suggested fix

1. Raise finding_verifier.MAX_TOKENS_PER_RESPONSE to match simple_text's 20000, or better, have
   the phase read its budget from configuration with 20000 as the default, so the two cannot drift
   again.
2. Do the same for utilities/agentic_enhancer/agent.py:37.
3. Consider making the budget a property of the PhaseBinding rather than a module constant in each
   consumer, so simple_text and adapter.complete callers get the same number by construction.
   That would collapse the 16-site census above into one place.
4. Add a test that asserts no production call site pins a budget below the simple_text default
   without an explicit, commented justification.

Note that a resumed run will not show the effect of a budget change: the phase fingerprint does not
include generation parameters, so old checkpoints are adopted, and errored checkpoints are restored
rather than retried. Both are filed as #286/#287.

What I am not claiming

• I am not claiming the 4096 cap explains all 182 incomplete verifications. Exactly 31 of
  them self-declare stop_reason == "max_tokens". The 92 "Max iterations reached", the 11 "no tool
  calls", and the 48 errored records are unattributed here.
• I am not claiming a budget increase repairs a measured percentage of anything. On a resumed
  run it repairs 0%, for the checkpoint reasons above. On a fresh scan the effect is unmeasured —
  I did not re-run.
• I am not contradicting Stage 2 loses ~30% of candidates to provider moderation refusals, with no aggregate reporting #212's own diagnosis of its refusals. That is a different route and a
  different proximate cause; the two are compatible.
• The census numbers are static counts at HEAD b501962 produced by the commands shown. They count
  call sites, not calls made at runtime.

---

Sibling gap in the same loop — #291 (added 2026-08-22, back-link). #291 reports that verify
appends raw tool results with no input cap while enhance caps both and documents why. It calls this
"the second instance of a verify-specific gap where the general fix exists elsewhere in-tree" and
names this issue as the first.

The two are independent and neither subsumes the other: this issue is the output budget
(max_tokens=4096 passed directly to adapter.complete, bypassing the simple_text default raised
in PR #242); #291 is the input side (no cap on what gets appended to the conversation). #291's own
"What I am not claiming" section says so — "I am not claiming the 4096 output budget (#290) is or is
not the dominant factor. These are independent gaps in the same loop and either may matter more."

Recorded here because the reference was one-way: #291 cited this issue twice and this body cited
nothing back, so a reader arriving here had no route to the sibling.

[gadievron]

Fix this one after the others, or the fix will look like it worked when it did not.

A budgets-first change is silently absorbed on any resumed run, for three independent reasons that
each land elsewhere:

• the verify checkpoint fingerprint omits generation parameters (Backend fingerprint omits generation parameters, so changing max_tokens does not invalidate verify checkpoints #287), so raising max_tokens
  does not invalidate existing checkpoints;
• _cp_is_error cannot fire on the errored records (Verify checkpoints: errored units are restored as completed on resume, because _cp_is_error tests a value nothing writes on the normal path #286 — they carry verification key-shape
  ('incomplete',) with no correct_finding key), so those units are restored as finished work
  rather than re-attempted;
• the step report cannot express the failure anyway (scan.report.json is constructed without status or errors and never reads them from step reports #285), so the post-fix run reports
  status: "success", errors: [].

Net effect: re-running after a budget change re-verifies nothing, and every signal an operator would
check says the change succeeded.

Suggested order: #285 (status honesty) → #286/#296 (one vocabulary for "this unit failed") →
#287 (fingerprint key) → then this issue. With those in place a budget change becomes measurable:
the failure is visible, the retry actually happens, and stale checkpoints invalidate.

If you would rather validate this issue in isolation first, use a clean scan directory — that
sidesteps all three, at the cost of a full re-scan.

[gadievron]

Scope correction from the run's own checkpoints: this issue is real, but it does not explain the 92
"Max iterations reached" verifications. Those are input-bound, not output-bound.

I had assumed the 4096 pin was a plausible cause for the 92 and was about to test it with a paid
re-run. The existing checkpoints settle it for free, and refute that assumption.

The verify checkpoints record usage.input_tokens and usage.output_tokens separately, so
output-per-turn is directly computable:

group                n    out/turn (median)   in/turn (median)   % of the 4096 cap
maxiter (the 92)     92   1,040               52,909             25%
truncated (the 31)   31   1,623               45,188             40%
completed (the 41)   41     982               24,434             24%

units among the 92 whose output/turn is within 10% of the cap:  0 of 92

The 92 emit a median of 1,040 output tokens per turn against a 4,096 cap — statistically
indistinguishable from the units that completed successfully (982), and not one of the 92 comes
near the ceiling. A budget that is never reached cannot be what stops them. They also all sit at
exactly iterations=20, whereas completed units finish at a median of 10 — they exhaust the loop,
they are not cut off mid-turn.

What still justifies this issue: the 31 verifications that self-declare
"Verification incomplete (finish call truncated at max_tokens)" are genuine budget truncations, and
finding_verifier.py:405-407 still calls adapter.complete directly with
MAX_TOKENS_PER_RESPONSE = 4096, so PR #242's raised default never reaches this stage on any route.
That is a real defect worth fixing on its own terms — I am narrowing the claim, not withdrawing it.

What this issue should not be sold as: a fix for the 92. If someone raises the budget and expects
the incomplete rate to drop, they will be disappointed, and — per a reviewer's point — a larger output
budget grows the conversation faster, which makes the input pressure worse.

Where the 92 actually point: the same table shows them consuming 52,909 input tokens per turn
against 24,434 for completed units — 2.2x. That is the unbounded tool-result accumulation filed as
#291 (verify appends json.dumps(outcome) raw while enhance caps both inputs). I have added the
same evidence there.

What is still open: this census eliminates the output budget as the cause of the 92. It does not
separate "input growth starves the loop" from "these units genuinely need more than 20 turns". The
experiment that would separate those is an input-cap arm or a raised MAX_ITERATIONS arm — not a
max_tokens arm.

A gap worth noting for anyone re-deriving this: the checkpoints record totals, iterations, and
the usage split, but no per-turn stop_reason. That is why this needed a census rather than a
lookup, and why the 31 are identifiable only through a string in explanation.

[gadievron]

Fix opened in #382 (CI green, all checks passing). Both agent loops (verify + enhance) now take their output budget from llm.helpers.DEFAULT_MAX_TOKENS — the SAME constant as simple_text's default — so the two budget paths cannot drift apart again (4096 → 20000, a cap not a floor). Sequenced behind #285/#286/#287 as the ordering comment requested (#376/#377/#378), so a budget change is now measurable. Scope notes in the PR: the config/PhaseBinding suggestions are architecture changes (deferred); the 16-site census guard is not added (the other 4096 sites are single-shot calls this issue did not analyze). The input-side sibling gap remains #291.