feat(agent): bound, isolate, and account for every drive

CHANGELOG.md

@@ -37,6 +37,17 @@ match exactly, or the workflow fails the publish.
   quarantine, and a target project's configured retries count each test's
   final attempt as its outcome.
 
+- **The pipeline always terminates, fails locally, and shows its cost.** Every
+  external await is now time-boxed (model calls — with one backed-off retry on
+  a transient failure — shell commands, Playwright invocations, the rac graph
+  export, git); a runner exception counts as a failed fidelity attempt with
+  the reason on the verdict instead of aborting the gate; one capability's
+  failure in a scoped run no longer discards its siblings' results;
+  observations are capped per turn so the transcript can't grow without
+  bound. The bundled adapters surface provider token usage (shown in the QA
+  summary), and `--verbose` streams a per-turn audit trail (tool calls,
+  errors, model latency) to stderr.
+
 ## 2026.07.1 — the "any model" release
 
 The release that makes **bring-your-own-model** mean *any* model — and proves Proofkeeper on itself. Everything since the first cut:

lore-proofkeeper/designs/design-drive-resilience.md

@@ -0,0 +1,98 @@
+---
+schema_version: 1
+id: PK-KWFWJNJ7NHN4
+type: design
+---
+# Drive Resilience — Bounded Time, Isolated Failure, Visible Cost
+
+## Context
+
+A review found the pipeline had no wall-clock bounds anywhere (`grep` for
+timeout/retry found only the extension loader), all-or-nothing failure in the
+scoped pool and the fidelity gate, unbounded transcript growth, and zero cost
+or audit visibility. This design adds the resilience layer without changing
+any verdict semantics.
+
+## User Need
+
+An operator running Proofkeeper unattended — in CI or over a large scoped
+change — needs the run to always terminate, to lose only the failing
+capability when something breaks, and to be able to see afterwards what the
+agent did and what it cost.
+
+## Design
+
+- **Time-box every external await.** The drive wraps `model.complete` in a
+  timeout (default 2 min, `modelTimeoutMs`) with one backed-off retry
+  (`modelRetryBackoffMs`, default 2 s); `runCommand` gains
+  `timeout`/`maxBuffer` (2 min / 16 MB), mirrored in the emitted spec's inline
+  helper so record and replay stay in agreement; the Playwright invocation
+  (10 min, `timeoutMs` option), the rac export (2 min), and the git diff
+  (1 min) get `execFile` timeouts.
+- **Failure is local.** The fidelity gate wraps each attempt: a runner
+  exception is a failed attempt recorded on `FidelityVerdict.errors`, so "the
+  test failed" and "the run broke" are distinguishable and the gate always
+  completes. The scoped pool wraps the whole per-capability `runQa` in
+  try/catch, filling the existing `ScopedCapabilityResult.error` seam instead
+  of rejecting the pool.
+- **Bounded observation.** `renderObservation` clips text and ARIA blocks at
+  8,000 chars each with an explicit `[truncated N chars]` marker; console and
+  network windows were already bounded.
+- **Visible cost and conduct.** `ModelResponse.usage` (adapter-mapped from
+  both providers) accumulates into `DriveResult.tokens`, rendered in the QA
+  summary. `DriveOptions.onStep` emits a per-turn audit event (tool calls,
+  outcomes, model latency); the CLI's `--verbose` writes it to stderr as it
+  happens.
+
+## Constraints
+
+- No verdict semantics change: stable still means N green attempts; an errored
+  attempt is simply a failed one with a reason.
+- Additive public surface only (`usage`, `tokens`, `errors`, `onStep`,
+  timeout options); every existing caller compiles unchanged.
+- The emitted spec's helper must match `runCommand` byte-for-byte in
+  behavior — the two are changed together.
+
+## Rationale
+
+Timeouts belong at each shell-out/await site (the only places a hang can
+start), not in a global watchdog that would kill work it cannot attribute.
+Retrying exactly once catches the dominant transient-blip case without hiding
+a dead provider. Filling the pool's existing `error` field keeps the scoped
+result shape stable for the PR-comment renderer.
+
+## Alternatives
+
+- **AbortController threaded through ModelClient.** Deferred: it changes the
+  BYO-model interface every custom adapter implements; a race-based timeout
+  unblocks the loop today and a signal can be added additively later.
+- **A global drive watchdog.** Rejected: coarser than per-site caps and it
+  cannot say *what* hung.
+- **Configurable observation budget.** Deferred until a real page needs it;
+  the marker makes truncation visible when it happens.
+
+## Accessibility
+
+Not applicable — timeouts and logging; the `--verbose` stream is plain text.
+
+## Style Guidance
+
+Timeout errors name the cap that fired ("model call timed out after 120000ms")
+and retry errors name both failures, so a transcript reads as a diagnosis.
+
+## Open Questions
+
+- Whether a spend ceiling (`--max-tokens-budget`) should abort a drive
+  mid-run. Usage is now measured, which is the prerequisite.
+
+## Related Requirements
+
+- req-drive-resilience
+
+## Related Roadmaps
+
+- autonomous-qa-enhancements
+
+## Status
+
+Accepted

lore-proofkeeper/requirements/req-drive-resilience.md

@@ -0,0 +1,62 @@
+---
+schema_version: 1
+id: PK-KWFWJMV2DBWA
+type: requirement
+---
+# Drive Resilience — Timeouts, Isolation, Budgets, Audit
+
+## Problem
+
+Nothing in the pipeline bounded wall-clock time or spend, and one failure could
+destroy unrelated work. A stalled model call, a hung shell command, a hung
+Playwright or rac process each blocked the pipeline forever; a single transient
+provider error aborted an otherwise-recoverable capability; a throwing drive
+inside a scoped run rejected the pool and discarded every sibling capability's
+completed result; a runner exception aborted the fidelity gate instead of
+counting as a failed attempt. Meanwhile the transcript grew unbounded (a full
+observation re-appended every turn), token spend was invisible, and the loop
+left no record of what the agent actually did.
+
+## Requirements
+
+- [REQ-001] Every external await is time-boxed: model calls, shell commands (recorded and replayed), Playwright invocations, the rac graph export, and the git diff each carry a wall-clock cap whose expiry surfaces as an error, never a hang.
+- [REQ-002] A failed model call is retried once with backoff before the drive gives up, and the final error names both failures.
+- [REQ-003] A runner exception during the fidelity gate counts as a failed attempt with a recorded reason on the verdict; the gate always completes its N attempts.
+- [REQ-004] In a scoped run, one capability's exception becomes that capability's error entry; sibling capabilities' results are never discarded.
+- [REQ-005] Observation text and ARIA blocks are capped per turn with an explicit truncation marker, bounding transcript growth.
+- [REQ-006] Provider-reported token usage is surfaced by the bundled adapters, accumulated per drive, and shown in the QA summary; `--verbose` logs each turn's tool calls, errors, and model latency as an audit trail.
+
+## Success Metrics
+
+- A stalled model call errors at the cap instead of hanging; a transient 5xx
+  no longer aborts a capability.
+- A scoped run with one throwing capability still reports every sibling's
+  verdict.
+- A drive on a usage-reporting provider prints its token totals; a hung
+  `sleep`-style command errors within its cap.
+
+## Risks
+
+- Caps that are too tight fail slow-but-healthy runs. Mitigation: generous
+  defaults (2 min model/command, 10 min per Playwright invocation), and the
+  model timeout and command timeout are overridable.
+- A retry doubles cost on genuinely dead providers. Mitigation: exactly one
+  retry, with both errors reported.
+
+## Assumptions
+
+- Provider `usage` fields (Anthropic `input_tokens`/`output_tokens`, OpenAI
+  `prompt_tokens`/`completion_tokens`) remain stable contract surfaces.
+- The head of a page's text/ARIA carries the signal locators need, so clipping
+  the tail loses little.
+
+## Related Roadmaps
+
+- autonomous-qa-enhancements
+
+## Verified By
+
+- `tests/drive-loop.test.ts`
+- `tests/fidelity.test.ts`
+- `tests/scoped-qa.test.ts`
+- `tests/observe.test.ts`

lore-proofkeeper/roadmaps/autonomous-qa-enhancements.md

@@ -86,6 +86,14 @@ finish, assertion-free sessions refused, exact locator matching on record and
 replay, and contract anomalies (schema versions, empty or retried reports)
 refused rather than guessed at. Serves the faithful-tests outcome at its core.
 
+### Drive resilience
+
+Bound every external await (model, shell, Playwright, rac, git) with a
+wall-clock cap, retry transient model failures once, isolate one capability's
+failure from its siblings, cap observation growth, and make token spend and
+per-turn conduct visible. Serves the fast-reliable-scoped-QA outcome under
+real-world failure.
+
 ## Success Measures
 
 - A pull request shows exactly one Proofkeeper QA comment regardless of how many

src/agent/adapters/claude.ts

@@ -38,6 +38,7 @@ interface AnthropicContentBlock {
 interface AnthropicMessage {
   stop_reason?: string;
   content: AnthropicContentBlock[];
+  usage?: { input_tokens?: number; output_tokens?: number };
 }
 
 /** The slice of the Anthropic SDK client this adapter calls. */
@@ -100,8 +101,12 @@ export function fromAnthropicResponse(message: AnthropicMessage): ModelResponse
       textParts.push(block.text);
     }
   }
-  if (toolCalls.length > 0) return { toolCalls };
-  return { done: textParts.join("\n") };
+  const usage =
+    message.usage !== undefined
+      ? { usage: { inputTokens: message.usage.input_tokens ?? 0, outputTokens: message.usage.output_tokens ?? 0 } }
+      : {};
+  if (toolCalls.length > 0) return { toolCalls, ...usage };
+  return { done: textParts.join("\n"), ...usage };
 }
 
 export class ClaudeModelClient implements ModelClient {

src/agent/adapters/openai.ts

@@ -54,6 +54,7 @@ interface OpenAIResponseMessage {
 
 interface OpenAICompletion {
   choices?: { message?: OpenAIResponseMessage }[];
+  usage?: { prompt_tokens?: number; completion_tokens?: number };
 }
 
 /** The slice of `fetch` this adapter calls. Inject a double for tests. */
@@ -116,8 +117,17 @@ export function fromOpenAIResponse(completion: OpenAICompletion): ModelResponse
     const name = call.function?.name;
     if (name) toolCalls.push({ name, arguments: parseToolArguments(call.function?.arguments) });
   }
-  if (toolCalls.length > 0) return { toolCalls };
-  return { done: message.content ?? "" };
+  const usage =
+    completion.usage !== undefined
+      ? {
+          usage: {
+            inputTokens: completion.usage.prompt_tokens ?? 0,
+            outputTokens: completion.usage.completion_tokens ?? 0,
+          },
+        }
+      : {};
+  if (toolCalls.length > 0) return { toolCalls, ...usage };
+  return { done: message.content ?? "", ...usage };
 }
 
 export class OpenAICompatibleModelClient implements ModelClient {