[stacked on #2136 ← #2124] Add bridge record mode, real provider recordings, and the parity harness - #2153
Merged
SawyerHood merged 12 commits intoAug 21, 2026
Conversation
SawyerHood
force-pushed
the
bb/provider-recordings
branch
from
August 21, 2026 05:50
b8f4415 to
2b408d8
Compare
Collaborator
Author
|
Coordinator review — APPROVE (pending CI green on This is the regression oracle the whole migration depends on (A2/A3), and it's built right:
Based on
|
SawyerHood
added a commit
that referenced
this pull request
Aug 21, 2026
… merge) Duplicates PR #2153 (bb/provider-recordings, based on main). WS1b-codex needs the recorded codex cells and the parity harness as its regression oracle, and the stack base (WS1a #2136 on the contract #2124) does not contain them. Drop this commit when the stack rebases onto main after #2153 lands.
SawyerHood
added a commit
that referenced
this pull request
Aug 21, 2026
… ratchet A full-stack dry run surfaced a scope gap: the ratchet's test exclusions covered test/, tests/, and __tests__/ but not this repo's src/testing/ convention, so the parity and conformance harnesses landing in #2153 (which must name providers to map each recording to its bridge module and fake child) registered as 10 new core carve-outs. That is test tooling, the same category as __tests__, not a provider branch in core. Add "testing" to the excluded path segments, document why in the header, and add a fixture test proving a provider id under src/testing/ does not count. Regenerated baseline: 213 references across 56 files (one existing src/testing hit was already being miscounted as core). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
SawyerHood
added a commit
that referenced
this pull request
Aug 21, 2026
… ratchet A full-stack dry run surfaced a scope gap: the ratchet's test exclusions covered test/, tests/, and __tests__/ but not this repo's src/testing/ convention, so the parity and conformance harnesses landing in #2153 (which must name providers to map each recording to its bridge module and fake child) registered as 10 new core carve-outs. That is test tooling, the same category as __tests__, not a provider branch in core. Add "testing" to the excluded path segments, document why in the header, and add a fixture test proving a provider id under src/testing/ does not count. Regenerated baseline: 213 references across 56 files (one existing src/testing hit was already being miscounted as core). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
SawyerHood
added a commit
that referenced
this pull request
Aug 21, 2026
… merge) Duplicates PR #2153 (bb/provider-recordings, based on main). WS1b-codex needs the recorded codex cells and the parity harness as its regression oracle, and the stack base (WS1a #2136 on the contract #2124) does not contain them. Drop this commit when the stack rebases onto main after #2153 lands.
SawyerHood
added a commit
that referenced
this pull request
Aug 21, 2026
… merge) Duplicates PR #2153 (bb/provider-recordings, based on main). WS1b-codex needs the recorded codex cells and the parity harness as its regression oracle, and the stack base (WS1a #2136 on the contract #2124) does not contain them. Drop this commit when the stack rebases onto main after #2153 lands.
SawyerHood
changed the title
SawyerHood
force-pushed
the
bb/provider-recordings
branch
from
August 21, 2026 10:01
0923c1c to
39066e1
Compare
SawyerHood
changed the base branch from
main
to
bb/ws1a-assembler-testing-kit-stack-on-2124-thr_jkbj56vr97
August 21, 2026 10:01
This was referenced Aug 21, 2026
SawyerHood
added a commit
that referenced
this pull request
Aug 21, 2026
…lanes
recordings/parity-allowlist.json names every byte-inequivalence this PR
intends against origin/main, per cell, with the reason — 26 entries, all
scoped to provider codex (claude-code and acp-cursor replay with zero diffs):
- events /*/item/presentation on the 15 cells that have items — the new
presentation field, items otherwise byte-identical;
- events /*/item/server on user-question — server "bb" on the bb-injected
AskUserQuestion call;
- subagent events /6 and /28 and rows /0/children/0/{toolName,output} — the
native sub-agent spawn is a delegation item, not a spawnAgent tool item;
- one event index per goal-bearing cell (archived-resume, compaction,
missing-rollout, resume, steer, stop-interrupt) — thread/goal/cleared is
the provider-codex/goal extension state with a null payload.
The codex bridge→runtime.current.ndjson lanes are re-recorded against this
bridge with `pnpm rerecord --provider codex --plan-with <main checkout>`
(#2153's mechanism); the recorded lanes and the other providers' current
lanes are untouched. Row-count pins unchanged; self-suite 43/43.
Co-Authored-By: Claude <noreply@anthropic.com>
SawyerHood
added a commit
that referenced
this pull request
Aug 21, 2026
…lanes
recordings/parity-allowlist.json names every byte-inequivalence this PR
intends against origin/main, per cell, with the reason — 26 entries, all
scoped to provider codex (claude-code and acp-cursor replay with zero diffs):
- events /*/item/presentation on the 15 cells that have items — the new
presentation field, items otherwise byte-identical;
- events /*/item/server on user-question — server "bb" on the bb-injected
AskUserQuestion call;
- subagent events /6 and /28 and rows /0/children/0/{toolName,output} — the
native sub-agent spawn is a delegation item, not a spawnAgent tool item;
- one event index per goal-bearing cell (archived-resume, compaction,
missing-rollout, resume, steer, stop-interrupt) — thread/goal/cleared is
the provider-codex/goal extension state with a null payload.
The codex bridge→runtime.current.ndjson lanes are re-recorded against this
bridge with `pnpm rerecord --provider codex --plan-with <main checkout>`
(#2153's mechanism); the recorded lanes and the other providers' current
lanes are untouched. Row-count pins unchanged; self-suite 43/43.
Co-Authored-By: Claude <noreply@anthropic.com>
The bootstrap tees both sides of the runtime wire, bridges tee their provider child through experimental_recordProviderChildIo (Claude through the Agent SDK spawn seam, pi at its SDK event boundary), the daemon forwards the directory, and the runtime scopes it per provider. Off by default. Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
The shell env doubles as the agent shell environment, so the variable reached provider children through envVars. Read it from the daemon's own env so only the bridge process sees it. Co-Authored-By: Claude <noreply@anthropic.com>
52 recorded cells across codex, claude-code, acp-cursor and pi, redacted by scripts/provider-recordings/redact.mjs; a generic replay child that turns a recording into the fake provider; the replay harness, allowlist diff and `pnpm parity` CLI; and the self-suite that pins per-cell counts and checks every replayable cell reproduces its recording through the current bridge. Co-Authored-By: Claude <noreply@anthropic.com>
…on option Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
…uiet period Co-Authored-By: Claude <noreply@anthropic.com>
…any machine The recorded cwd and the Claude source transcript belong to the recording machine; CI has neither. Each replay now rewrites cwd to a temp workspace, gives the Claude bridge its own CLAUDE_CONFIG_DIR, and seeds a minimal source transcript for every recorded thread/fork. Co-Authored-By: Claude <noreply@anthropic.com>
Harness fixes found while stacking the codex bridge migration (#2164) on WS1a's v3-only assembler. None of them changes a recording; all are safe on this branch (self-suite 43/43, codex/claude/acp recorded conformance green). - A recording is never rewritten. `pnpm rerecord [--plan-with <checkout>]` writes the current bridge's side of the wire to bridge→runtime.current.ndjson beside the recorded lane (request ids kept by method and order so the recorded runtime responses still match; lines placed after the runtime entry sent last before them). The self-suite and recorded conformance pin and compare against the current lane when one exists; `pnpm parity` paces a pre-migration leg from the recorded lane and the current leg from the current lane (`planFromCurrentLane`), so each assembler plans from a lane it parses whole. `--plan-with` names a checkout whose assembler parses the recorded lane for the re-recording itself. - `pnpm parity --dump-dir <dir>` writes both legs' normalized event and row lists per cell, for allowlist entries that must name a list index (the CLI renders values truncated to 160 chars). - The replay drains the pre-request stream (300 ms quiet once the gates are met) before sending each runtime request, and the replay child leaves a 50 ms gap after a response line: a steer's ack follows an `await` on the bridge's own request, and on a loaded runner the next notification otherwise lands in the same read and the ack moves one position (two CI flakes on #2164). - The self-suite holds allowlist entries to their form (PR, reason, path) instead of requiring an empty list, which a migration PR cannot satisfy; a non-replayable provider whose recorded lane predates the grammar is reported, not failed, until re-recorded live. Co-Authored-By: Claude <noreply@anthropic.com>
…nt lane Co-Authored-By: Claude <noreply@anthropic.com>
The delta collector lives at @bb/provider-bridge-protocol/testing on WS1a, not @bb/agent-runtime/test; provider-parity drops its agent-runtime dependency. The recorded bridge lanes are v2, which the v3-only assembler refuses, so every replayable cell gets a bridge→runtime.current.ndjson written by `pnpm rerecord --plan-with <main checkout>` through this branch's bridges — redacted in place, assembling to the unchanged pins. Co-Authored-By: Claude <noreply@anthropic.com>
SawyerHood
force-pushed
the
bb/provider-recordings
branch
from
August 21, 2026 10:45
39066e1 to
d606b64
Compare
SawyerHood
added a commit
that referenced
this pull request
Aug 21, 2026
…lanes
recordings/parity-allowlist.json names every byte-inequivalence this PR
intends against origin/main, per cell, with the reason — 26 entries, all
scoped to provider codex (claude-code and acp-cursor replay with zero diffs):
- events /*/item/presentation on the 15 cells that have items — the new
presentation field, items otherwise byte-identical;
- events /*/item/server on user-question — server "bb" on the bb-injected
AskUserQuestion call;
- subagent events /6 and /28 and rows /0/children/0/{toolName,output} — the
native sub-agent spawn is a delegation item, not a spawnAgent tool item;
- one event index per goal-bearing cell (archived-resume, compaction,
missing-rollout, resume, steer, stop-interrupt) — thread/goal/cleared is
the provider-codex/goal extension state with a null payload.
The codex bridge→runtime.current.ndjson lanes are re-recorded against this
bridge with `pnpm rerecord --provider codex --plan-with <main checkout>`
(#2153's mechanism); the recorded lanes and the other providers' current
lanes are untouched. Row-count pins unchanged; self-suite 43/43.
Co-Authored-By: Claude <noreply@anthropic.com>
SawyerHood
added a commit
that referenced
this pull request
Aug 21, 2026
…le ProviderInfo (#2148) > **Stacked on #2153 (recordings + parity harness) ← #2136 (WS1a) ← #2124 (contract)** — re-stacked linearly at the coordinator's request so the full stack merges conflict-free. Base: `bb/provider-recordings`. DO NOT MERGE — coordinator review. ## What was wrong Provider facts lived in core tables keyed by provider id, and the shared server↔daemon execution contract carried Claude/Codex-named knobs: - `PRODUCT_PROVIDER_ORDER`, `RESERVED_PROVIDER_ID_OWNERS`, `PRODUCT_DEFAULT_PROVIDER_ID` in the server registry. - Five `codex*`/`claudeCode*` keys on the shared `AppSettings`, resolved in `thread-commands.ts` by `providerId === …`. - `claudeCodePermissionMode`, `workflowsEnabled`, `memoryEnabled`, `providerSubagentsEnabled` on `RuntimeThreadExecutionOptions` (the G2 allowlist entry), plus a `providerId !== "claude-code"` guard in the runtime. - Three copies of the Claude model catalog (plugin, server, app) and a vendored `PLACEHOLDER_PROVIDER_INFOS` roster in the app; sign-in hints, install URLs, brand prefixes and plan-mode copy in per-provider tables on web and mobile. - The contract PR added the additive declaration / `ProviderInfo` fields; nothing populated or consumed them (G10 gaps: `family`, `models`, `env`, `deriveProviderOptions`, `promptMode`). ## What changed Spec: `WS2a-registry.md` layers 1–7 (docs/provider-plugin-api.md §1, §6). **L1 — `bb.providers.register` + single `ProviderInfo`.** New `bb.providers` namespace on `BbPluginApi`; `bb.agents.experimental_registerProvider` stays as an alias sharing the same registration table (additive-then-delete: both work; the four first-party `server.ts` files moved to the new call). `buildPluginProviderRegistration` now projects `strings`, `reasoningLevels` (declared options, or the coarse ladder labelled), `serviceTiers` (declared, or `default`/`fast` when `supportsServiceTier`), `extensionKinds` namespaced by the **owning plugin id**, and `family`. New declaration fields, validated in the shared host policy: `experimental_family`, `experimental_models.fallback`, `experimental_env.passthrough`, `experimental_deriveProviderOptions`. **L2 — options hook + `promptMode`.** The server runs the owning plugin's `experimental_deriveProviderOptions(ctx)` on every session/turn command (`ctx = { threadId, projectId, model, permissionMode, promptMode?, settings }`, `settings` = the plugin's own non-secret `bb.settings` values, read synchronously) and sends the bounded-JSON result as a REQUIRED `options.providerOptions`. `promptMode: "plan"` is set only when the prompt carries the provider's *declared* `plan` composer command (Pi never sees it). `claudeCodePermissionMode` + the three booleans are deleted from `runtimeThreadExecutionOptionsSchema`, `ProviderExecutionContext`, and the runtime's literal-id guard; the bridge adapter merges the bag over the static bridge options instead of rest-sweeping "provider-flavored" fields. `bridgeExecutionOptionsSchema` gains `promptMode`. The Claude plugin maps `promptMode` to its native plan flag inside its own bag; the Claude/Codex bridges already read their knobs from the bag, so no bridge code changed. **L3 — provider-scoped settings.** The five keys are gone from `appSettingsSchema`; `provider-codex` defines `memoryEnabled`/`subagentsDisabled`, `provider-claude-code` adds `workflowsDisabled`, both derived into `providerOptions` by their hooks. Drizzle data migration `0105_provider_settings_to_plugins` copies stored values into `plugin_settings` (existing plugin rows win) and retires the shared rows. `supportsWorkflows` is deleted from the declaration and `ProviderServerCapabilities` (it only ever fed `workflowsEnabled`). Web: the hardcoded Codex/Claude settings pages, `SETTINGS_PROVIDER_ENTRIES`, `/settings/providers/:providerId` are gone; a generic **Settings → Providers** bucket lists every registered provider with order + default controls, and per-provider knobs live on the plugin's settings page. Mobile: `ProviderSettingsScreen` and its route are gone (plugin detail already renders `bb.settings` forms). **L4 — flat ids + user order.** `RESERVED_PROVIDER_ID_OWNERS`, `reservedProviderIdProblem`, `PRODUCT_PROVIDER_ORDER`, `PRODUCT_DEFAULT_PROVIDER_ID` deleted. Collision = first live registration wins (`assertProviderRegistrable` keeps only the "has a bridge" check). `registry.list()` orders by an install rank the plugin runtime supplies (bundled plugins by their `BUNDLED_PLUGINS` index — reordered to codex, claude-code, pi, acp so today's picker order is preserved — then others by `installedAt`, then registration sequence), under two new app settings `providerOrder: string[]` and `defaultProviderId: string | null` (`bb settings general …` works unchanged, docs/configuration.md updated). `thread-default-policy` resolves the default as the user's choice when registered+available, else the first available listed provider. **L5 — duplicate catalogs.** Server `claude-code-fallback-models.ts` deleted; the model-list route serves `registration.fallbackModels` (from the declaration) on a transient probe failure. App `PLACEHOLDER_PROVIDER_INFOS` and `CLAUDE_CODE_PLACEHOLDER_MODELS` deleted (a cold cache waits for the first probe; a remembered per-routing roster still replays). The Claude catalog data moved to an import-free `src/model-catalog-data.ts` (the server loads `server.ts` with only the SDK root specifier resolvable, so it must not pull in `@get-bb/plugin-sdk/provider-bridge`). Web + mobile usage banners, the picker's brand strip, the missing-CLI install link, and the plan-mode permission display now read `ProviderInfo.strings` — the mobile usage table's keys (`claudeCode`, `cursor`) never matched the server's provider-id keys, which this fixes as a side effect. `effective-prompt-mode` keys on the presence of `strings.planModeCopy`, not `providerId === "claude-code"`. **L6 — directory.** `bb.sdk.providers.list()` already existed (verified). New `app.experimental_useProviders()` (`{ status, providers: ProviderInfo[] }`) in the SDK, app implementation, and test harness (`providers` option); `provider-retry` and `automations` read provider names from it instead of local tables. Icons remain vendored — see the api_to_audit entry for why. **`env.passthrough`** replaces the daemon's hardcoded `BB_CLAUDE_CODE_EXECUTABLE` forwarding: `bridgeLaunch.envPassthrough` (REQUIRED on the wire) names the variables; the runtime picks exactly those from the daemon env into the bridge process env. #2153's record-mode forward (`BB_PROVIDER_BRIDGE_RECORD_DIR`, read from the daemon's own env) stays beside it in `providerProcessEnvFromShellEnv` — it is a daemon concern, not a provider declaration. **Wire:** `HOST_DAEMON_PROTOCOL_VERSION` 148 (WS1a) → **149**; protocol.ts keeps all three log entries (147 contract, 148 WS1a, 149 this PR). Changes: `options` shape (fields removed, `providerOptions` required, `promptMode` optional) and `bridgeLaunch.envPassthrough` required. **Docs / surfaces:** docs/api_to_audit.md (registration entry rewritten for flat ids + install order; target-state-fields entry expanded with the hook/models/env/family and migration; new `experimental_useProviders` entry), bb-plugin-authoring SKILL.md (`bb.providers.register` with every new field), docs/configuration.md (Providers bucket, `providerOrder`/`defaultProviderId`, plugin-scoped knobs), docs/provider-bridge-protocol.md alias rename, echo-provider example. ### G10 doc-sync gaps (`provider-plugin-doc.test.ts`) `family`, `models`, `env`, `deriveProviderOptions`, `promptMode` → mapped to real fields. `maintenance` stays a gap, retargeted to the stabilization audit: it is a pure fold of the three `experimental_provider{Health,Usage,Installation}` booleans, and declaring the same three facts twice during the transition would break one-fact-one-place. ### Deliberately NOT done here (noted for WS4) - `AgentRuntimeSkillRoot` is still a closed union keyed by first-party id (WS1a's flag). Dissolving it needs the `skills/configure` root to carry both the skills dir and the plugin root (Claude consumes the latter), i.e. a bridge-protocol + Claude bridge change — WS4's lane. - `provider-registry.ts` keeps 9 `isAcpProviderId` hits for the dynamic ACP tier (custom agents resolved from config) — WS2b deletes the tier. ### Relation to WS1a Now stacked on top of WS1a (#2136), so there is no sibling overlap to reconcile: the restack commit ports the scripted-echo launch helper and WS1a's new dispatch / extension-payload tests to this PR's shapes (`envPassthrough`, `readSettings`, no `supportsWorkflows`) and keeps `HOST_DAEMON_PROTOCOL_VERSION = 149` with all three log entries. ## Guardrails **G1 (provider-literal ratchet, `scripts/check-provider-literal-ratchet.mjs` from #2120 — not in this base, so no baseline file is committed here; measured with the script checked out from that branch):** `214 → 149` references, `57 → 41` core files. | file | before | after | |---|---|---| | `apps/app/src/components/pickers/model-brand-prefix.ts` | 2 | 0 | | `apps/app/src/components/pickers/model-load-error-message.tsx` | 1 | 0 | | `apps/app/src/components/settings/UsageLimitsSettingsSection.tsx` | 2 | 0 | | `apps/app/src/components/settings/settings-nav.tsx` | 2 | 0 | | `apps/app/src/hooks/queries/system-queries.ts` | 7 | 0 | | `apps/app/src/views/SettingsView.tsx` | 4 | 0 | | `apps/mobile/src/data/settings/usage-limits-model.ts` | 8 | 0 | | `apps/mobile/src/screens/settings/ProviderSettingsScreen.tsx` | 4 | 0 (deleted) | | `apps/mobile/src/screens/settings/SettingsScreen.tsx` | 2 | 0 | | `apps/mobile/src/screens/shell/hrefs.ts` | 2 | 0 | | `apps/server/src/services/providers/provider-registry.ts` | 24 | 9 | | `apps/server/src/services/system/execution-options.ts` | 1 | 0 | | `apps/server/src/services/threads/thread-commands.ts` | 6 | 0 | | `apps/server/src/services/threads/thread-default-policy.ts` | 4 | 0 | | `packages/agent-runtime/src/execution-options.ts` | 1 | 0 | | `packages/client-core/src/prompt/effective-prompt-mode.ts` | 2 | 0 | | `packages/plugin-sdk/src/app-contract.ts` | 2 | 0 | **G2 (`provider-contract-purity`):** allowlist now **empty** — `claudeCodePermissionMode` removed with the field. **Registry equality:** `first-party-provider-plugins.test.ts` pins the client-read `ProviderInfo` fields (id, displayName, logoUrl, available, the three maintenance booleans, capabilities, composerActions) for codex / claude-code / pi / acp-cursor to their pre-PR values, plus the picker order, and asserts the new projections are filled. ## How I verified All with `--concurrency 4`: - `turbo run typecheck` — **74/74 tasks green** (whole repo). - `turbo run build` — 18/18 green. `turbo run lint` — green (pre-existing app warnings only). - `turbo run test` for `@bb/server` (193 files / 1817 tests; the only failure is the known local-umask `internal-skill-trees` case, passes in CI), `@bb/app` (414 files), `@bb/mobile` (124), `@bb/agent-runtime` (31), `@bb/host-daemon` (47), `@bb/domain`, `@get-bb/plugin-sdk`, `@bb/provider-bridge-protocol`, `@bb/host-daemon-contract`, `@bb/db`, `@bb/client-core`, `@bb/cli`, `@bb/server-contract`, `@bb/thread-view`, `@bb/scripts`, `@bb/plugin-build`, `@bb/demo-server`, and the plugins `provider-claude-code`, `provider-codex`, `provider-acp`, `provider-retry`, `automations`, `tasks` — all green. - Tests that fail before / pass after: `plugin-provider-registration.test.ts` (projection of strings/tiers/levels/kinds/family/models/env + the hook bound to settings + non-JSON rejection), `provider-registry.test.ts` (install-rank order, user order overlay, unknown default → null, freed ids), `thread-default-policy.test.ts` (user default + order), `thread-runtime-config.test.ts` (providerOptions per provider, hook context, promptMode only for plan-declaring providers), `first-party-provider-plugins.test.ts` (client-field pin), `migrate.test.ts` (0105 carries values, plugin rows win, shared rows retired), `ProvidersSettingsSection.test.tsx`, `execution-options.test.ts` (structural bag equality), mobile `settings-models.test.ts` (usage rows keyed by provider id). Not run: the live-CLI integration suite (`agent-runtime#test:integration`) and the corpus gate (not in this base). > AGENT GENERATED: by Claude Opus 5 --------- Co-authored-by: Claude <noreply@anthropic.com>
SawyerHood
added a commit
that referenced
this pull request
Aug 21, 2026
…rpus row snapshots, timeline perf, permission matrix (#2121) > **Stacked in the chain**: contract #2124 → WS1a #2136 → recordings #2153 → WS2a #2148 → **this PR** → codex #2164 → claude #2178 → acp. Base `bb/ws2a-registry-providerinfo-stack-on-2124-thr_hbi4kggyzb`. WS1a deleted the `ProviderAdapter` interface and the fake adapter, so the runtime permission matrix drives `handleRuntimeProviderRequest` through the real bridge-protocol adapter built for the scripted echo launch (`createProviderForId` + `createScriptedEchoLaunch`), with the initialize handshake setting `approvalEnforcedBy` and a canonical `interaction/request` on the wire. The 80 outcomes pinned on `main` are unchanged; the `tool_use` subject the v3 contract added gets 20 measured cells of its own (the type-level union guard demanded them). `resolvePermissionEscalation` now takes only the initiator, so the server table is 3 initiator cells + the 54 policy-shape cells. The row snapshots minted on `main` compare byte-identical (307/307, 0 diffs) under WS1a, #2153, and WS2a. Pre-stack history: `4af0367`. ## What was wrong The provider-plugin migration abandons byte-equivalence with the old goldens on purpose, which removes the regression oracle. Before any provider code moves, `main` needs machine checks on real data that every later layer can run: projected rows for the 307 production threads in the private corpus (A4), a timeline-build and event-size baseline, and the permission-decision matrix (A5/G12) pinned as a literal table before WS5 changes the unions. None of these existed. ## What changed No production code changes. Tests, test helpers, one script, one turbo task, docs. - **Corpus reader** — `packages/test-helpers/src/provider-corpus.ts`: `corpusAvailable()`, `listCorpusThreads({ provider?, reasons? })`, `loadCorpusThread(id)`. Reads `BB_PROVIDER_CORPUS_DIR` (`manifest.json`, `threads/<provider>/<id>/{meta.json,events.ndjson}`), validates rows at the boundary with zod (ids must be one safe path segment), resolves each thread through its manifest entry and fails unless `meta.json` and every event row agree on id, provider, reasons, and row count, and decodes event payloads with `@bb/domain` `parseStoredThreadEvent` + `buildThreadEventRow` — the same path as the server's `parseStoredEventRow`. `.gitignore` gets `**/provider-corpus/**` with re-includes for the two in-repo directories of that name. - **Row snapshots** — `apps/server/test/provider-corpus/row-snapshots.test.ts` + `corpus-harness.ts`. Each thread is inserted into in-memory SQLite with its original ids, sequences, and timestamps (raw `INSERT` so nothing is minted), then every timeline page is built via `buildThreadTimelineWithProfile` with the options the route uses — event budget from `defaultFeatureFlags`, inline-output limit from `DEFAULT_MAX_INLINE_OUTPUT_CHARS`, display name and plan command from the real provider registry (`createTestProviderRegistry()` loads the first-party plugin declarations) through `resolveProviderPlanCommand`, unhandled ops included, output truncation + preview — following the route's own `olderCursor`. Two variants per thread: `default` (turn rows summarized) and `nested` (`includeNestedRows`, children materialized). Snapshots go to `$BB_PROVIDER_CORPUS_DIR/snapshots/rows/<provider>/<threadId>.json`, keys sorted. Nothing is blanked: no wall-clock value reaches the rows, and write mode proves it by projecting every thread twice and requiring byte equality. Compare mode fails on any diff not covered by `snapshots/allowlist.json` (`threadId` | `provider` | `"*"` scope, JSON-pointer or `*`/`**` glob path, `pr`, `reason`), prints a unified diff for the first 3 differing threads plus a count, lists the entries it used, and fails on entries that cover nothing. - **Perf baselines** — `timeline-perf.test.ts`: 10 largest threads per provider, latest page and full page walk, 5 profiled builds each after a warm-up, stage p50s from `ThreadTimelineBuildProfile`, persisted `data` bytes median/p95/total, rows produced. Written to `snapshots/perf-baseline.json`; compare fails at baseline × 1.10 for build cost and × 1.15 for median event size. **Deviation from the brief, with data:** the gate uses a normalized cost — min build time ÷ min time of a fixed CPU workload that shares no code with the timeline (JSON codec + sort over a deterministic document, run once per sample right before the builds) — not raw p50/p95. Raw p50 of the same commit swung up to 30% between two back-to-back runs on this 16-core box at load ~6 (14 of 20 threads tripped a literal 1.10 gate on the very next run); each side's minimum discards its own contended samples, interleaving keeps both minima in one short window, and a workload outside the timeline path means a uniform regression still moves the ratio. Raw p50/p95 are still recorded and printed. Up to 3 attempts per thread (write mode keeps the median attempt; compare stops at the first pass), a 5 ms floor for tiny latest-page builds, and compare mode refuses a baseline written with different gate settings. The table header reports the load average and flags an oversubscribed machine. - **CI micro-benchmark** — same file, no corpus: `synthetic-thread.ts` builds a 10,019-event thread (every item kind, deltas, background tasks, usage events) and walks all 12 pages. Gate: minimum of 5 walks under 1,500 ms (local minimum 150–170 ms; the ceiling is ~10× so a slow runner passes while a quadratic regression still fails). - **Permission matrix** — `packages/agent-runtime/src/permission-matrix.test.ts`: the runtime chokepoint `handleRuntimeProviderRequest` over permission policy (5 members of the discriminated union) × approval subject (5: command, file_change, permission_grant, plan, tool_use) × `approvalEnforcedBy` (2) × deny availability (2) = 100 cells, each a literal; `satisfies Record<CellKey, Outcome>` plus `SameUnion` type guards against the domain unions (dropping a row or a union member fails `tsc`, verified on the stack), plus runtime assertions that every local subject kind parses as a payload. The request goes through WS1a's real bridge-protocol adapter. `apps/server/test/permissions/permission-matrix.test.ts`: `resolvePermissionEscalation` over the 3 initiators and the 54-cell runtime-permission-policy shape cross product (5 accepted), with the reviewer vocabulary pinned to the policy union at the type level. - **Seeds** — `scripts/provider-corpus/snapshot-rows.sh [write|compare]`, `@bb/server#test:provider-corpus` (uncacheable, `passThroughEnv` for the two variables — strict turbo env mode strips them from the plain `test` task), and a "Provider corpus" section in `docs/debugging-and-qa.md`. ### Permission matrix (runtime) Outcome: `forward` = reaches `onInteractiveRequest` (user decides); `auto-deny` = runtime answers deny; `encode-error` = runtime wants to auto-deny but `deny` is not in `availableDecisions`, so the provider gets a JSON-RPC error. Subject kind (command, file_change, permission_grant, plan, tool_use) never changes the outcome, so the table is collapsed over it (each row below is 5 cells). | policy (mode/escalation) | approvalEnforcedBy | deny available | outcome | | --- | --- | --- | --- | | accept-edits/ask | runtime | yes / no | forward | | accept-edits/ask | provider | yes / no | forward | | accept-edits/deny | runtime | yes | **auto-deny** | | accept-edits/deny | runtime | no | **encode-error** | | accept-edits/deny | provider | yes / no | forward | | auto/ask | runtime | yes / no | forward | | auto/ask | provider | yes / no | forward | | auto/deny | runtime | yes | **auto-deny** | | auto/deny | runtime | no | **encode-error** | | auto/deny | provider | yes / no | forward | | full/– | runtime | yes / no | forward | | full/– | provider | yes / no | forward | Bridge-kit predicate: `shouldAutoDenyInteractiveRequest` → ask: false, deny: true, null: false. Server escalation by initiator: user → ask; agent → deny; system → deny. (On `main` the function also took the thread and was measured identically for root, delegated-child, and fork threads; WS1a removed the unused argument.) Accepted runtime policy shapes (5 of 54): accept-edits/workspace/user/{ask,deny}, auto/workspace/automatic/{ask,deny}, full/full/–/–. Observations (pinned, not fixed): 1. Permission **mode** never auto-decides anything in the runtime or the server. Only escalation does, and escalation is purely a function of the turn initiator. Mode is translated into provider-native settings (codex `approvalPolicy`/sandbox, Claude SDK `permissionMode`) and enforced by the provider. So a runtime-enforced provider in `full` mode that did send an approval would prompt the user. 2. The `encode-error` cells: on an agent- or system-initiated turn, a runtime-enforced provider whose approval omits `deny` receives a JSON-RPC error instead of a decision. Codex forwards the provider's own decision list, so this is reachable in principle; no first-party bridge omits deny today. 3. `plan` approvals are treated like any other subject by the runtime: on a system-initiated turn with a runtime-enforced provider they are auto-denied. Claude is provider-enforced and always forwards `ExitPlanMode`, so only codex plan approvals can hit this. ## How you verified - `pnpm exec turbo run typecheck --filter=@bb/server --filter=@bb/test-helpers --filter=@bb/agent-runtime` — 6 tasks successful. - On the WS1a stack (`--concurrency 4`): `pnpm exec turbo run typecheck --filter=@bb/agent-runtime --filter=@bb/server --filter=@bb/test-helpers` green; `pnpm exec turbo run test --filter=@bb/agent-runtime --filter=@bb/test-helpers` — 31 files, 434 tests passed (102 of them the matrix); `pnpm exec turbo run test --filter=@bb/server` — 197 files, 1,886 tests passed, the one failure again the local umask assertion. With the corpus set, the row snapshots minted on `main` compare byte-identical under WS1a's v3 assembler: 307/307, 0 diffs. - On `main` before the stack, corpus absent: `pnpm exec turbo run test --filter=@bb/agent-runtime --filter=@bb/test-helpers` — 32 files, 504 tests passed (82 of them the matrix). `pnpm exec turbo run test --filter=@bb/server` — 196 files passed, 1 skipped (the row-snapshot suite), 1,884 tests passed; the single failure is the pre-existing `internal-skill-trees` file-mode assertion (this checkout's umask 0002 yields 0664 where the test expects 0644; it fails on clean `main` here and passes in CI). The two corpus suites report as skipped; the synthetic benchmark runs (`Synthetic 10019-event thread: 12 pages, 1165 rows projected, full walk p50 174 ms`). - Corpus present: `scripts/provider-corpus/snapshot-rows.sh compare` → 2 files, 328 tests passed (307 row snapshots + 20 perf threads + 1 synthetic). - Row snapshot write mode: **307 threads, 93,262 rows (top-level + nested children), 270.5 MB (283,656,293 bytes), 96.9 s wall**; compare mode on the same commit: 73.1 s, 0 diffs. After switching the harness from copied provider literals to the real registry, compare was still 307/307 with 0 diffs — the snapshots are byte-identical. - Perf gate on the final estimator: baseline written at load 9.2, compare on the same commit at load 5.7 passed 20/20 threads on the first attempt, ratios 0.80–1.09× of baseline. Every one of the 330,626 corpus events decodes with today's `parseStoredThreadEvent` (426 MB of `data`). - Allowlist machinery exercised by mutating one snapshot: compare fails with the unified diff; an entry covering `/variants/*/pages/*/rows/*/text` for that thread passes; a stale entry fails with "every snapshots/allowlist.json entry must cover at least one diff". - Exhaustiveness guards exercised: deleting one matrix row → `TS2741 Property '"full/-|plan|provider|deny-unavailable"' is missing`; dropping `"provider"` from the enforcer list → `SameUnion` becomes `false` and the `satisfies` rejects the extra keys. ### Perf baseline (write mode, this commit, load 9.2/16 cores; ms; norm = min build ÷ min `json-sort-v1` calibration) | thread | provider | events | data bytes p50/p95 | latest rows | latest p50/p95 ms | latest norm | pages | walk rows | walk p50/p95 ms | walk norm | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | thr_62bp724ett | claude-code | 10207 | 570/6375 | 18 | 23.3/27 | 0.103 | 8 | 241 | 922.3/1228 | 4.855 | | thr_zqfgksiw39 | claude-code | 9012 | 365/3587 | 50 | 90.5/119 | 0.388 | 7 | 279 | 532/564.4 | 2.466 | | thr_zc9yxt8mwk | codex | 8795 | 316/3025 | 10 | 24.9/58.3 | 0.102 | 5 | 45 | 323.1/395.5 | 1.510 | | thr_j4hfucc5bb | claude-code | 7679 | 331/1285 | 52 | 29.9/35.5 | 0.132 | 8 | 335 | 404.6/472.4 | 1.591 | | thr_f9jfidwbx8 | claude-code | 6163 | 454/4912 | 26 | 19.5/23.7 | 0.156 | 6 | 132 | 198.7/225.5 | 1.720 | | thr_w6i678ha85 | claude-code | 5826 | 371/2820 | 19 | 25.7/40 | 0.204 | 3 | 37 | 149.6/172.4 | 1.163 | | thr_qvau3b2d5b | claude-code | 5685 | 355/3613 | 111 | 36.9/50.4 | 0.298 | 5 | 168 | 154.3/173.4 | 1.311 | | thr_2pkc3q5imw | codex | 5487 | 316/3642 | 3 | 32.9/36.6 | 0.307 | 4 | 15 | 87/89.5 | 0.808 | | thr_7wvd28jzpj | claude-code | 5015 | 345/3497 | 36 | 25.8/31.6 | 0.219 | 3 | 94 | 130.7/137.4 | 1.102 | | thr_nbghhtnurj | codex | 4341 | 316/8824 | 11 | 20.8/27.8 | 0.180 | 4 | 35 | 90.4/94.6 | 0.849 | | thr_wb9c2q8gm4 | claude-code | 4228 | 378/2469 | 65 | 31.3/36.4 | 0.268 | 3 | 150 | 86.3/89 | 0.801 | | thr_ksr44swb3q | codex | 4063 | 316/4513 | 10 | 18.7/20.2 | 0.178 | 3 | 23 | 64/67.4 | 0.589 | | thr_tsfz8rtt72 | codex | 4055 | 316/4571 | 18 | 20.6/22 | 0.185 | 2 | 29 | 71/74.3 | 0.633 | | thr_qe8y73b26t | claude-code | 3880 | 245/991 | 2 | 2.8/2.9 | 0.026 | 2 | 9 | 54.3/55.5 | 0.520 | | thr_n3xqyz69nk | claude-code | 3454 | 341/4341 | 42 | 31.6/35.4 | 0.307 | 4 | 59 | 84.8/87.8 | 0.824 | | thr_nm27yahx34 | codex | 3314 | 174/5030 | 17 | 14.3/17.5 | 0.135 | 3 | 61 | 54.6/56.3 | 0.519 | | thr_ssix3mhxk5 | codex | 3088 | 316/3703 | 44 | 12.8/15 | 0.122 | 3 | 73 | 50/52.5 | 0.478 | | thr_72dhrw3s8a | codex | 3067 | 174/1659 | 5 | 35/37.3 | 0.305 | 2 | 14 | 45.6/63.9 | 0.430 | | thr_u8agadfekn | codex | 2892 | 334/2361 | 5 | 51.2/52.5 | 0.453 | 1 | 5 | 49/49.5 | 0.433 | | thr_me47xr5c3y | codex | 2466 | 316/4318 | 55 | 21.2/22.1 | 0.202 | 2 | 99 | 35.1/36.5 | 0.347 | Part of the provider-plugin migration: the Step 0 baselines PR from the design spec's "Regression confidence" section. No tracking issue. > AGENT GENERATED: by Claude Code (claude-marshmallow-ht-eap) --------- Co-authored-by: Claude <noreply@anthropic.com>
SawyerHood
added a commit
that referenced
this pull request
Aug 21, 2026
…bridge to grammar v3 with presentation (#2164) Stacked on #2121 (corpus harness + permission matrix, `bb/provider-baselines`) ← #2148 (WS2a) ← #2153 (recordings) ← #2136 (WS1a) ← #2124 (contract). WS1b-codex of the provider-plugin migration: the codex bridge speaks grammar v3 with a presentation on every item, its natives map to the core kinds, bb-injected tools carry their presentation, goals and the macOS permission profile are codex extension kinds, and the codex v2 path is deleted. **Do not merge.** Coordinator reviews, Sawyer merges the stack. ## Stack contract #2124 → WS1a #2136 → recordings #2153 (incl. #2177 and the `PARITY_INITIALIZE_ID` follow-up) → WS2a #2148 → corpus harness #2121 → **this PR** → WS1b-claude #2178. Six commits: the five codex layers below plus one `parity:` commit carrying this PR's allowlist entries and the codex `bridge→runtime.current.ndjson` lanes re-recorded against this bridge with #2153's `pnpm rerecord --provider codex --plan-with <main checkout>`. No vendored harness copies remain (they were needed only while #2153 was outside the stack). The A4 corpus check was run from a throwaway worktree with #2121's harness cherry-picked on top, since #2121 is not in the stack. ## What was wrong The codex bridge emitted v2-shaped items: no presentation (core thread-view kept codex's tool-name tables), native sub-agents as `tool` items named `spawnAgent`, `update_plan` as a turn-level event the UI discards, goals as core `thread/goal/*` events, bb-injected tools without `server`, and a macOS permission profile on a command approval failed the whole approval. Open work rode an out-of-band `thread/openWork` notification. ## What changed (one commit per layer; the last deletes) 1. **Presentation on every item.open/close** — `plugins/provider-codex/src/presentation.ts` is the one place codex tool-name knowledge lives: shell commands (wrapper stripped from the headline), file edits, the bundled `node_repl` server ("Ran JavaScript" with the call's title), other MCP servers by tool name, dynamic tools, collab verbs, web search/fetch, image views, reasoning, messages, plans, compactions, the synthesized sub-agent spawn. An invariant suite drives one item per codex native through the real translator and asserts every lifecycle delta carries one. 2. **Delegation + planSteps** — the synthesized native sub-agent is a foreground `delegation` (`childRef` = agent thread id, `label` = agentPath); a follow-up to a settled agent re-opens the same item and the row closes only when the agent owes nothing more (the exact open-work predicate); a dead app-server child settles its open delegations as failed on the wire. A collab call that names its receiver is a delegation to it; a bare `wait` stays a tool item with its collab presentation. `turn/plan/updated` becomes a settled `planSteps` snapshot per update. `RuntimeBackgroundWorkState` counts a pending delegation as open work. **Thread-view projects a `delegation` item to the existing delegation row with the child content nested** — without this the row and every child message under it vanish before the projection workstream lands (the projection suppresses orphans with a `parentToolCallId`). `planSteps` items are not projected yet (status quo: codex plans were discarded). 3. **bb-injected tools carry their presentation** (Q31) — `bb.agents.registerTool({ experimental_presentation })` (+ `docs/api_to_audit.md`); the server resolves one presentation per tool at its boundary (declaration → status labels → generic label + the plugin's branding glyph / `Toolbox`) onto `DynamicTool.presentation`; the codex bridge emits calls to injected tools as `{ server: "bb", tool }` with that presentation. ask-user-question and workflows declare theirs (AskUserQuestion and `bb_workflow_result` collapse by default). `HOST_DAEMON_PROTOCOL_VERSION` 149 → 150 (149 is WS2a's #2148, this PR's base; the history reads 147 → 148 → 149 → 150). The field is optional and `dynamicToolSchema` is not strict, so an older daemon strips it and keeps working; the bump follows the repository rule for a widened server↔daemon wire. Optional per A1: every committed recording's runtime lane predates the field and must keep replaying; the stabilization pass makes it required. 4. **Goals + macOS profile as codex extension kinds** — `provider-codex/goal` (state; `null` once cleared) and `provider-codex/macos-permission` (item) declared on the registration with plugin-owned zod schemas (`extension-kinds.ts`). The bridge emits goals as extension state. A command approval asking for macOS capabilities now reaches the user for the command; the profile rides the timeline as its own row saying bb cannot grant it (the plugin-rendered approval is WS5's). **Read-time conversion in core:** `parseStoredThreadEvent` decodes persisted `thread/goal/updated|cleared` rows into the extension state (the one path every stored-event read takes); thread-view goal extraction and the runtime's goal-clear wait read that state. The sidebar's latest-goal query becomes a latest-thread-state-by-kind query (partial index widened to `thread/extensionState/updated`, migration **0106** `thread_state_index` — regenerated with Drizzle so its snapshot chains from WS2a's 0105 `provider_settings_to_plugins`; `json_extract` kind filter). Verified against the corpus: the 721-row goal thread renders its goal unchanged. 5. **Delete the codex v2 path** — `thread.goal`/`thread.goalCleared` leave the grammar and assembler (G3 snapshot updated; `PROVIDER_BRIDGE_PROTOCOL_VERSION` stays 2 under the grammar range, as WS1a's v2 deletion did); the `thread/openWork` notification leaves the protocol, adapter and reaper (an unknown notification is ignored); the bridge drops its open-work reporting and the hard-coded AskUserQuestion presentation. Kept: the bridge's knowledge of its spawn/resume collab verbs for a receiver-less call — tool-name knowledge in the bridge is the point, not a remnant. The `thread/goal/*` domain event types remain as read-only legacy vocabulary. Naming note: the spec says `codex/goal`; the extension-kind namespace is the **plugin id**, which is `provider-codex`, and that is how the registry resolves the schema. ## Regression oracle All turbo invocations with `--concurrency 4`; perf suite ignored (known-noisy, flagged on #2121). - **Conformance**: codex scripted suite + recorded conformance over all 17 recorded cells (incl. archived-resume, auth-failure, empty-rollout, missing-rollout) green. - **Parity (A2)**, `pnpm parity --old <origin/main worktree at f6fb434> --new . --provider codex`: **16 passed, 0 failed, 1 skipped** (process-scoped `model-list`). Event and row counts equal in every cell. `recordings/parity-allowlist.json` names every intended byte-inequivalence with `#2164` and a reason — 26 entries, four classes: `presentation` on items (15 cells), `server: "bb"` on the AskUserQuestion call (user-question), the sub-agent spawn as a `delegation` item + its row (`subagent`, events `/6` `/28`, rows `/0/children/0/{toolName,output}`), and `thread/goal/cleared` → `provider-codex/goal` extension state (one event index in each of 6 goal-bearing cells). Zero unlisted diffs, zero stale entries. claude-code **13/13** and acp-cursor **10/10** replay against main with **zero diffs** (no entries). - **Corpus (A4)**, 307 threads / 93,262 rows: **zero diffs**, claude-code and codex alike; no corpus allowlist entry was needed (the only read-time change, goals, projects identically by design). - **G11**: `provider/unhandled` flat on every codex cell (0→0; auth-failure 1→1) and on the corpus. - **G1**: 209 → 209 (no provider-id literal added or removed in core; `"provider-codex/goal"` is not a provider-id literal by the ratchet's regex). - **Parity self-suite** (#2153's, in the stack): 43/43 with the codex current lanes re-recorded against this bridge; row-count pins **unchanged** (every current lane assembles to exactly the pinned counts); the claude/acp current lanes #2153 produced on WS1a's bridges hold as they are. Tests on the final stack (all forced, `--concurrency 4`, one turbo at a time): db 406 (migration chain 0104 → 0105 → 0106 incl. replay-on-existing-DB cases), agent-runtime 337, codex 193 (scripted + recorded conformance), host-daemon-contract 52, provider-parity 43; typecheck green for codex, agent-runtime, provider-parity, server, db. Earlier full sweep before the re-stacks: domain 150, thread-view 382, provider-bridge-protocol 218, plugin-sdk 127, host-daemon 552, integration 55, claude-code 262, acp 182, workflows 223, ask-user-question 36, scripted echo 1; server 1828/1829 (the one failure is the known local umask case in `internal-skill-trees`, passes in CI); typecheck green across 25 packages incl. app, mobile, cli. ## Not done - **Live QA cells via bb-dev-app** (turn/steer/stop/approve/deny/question/subagent/resume/fork/plan with screenshots) — not run; the coordinator schedules them separately. - `planSteps` rows are not projected by thread-view (no regression: codex plans were discarded before); the projection workstream owns it. - The plugin-rendered macOS approval (the profile round-trips only as a visible row today) — WS5. > AGENT GENERATED: by Claude Opus 5 --------- Co-authored-by: Claude <noreply@anthropic.com>
SawyerHood
added a commit
that referenced
this pull request
Aug 21, 2026
… presentation (#2178) Stacked on #2164 (WS1b-codex, `bb/ws1b-codex-codex-bridge-to-v3-stack-on-2136-thr_xnid5ftd87`), which sits on WS2a #2148 → the corpus harness #2121 → the recordings harness #2153 → WS1a #2136 → the contract #2124 (SDK 0.4.11 at the bottom). WS1b-claude of the provider-plugin migration: the Claude Code bridge speaks grammar v3 with a presentation on every item, Claude's tool-name knowledge moves out of core thread-view into the bridge, and the bridge's last Claude-specific result structure is deleted. **Do not merge.** Coordinator reviews, Sawyer merges the stack. Stacked on codex rather than WS1a on purpose: codex added the minimal thread-view projection of `delegation` items (row + nested child content) that Claude's `Agent`/`Task` sub-agents need. The recordings + parity harness beneath this PR is the real #2153 (in the chain, with `pnpm rerecord` and the `PARITY_INITIALIZE_ID` fix); the one harness file this PR touches is `redact.mjs` (bare-message ndjson, dash-encoded home paths). The corpus harness #2121 is in the chain too; the A4 run below is native to this tree. ## What was wrong The Claude bridge emitted v2-shaped items: no presentation (core thread-view kept Claude's tool-name tables — `Read`/`Grep`/`Glob` intents, the `Agent`/`Task` delegation row, the TodoWrite/Task* suppression list and todo reducer), `Read` as an opaque `tool` item (the top generic tool in the production corpus: 7,568 calls), `Agent` as a `tool` item named Agent, TodoWrite/TaskCreate/TaskUpdate as tool rows a core reducer had to understand, bb-injected tools as `mcp__bb-bridge__<name>` without `server`, and a structured task-tool `result` that core parsed — and which never matched in production, because the persisted result was the SDK's friendly string, so the task banner did not work for the Task tools. ## What changed (one commit per layer; the last deletes) 0. **Transcript → SDK-stream converter and fixtures** (the spec's first task). `scripts/provider-recordings/convert-claude-transcript.mjs` turns a `~/.claude/projects` session (plus its `<session>/subagents/agent-*.jsonl` sidechains, interleaved by timestamp with `parent_tool_use_id` from the subagent's `toolUseId`) into the SDK stream the bridge would have seen. A transcript has no `result`, no `system/init` and no `task_*` family, so the converter synthesizes them deterministically: a `result` per segment (a root message that stops with a non-tool-use reason, the next prompt, EOF), `task_started`/`task_updated`/`task_notification` for Agent calls from the call, its result (`async_launched` ⇒ backgrounded) and the `<task-notification>` resume, `api_retry`/model-fallback/`compact_boundary` from the system records. Human prompts only delimit turns — a live stream never echoes them (verified against every committed recording); CLI-injected user messages (isMeta context, task notifications) do stream and are kept. `convert-claude-transcripts-sample.sh` rebuilds the committed sample: **12 sessions/windows, 1,076 messages (331 sidechain records), 1.85 MB after redaction**, all from the owner's own corpus threads: plan mode + AskUserQuestion + ExitPlanMode, WebSearch, WebFetch + Read, Edit/Write, foreground and backgrounded Agents with their sidechains, TaskCreate/TaskUpdate + `model_refusal_fallback`, Workflow + Monitor + TaskStop, TaskOutput, ScheduleWakeup, SendMessage, `api_retry`, `compact_boundary`, `mcp__bb-bridge__` tools. Grep, Glob, TodoWrite, MultiEdit and NotebookEdit appear in none of the 2,559 local transcripts; those paths are covered by scripted unit tests. `transcript-fixtures.test.ts` drives each fixture through the `sdk/message` envelope into a real assembler and checks structural invariants (every tool_use opens an item its tool_result settles, sidechain items nest under the spawning call, every turn settles, nothing left open, every started item presented) plus a pinned projection per fixture in `expected.json` (item kinds, tool names, plan snapshots, `provider/unhandled`, which may only go down). 1. **Presentation on every item; the v3 kinds** — `plugins/provider-claude-code/src/presentation.ts` is the one place Claude's tool-name knowledge lives; `tool-classification.ts` maps every tool_use to its shape with that presentation. `Read` → `fileRead`; `Grep` → `search{content}`, `Glob` → `search{path}`; `Edit`/`Write`/`MultiEdit`/`NotebookEdit` → `fileChange` with per-verb labels; `Bash` → `command` (a backgrounded call is labelled as a launch); `WebSearch`/`WebFetch` as before with presentation. `Agent`/`Task` → `delegation` (`childRef` = the call id, which is how the SDK identifies the sub-agent's stream: `parent_tool_use_id`; `background: true` for `run_in_background`, settling at the launch ack on the thread-scoped family; summary = the result text without Claude's `agentId:`/`<usage>` lines; sub-agent type and model in the presentation detail). `TodoWrite` → a collapsed call row plus a settled `planSteps` snapshot from its arguments; `TaskCreate`/`TaskUpdate`/`TaskList`/`TaskGet` → a collapsed call row plus a `planSteps` snapshot of the thread's folded task list (`plan-fold.ts`, reading the SDK's envelope-level `tool_use_result`) after each successful call — channel-keyed close deltas, the latest superseding, the same shape as codex `update_plan`. `ToolSearch`, `TaskOutput`, `Monitor`, `ScheduleWakeup`, `SendMessage`, `AskUserQuestion`, `TodoRead`, `BashOutput` → `tool` with `presentation.suppress`; plan mode, Workflow, TaskStop, Skill, StructuredOutput, worktrees, ListAgents → `tool` with their own labels/glyphs/titles; an unknown tool reads `Running <tool>`/`Ran <tool>`. `mcp__<server>__<tool>` splits into `{ server, tool }`. The compaction item and every close-without-open fallback carry one too; the close re-states the open's. **Thread-view keeps the new kinds rendering** until the presentation-driven projection lands (the same minimal bridge codex added for `delegation`): `fileRead`/`search` items project to the tool row with the intents the legacy Read/Grep/Glob calls produced (tested equal to the legacy rows); a tool call whose presentation says `suppress` is hidden like the legacy name list (failures still render); the todo banner reads a `planSteps` snapshot as-is. No persisted-event projection changed: the legacy tables stay for old rows (G1 unchanged). 2. **Background tasks carry their presentation** — workflows, backgrounded shells and backgrounded sub-agents stay the core `backgroundTask` kind (genericity rule) and say how they read on open/close. Model fallback and `/clear` stay the core events they are. 3. **bb-injected tools** (Q31) — a `mcp__bb-bridge__<name>` call is `{ server: "bb", tool: <bare name> }` with the presentation the server resolved onto the `DynamicTool` definition, learned through `configureInjectedTools` at session construction; a definition without one presents generically under bb's glyph. The server's `statusLabels` enrichment skips items with a server, so nothing relies on it. 4. **Delete the Claude v2 translation path** — the task tools' structured `result` (the shape core's legacy todo reducer read) is gone; the tool row carries the text result like every tool and `planSteps` is the one structured form core sees. The bridge no longer imports the SDK's claude task-tool schemas. Kept on purpose: the bridge's knowledge of its own tool names, and the close-without-open fallback. `HOST_DAEMON_PROTOCOL_VERSION` stays **150**: nothing on the server↔daemon wire changed (translation and an item-shape change inside the `thread/delta` lane only). ## Regression oracle All turbo invocations with `--concurrency 4`; perf suite ignored (known-noisy, flagged on #2121). Old leg: an `origin/main` worktree at `f6fb434ab`. - **Conformance**: claude scripted suite + recorded conformance over all 14 recorded cells (incl. auth-failure, plan-mode, subagent, user-question) green. The claude `bridge→runtime.current.ndjson` lanes are re-recorded with this bridge (`pnpm rerecord --plan-with <main checkout> --provider claude-code`); the recordings themselves are untouched and the self-suite's row-count pins are **unchanged** (43/43). - **Parity (A2)**, `pnpm parity --old <origin/main worktree> --new . --provider claude-code`: **13 passed, 0 failed, 1 skipped** (process-scoped `model-list`). Event and row counts equal in every cell. `recordings/parity-allowlist.json` gains 27 entries naming `#2178`, three classes: - `presentation` on items — `/*/item/presentation`, events, 9 cells (approval-allow, approval-deny, compaction, plan-mode, steer, subagent, turn-tools, user-question, web-search; the other 4 cells have no items). - Read → `fileRead` — plan-mode events `/2`, `/4`, `/24`, `/25`; rows `/0/children/{0,5}/{toolName, toolArgs, output, activityIntents/0/command, activityIntents/0/name}` (the row projects from a fileRead item: no tool name or arguments, the file contents are not row data, the intent command is `Read <path>`). - Agent → `delegation` — subagent events `/6`, `/13`; rows `/0/children/0/toolName` ("delegation") and `/0/children/0/subagentType` (the delegation item has no such field; the type rides the presentation detail). Zero unlisted diffs, zero stale entries. codex **16/16** and acp-cursor **10/10** replay against main with **zero new diffs** (no entries added; nothing of theirs touched). The expected "suppressed low-value rows" and "planSteps" classes produce no parity diff: no recorded cell calls Monitor/TaskOutput/ScheduleWakeup/SendMessage or TodoWrite/Task*, and ToolSearch/AskUserQuestion were already hidden by name. - **Corpus (A4)**, 307 threads / 93,262 rows: **zero diffs**, claude-code and codex alike, no corpus allowlist entry needed. Persisted Claude events are `toolCall` items without presentation, and every thread-view change here applies only to the v3 kinds and to `presentation.suppress` — the legacy tables and reducer are untouched — so old rows project identically by construction. - **G11**: `provider/unhandled` flat on every claude cell (0→0; compaction and steer 2→2) and on the corpus; pinned per transcript fixture (23 across 12 fixtures, every one a CLI-injected string-content `user` message — compaction summaries, `<task-notification>` resumes — which visibility classifies `unknown` today; lowering that is a separate change). - **G1**: unchanged (no provider-id literal added or removed in core; the legacy Claude tool-name tables stay for persisted rows). Tests (forced): thread-view 385, claude-code 322, codex 193, provider-bridge-protocol 218, provider-parity 43; server 1832/1833 (the one failure is the known local umask case in `internal-skill-trees`, which passes in CI). Typecheck green for the claude plugin, thread-view, server and agent-runtime on the re-stacked base. ## Not done - **Live QA cells via bb-dev-app** (turn/steer/stop/approve/deny/question/subagent/resume/fork/plan with screenshots) — not run; the recorded cells, the transcript fixtures and parity were the oracle. - `planSteps` rows are not projected as timeline rows (status quo: TodoWrite/Task* rows were hidden); the banner reads them. The presentation-driven projection workstream owns the rows. - The delegation row loses the "(Explore)" sub-agent-type suffix for new Claude sub-agent rows until the projection reads the presentation detail (allowlisted; old rows unaffected). - Backgrounded `Agent` calls keep today's two-row structure (the delegation settles at the launch ack; the `local_agent` background task tracks the work). Folding the task into a single background delegation is a larger change to the runtime's open-work tracking and the background-commands card. - String-content `sdk/user` messages (task-notification resumes, compaction summaries) still surface as `provider/unhandled`; the fixtures show they are the whole G11 residue for Claude. A one-line visibility change would lower it but changes parity for the steer/compaction cells, so it is left for a deliberate follow-up. > AGENT GENERATED: by Claude Opus 5 --------- Co-authored-by: Claude <noreply@anthropic.com>
SawyerHood
added a commit
that referenced
this pull request
Aug 21, 2026
… ratchet A full-stack dry run surfaced a scope gap: the ratchet's test exclusions covered test/, tests/, and __tests__/ but not this repo's src/testing/ convention, so the parity and conformance harnesses landing in #2153 (which must name providers to map each recording to its bridge module and fake child) registered as 10 new core carve-outs. That is test tooling, the same category as __tests__, not a provider branch in core. Add "testing" to the excluded path segments, document why in the header, and add a fixture test proving a provider id under src/testing/ does not count. Regenerated baseline: 213 references across 56 files (one existing src/testing hit was already being miscounted as core). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This was referenced Aug 21, 2026
SawyerHood
added a commit
that referenced
this pull request
Aug 21, 2026
) ## What was wrong Codex labels each reconnect attempt with a structured `codexErrorInfo` (for example `{ responseStreamDisconnected: { httpStatusCode } }`, `willRetry: true`, failure text in `additionalDetails`), then reports the terminal failure for the same stream error with `codexErrorInfo: "other"` and the failure text moved to `message`. That downgrade is upstream: codex-rs `notify_stream_error` always labels retries `ResponseStreamDisconnected`, while `CodexErr::to_codex_protocol_error` maps `CodexErrorDetails::Stream` to `CodexErrorInfo::Other`. The bridge trusted the terminal value, so the final timeline row lost the `stream-disconnected` category and rendered as a generic **Provider error** (the detail text is longer than the 80-char title budget, so the disconnect cause was only visible after expanding the row). Issue: #1840. The independent report URL (https://get-bb.github.io/reports/issues/1840.html) returns 404; the issue body and #1563 carry the repro. ## What changed - `plugins/provider-codex/src/delta-translation.ts`: the translation state remembers the retry-time `codexErrorInfo` and failure text per codex `threadId\0turnId`. A terminal (`willRetry: false`) error whose `codexErrorInfo` is `other` and whose failure text equals the remembered retry text reuses the retry classification. The context is consumed by the terminal error, dropped on `turn/completed`, and never crosses turns. Unrelated terminal errors and every non-`other` terminal value keep the provider-reported classification. No provider prose is parsed. - `plugins/provider-codex/src/translator.ts`: `thread/closed` also clears the retry context for that codex thread (exported `clearCodexEventTranslationThreadState`). - No wire change between server and host daemon: the `provider/error` event shape is unchanged, only the value of `errorInfo` on this one path. No CLI or doc surface changes. Relation to #1563: that PR (same design, by @ymichael and @brsbl) targets `plugins/provider-codex/src/event-translation.ts`, which #1834 deleted when it moved Codex onto the narrow-grammar delta translator. It no longer merges (`git merge-tree` reports a content conflict in `delta-translation.ts`). This PR ports that design onto `delta-translation.ts`, with a flat `Map` keyed by thread+turn instead of nested maps. ## How you verified Tests added in `plugins/provider-codex/src/translator.test.ts` (`codex terminal retry-error classification`): - carries the retry classification into the degraded terminal error (and the context is consumed, so a repeat stays `other`) - does not relabel an unrelated terminal error after a reconnect - scopes the retry context to the turn and drops it on `turn/completed` - drops the retry context when the codex thread closes Fail-before: with `delta-translation.ts` and `translator.ts` restored from `origin/main`, `pnpm exec turbo run test --filter=bb-plugin-provider-codex --force -- --run src/translator.test.ts` fails the first test: ``` × carries the retry classification into the degraded terminal error AssertionError: expected [ { type: 'provider/error', …(7) } ] to deep equally contain ObjectContaining{…} expected "category": "stream-disconnected", "providerCode": "responseStreamDisconnected" received "category": "unknown", "providerCode": "other" ``` The three negative tests pass on `origin/main` as expected (they pin that the guard does not over-apply). Pass-after, from the committed tree (`git status --porcelain` empty): - `pnpm exec turbo run typecheck test --filter=bb-plugin-provider-codex` → `Tasks: 7 successful, 7 total` (16 test files, 176 tests passed) - `pnpm exec turbo run build` → `Tasks: 18 successful, 18 total` Manual replay of the incident's two events (reconnect with `responseStreamDisconnected`, then terminal `other` with the same `stream disconnected before completion: ...` text) through `createCodexEventTranslator` via `node --conditions=source --import tsx`: the terminal delta now carries `errorInfo: { category: "stream-disconnected", providerCode: "responseStreamDisconnected", httpStatusCode: null }`. ## Rebase Rebased onto `origin/main` after the grammar-v3 bridge stack landed (#2124, #2136, #2153, #2148, #2164). That stack rewrote `delta-translation.ts` (presentation on every item, `injectedToolsByName` on the translation state, delegation items) and `translator.ts` (`clearClosedThreadState` now returns the closes for open delegations), but it did not touch the `error` case, `toProviderErrorInfo`, or `turn/completed`, so the fix maps onto the new code unchanged: - The only textual conflict was in `CodexEventTranslationState` / `createCodexEventTranslationState`, where main added `injectedToolsByName` next to where this PR adds `retryErrorsByTurnKey`. Resolved by keeping both fields. - `clearCodexEventTranslationThreadState` is still called from `clearClosedThreadState` in `translator.ts`, before it returns the delegation closes that main added. - The tests merged cleanly into `translator.test.ts`; they run through the grammar-v3 `createDeltaAssembler` harness on main, and the `provider/error` event shape they assert is unchanged. Re-verified on the new base (`798b720ef`, one commit on top of `origin/main`): - Fail-before: with `delta-translation.ts` and `translator.ts` restored from `origin/main`, `pnpm exec turbo run test --filter=bb-plugin-provider-codex --force -- --run src/translator.test.ts -t "codex terminal retry-error classification"` → `1 failed | 3 passed`; `carries the retry classification into the degraded terminal error` fails with expected `"category": "stream-disconnected", "providerCode": "responseStreamDisconnected"`, received `"category": "unknown", "providerCode": "other"`. The bug is still present on current main. - Pass-after, from the committed tree (`git status --porcelain` empty): `pnpm exec turbo run typecheck test --filter=bb-plugin-provider-codex --force` → `Tasks: 7 successful, 7 total`, `Test Files 18 passed (18)`, `Tests 197 passed (197)`. `pnpm exec turbo run build` → `Tasks: 18 successful, 18 total`. Fixes #1840 > AGENT GENERATED: by Claude Opus 5 ## Independent verification Verified on a fresh checkout of `bb/fix-1840-codex-stream-disconnect` (282f32e, one commit on top of `origin/main`; `git merge-base --is-ancestor origin/main HEAD` true, GitHub reports MERGEABLE). Root cause checked against current upstream sources (not from the PR description): `codex-rs/core/src/session/mod.rs` `notify_stream_error` hard-codes `CodexErrorInfo::ResponseStreamDisconnected` for every retry notification, `codex-rs/core/src/responses_retry.rs` returns the raw `CodexErr` once retries are exhausted, and `codex-rs/protocol/src/error.rs` `to_codex_protocol_error` has no arm for `CodexErrorDetails::Stream` so it hits `_ => CodexErrorInfo::Other`. The app-server maps `EventMsg::StreamError` to `error` with `willRetry: true` plus `additionalDetails`, and `EventMsg::Error` to `willRetry: false` with `additional_details: None`. The PR's correlation (retry `additionalDetails` vs terminal `message`, same thread+turn, `other` only) matches that wire shape exactly. Commands: - `pnpm install --frozen-lockfile --prefer-offline` and `pnpm exec turbo run build` (18/18). - Fail-before: `git checkout origin/main -- plugins/provider-codex/src/delta-translation.ts plugins/provider-codex/src/translator.ts`, then `pnpm exec turbo run test --filter=bb-plugin-provider-codex --force -- --run src/translator.test.ts -t "codex terminal retry-error classification"`: 1 failed, 3 passed. Failing assertion: `carries the retry classification into the degraded terminal error` -> `AssertionError: expected [ { type: 'provider/error', ...(7) } ] to deep equally contain ObjectContaining{...}`, expected `"category": "stream-disconnected", "providerCode": "responseStreamDisconnected", "httpStatusCode": 502`, received `"category": "unknown", "providerCode": "other", "httpStatusCode": null`. - Pass-after: restored the PR sources (`git status --porcelain` empty), `pnpm exec turbo run typecheck test --filter=bb-plugin-provider-codex --force` -> `Tasks: 7 successful, 7 total`, `Test Files 16 passed (16)`, `Tests 176 passed (176)`. - Only `packages/thread-view/src/error-display.ts` consumes the `stream-disconnected` category (title text); no runtime recovery keys on it, so the blast radius is the timeline row title. Repro on the fixed branch: a real Codex stream outage cannot be triggered deterministically here, so I replayed upstream-shaped events through `createCodexEventTranslator` directly (`node --conditions=source --import tsx`): four `Reconnecting... n/5` retries labelled `responseStreamDisconnected` with the failure text in `additionalDetails`, then a terminal `other` with that text as `message` and `additionalDetails: null`. Fixed branch: `{"category":"stream-disconnected","providerCode":"responseStreamDisconnected","httpStatusCode":null}`. Same script with `origin/main` sources: `{"category":"unknown","providerCode":"other"}`. Extra negative replays on the fixed branch all stayed correct: unrelated terminal text after a retry stays `unknown`; a structured terminal value (`responseTooManyFailedAttempts`, 503) is never overridden by the remembered retry; a thread-scoped retry (no `turnId`) does not relabel a turn-scoped terminal; a retry on a different codex thread does not leak across threads. CI: all checks pass (Checks, Package Smoke ubuntu+macos, Tests app-1/2/3, integration, server, packages, version check). Residual risks (minor, not blocking): the correlation needs the terminal `message` to equal the last notified retry's `additionalDetails`; the terminal `CodexErr` is the attempt after the last notified retry, so if the inner reqwest text differs between attempts the row falls back to today's generic label (never to a wrong one). Retry context for a turn whose child dies without `thread/closed` or `turn/completed` lives in the per-session translator until the session is released (a few bytes). The upstream mapping gap in codex-rs remains. > AGENT GENERATED: by Claude Opus 5 ## Independent verification (post-rebase) Re-verified after the rebase onto the grammar-v3 bridge. Checked out `798b720ef` (one commit on top of `cf00cfe06`); `origin/main` had since gained #2120 (provider-literal ratchet), which merges cleanly and excludes `plugins/provider-*`, so it cannot affect this PR (`node scripts/check-provider-literal-ratchet.mjs --base origin/main` -> `ratchet OK: 148 references across 40 core files`). Fix still targets the right code path in the rewritten translator: the v3 stack left the `error` case, `toProviderErrorInfo`, and `turn/completed` unchanged; `clearCodexEventTranslationThreadState` runs inside `clearClosedThreadState` before the delegation closes it now returns; `translateEvent` still routes `error` events through `translateCodexEventToDeltas(event, eventTranslationState)` with the single per-session state. Nothing downstream reclassifies: `@bb/agent-runtime` `shouldRestartCodexThreadAfterEvent` keys only on `rate-limit`/`unauthorized` categories (a retry-time label is always `stream-disconnected`, so restart policy is unchanged) and `packages/thread-view/src/error-display.ts` is the only consumer of the category. Commands (all from the committed tree): - Fail-before on current main source: `git checkout origin/main -- plugins/provider-codex/src/delta-translation.ts plugins/provider-codex/src/translator.ts`, then `pnpm exec vitest run src/translator.test.ts -t "codex terminal retry-error classification"` in `plugins/provider-codex` -> `1 failed | 3 passed`; `carries the retry classification into the degraded terminal error` fails with expected `"category": "stream-disconnected", "providerCode": "responseStreamDisconnected", "httpStatusCode": 502`, received `"category": "unknown", "providerCode": "other", "httpStatusCode": null` (`src/translator.test.ts:1272`). - Pass-after (sources restored, `git status --porcelain` empty): same vitest command -> `4 passed`. `pnpm exec turbo run typecheck test --filter=bb-plugin-provider-codex --force` -> `Tasks: 7 successful, 7 total`, `Test Files 18 passed (18)`, `Tests 197 passed (197)`. - Dependents: `pnpm exec turbo run typecheck test --filter=@bb/provider-parity --filter=@bb/agent-runtime --filter=@bb/provider-bridge-protocol --force` -> `Tasks: 10 successful, 10 total` (parity 43 passed, agent-runtime 439 passed, bridge-protocol 217 passed). The parity suite replays every committed recording, including `recordings/codex/auth-failure`, through the current bridge with zero diffs. - `pnpm exec prettier --check` and `pnpm exec eslint` on the three touched files: clean. Repro on the fixed branch: replayed the issue's event sequence (four `Reconnecting... n/5` retries labelled `responseStreamDisconnected` with the failure text in `additionalDetails`, then terminal `other` with that text as `message`) through `createCodexEventTranslator` via `node --conditions=source --import tsx`: terminal delta is `{"category":"stream-disconnected","providerCode":"responseStreamDisconnected","httpStatusCode":null}`. A terminal-only event with no preceding retry stays `unknown`/`other` (by design). Cross-checked against upstream `codex-rs/protocol/src/error.rs` (`Stream(..)` still falls to `_ => CodexErrorInfo::Other`; terminal `message` is `self.to_string()`, the same text `notify_stream_error` puts in `additional_details`). CI: all 11 check-runs on `798b720ef` succeed (Checks, Package Smoke ubuntu+macos, Tests app-1/2/3, integration, server, packages, version check x2); 2 skipped (node-compat smoke, iOS flows). Residual risk (minor, not blocking): the real recorded `auth-failure` cell shows Codex's failure text can carry per-request `cf-ray`/`request id` values that differ between the last retry and the terminal attempt; there the exact-text guard does not fire and the terminal row stays the generic label exactly as on main (for that 401 case a `stream-disconnected` label would arguably be wrong anyway, and the runtime's 401 restart text pattern still matches). Retry context for errors without a `turnId` is only dropped on `thread/closed`. > AGENT GENERATED: by Claude Opus 5 Co-authored-by: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #2136 (WS1a,
bb/ws1a-assembler-testing-kit-stack-on-2124-thr_jkbj56vr97), itself stacked on the contract #2124. Landing order: contract → WS1a → this → WS2a → #2121 → codex → claude.Why stacked, not independent.
@bb/provider-parityand the threebridge.recorded-conformance.test.tsfiles importcreateBridgeDeltaEventCollector, which lives at@bb/agent-runtime/test/bridge-delta-assemblyonmainand at@bb/provider-bridge-protocol/testingon WS1a. No ordering of two independent PRs yields a buildablemain, so this PR follows the move (and dropsprovider-parity's@bb/agent-runtimedependency). The recordedbridge→runtimelanes are grammar v2, which WS1a's v3-only assembler refuses, so every replayable cell now also carries abridge→runtime.current.ndjsonwritten bypnpm --filter @bb/provider-parity rerecord --plan-with <main checkout>through this branch's bridges; they assemble to the unchanged pins inrow-counts.json. The 10 pi cells (in-process SDK, not replayable) keep their pins and are reported, not failed.Do not merge. Coordinator reviews, Sawyer merges the stack.
What was wrong
The provider-plugin migration will break byte-equivalence with today's translators on purpose, which removes the regression oracle the goldens provide. Calibration sessions are scripted fakes and the integration suite runs a fake adapter, so nothing checks a bridge against what the provider CLIs really emit across the behavior matrix. Design: "Regression confidence" (A2, A3) and "Corpus" in the ideal-state provider API spec. No bridge may migrate before this lands.
What changed
Bridge record mode (
BB_PROVIDER_BRIDGE_RECORD_DIR, off by default, additive)bridge-worker-entry.tstees both sides of the runtime wire for every bridge, first- or third-party, before the bridge module loads.experimental_recordProviderChildIo(child, { threadId })(codexapp-server-connection.ts, acpagent-connection.ts). Claude's pipe belongs to the Agent SDK, sosdk-session.tstakes the SDK'sspawnClaudeCodeProcessseam over whenexperimental_isProviderBridgeRecording(). Pi runs in-process and records its SDK event boundary (AgentSessionEventin, prompt/abort/compact out).<dir>/<providerId>/<threadId>/<direction>.ndjson(_processfor lines that belong to no thread); one{ts, run, seq, dir, line}per line, appended per line, never buffered. Responses land in the scope of the request they answer.runidentifies the bridge process so a thread served by several processes merges back in order.withoutBridgeRuntimeEnvand the existingBB_*allowlist both strip it from provider children.docs/provider-bridge-protocol.md("Record mode"),docs/debugging-and-qa.md,docs/configuration.md,docs/api_to_audit.md(both newexperimental_SDK members).Recordings — 52 redacted cells under
packages/provider-bridge-protocol/recordings/<provider>/<cell>/with amanifest.jsoneach (provider, cell, CLI version, date, description, lane line counts).scripts/provider-recordings/redact.mjsrewrites paths under$HOME, emails,ls -lowner columns, token shapes (bbde_,ghp_,github_pat_,sk-,sk-ant-,xox?-,bbcm_, JWTs, bearer values,Authorizationheaders), secret-shaped env keys, collapses Claude'ssystem/initandcontrol_responsecatalogs to names, and trims strings over 2,000 chars to head/tail with a marker. It is idempotent and exits 3 if any pattern survives.package-cells.mjscuts the raw per-thread recordings into cells. Raw recordings stay in~/.bb/provider-recordings/raw(gitignored).Parity harness (A2)
@bb/provider-bridge-protocol/testing/parity:replayRecordingdrives the recordedruntime→bridgelane into a bridge process andreplay-provider-child.mjsplays the provider lanes back as the bridge's child — the recording is the script. The child gates each recorded provider line on the live bridge's own writes (method/subtype for requests, id for responses), maps bridge-minted ids to recorded ones, cuts the lanes into one segment per spawned child, paces spontaneous lines behind the harness's cursor so steers and interrupts land between the same two provider lines they did live, and never hangs a divergent bridge (stall → generic answer, logged). Serves JSON-RPC (codex, ACP) and the Claude CLI control protocol. Each replay runs in its own temp workspace and (for Claude) its ownCLAUDE_CONFIG_DIR, with a seeded source transcript per recordedthread/fork, so recordings replay on any machine.compareParity(old, new, allowlist)diffs normalized events (the resurrecteddiffCalibrationStreams,#2140had removed it), normalized rows, and grammar drops. Allowlist entries are{provider|"*", cell|"*", layer, path (JSON pointer with*/**), pr, reason}; an entry that masks nothing is reported stale and fails.@bb/provider-paritywires the real delta assembler and@bb/thread-viewprojection (mirroringtimeline.ts), ownspnpm parity --old <checkout> --new . [--provider] [--cell] [--dump-dir], and loads each leg's assembler and projector from that leg's own checkout (its@bb/provider-parity, else the collector at its pre/post-WS1a home), so amain--oldleg assembles withmain's assembler.parity.self.test.ts(CI): every cell assembles to the counts pinned inrecordings/row-counts.json(events, rows;provider/unhandledand grammar drops may only go down — G11), every allowlist entry names a PR, reason, and pointer (staleness is judged bypnpm parity --old <main>, which an old==new replay cannot), and every replayable cell replays through the current bridge with zero event/row/grammar diffs and zero stalls.UPDATE_PARITY_ROW_COUNTS=1rewrites the pins deliberately.Current bridge lanes (merged from #2177, reviewed as harness owner) — a recording is never rewritten. When a bridge change alters what the bridge emits,
pnpm --filter @bb/provider-parity rerecord [--plan-with <recording-time checkout>]writesbridge→runtime.current.ndjsonbeside the recorded lane; the self-suite and recorded conformance pin/compare against it when present, andpnpm paritypaces the old leg from the recorded lane and the new leg from the current one. Also from #2177:pnpm parity --dump-dir, a 300 ms drain before each replayed runtime request and a 50 ms gap after a replayed response (the steer-ack position race seen twice in CI). Two follow-ups on top: the harness's owninitializeresponse is kept out of re-recorded lanes, and re-recorded lanes pass throughredact.mjsbefore they are written (a bridge error can quote the replay child's command line). 39 current lanes are committed (every replayable cell) — see "Why stacked" above.Conformance —
checkRecordedCellReplay+replayRecordedCellsadd the recorded-traffic scenario set (recorded/<cell>/{replays, events-schema-valid, grammar, turn-lifecycle, not-empty}); each first-party bridge getsbridge.recorded-conformance.test.tsover turn, steer, stop, approval allow/deny, question, resume, fork.No translation behavior change. No wire change;
HOST_DAEMON_PROTOCOL_VERSIONis untouched.How you verified
On the stacked head
39066e19b(WS1a base), one Turbo invocation at a time at--concurrency 4: typecheck 12/12 tasks (protocol, parity, agent-runtime, host-daemon, plugin-sdk, codex, claude-code, acp); tests — protocol 218, codex 173, claude-code 262, acp 182 (eachbridge.recorded-conformance.test.tsgreen), parity self-suite 43/43 withrow-counts.jsonuntouched;pnpm parity --old . --new .39 passed / 0 failed / 13 skipped;pnpm rerecord --plan-with /home/sawyer/projects/bb39 OK / 0 STALL; redaction sweep over recordings + current lanes0 survivors, idempotent.Earlier, on the
main-based head0923c1c59(after merging #2177): CI green (Checks, Package Smoke ×2, Tests app-1/2/3, integration, packages, server); 21/21 Turbo tasks, 1,848 tests (protocol 101, parity 43, agent-runtime 418, host-daemon 552, plugin-sdk 117, codex 173, claude-code 262, acp 182); the steer/stop-interrupt cells stable across 3 repeated runs. New tests in this PR: recorder routing/tee/oversize, worker-entry record tee,withoutBridgeRuntimeEnvstrips the knob, daemon forwards it to bridge processes but not the shell env, the 43-test parity self-suite, three recorded-conformance suites.An earlier CI run on
mainfailed inpackagesbecause ACP and Claude replays depended on the recording machine (cwd,~/.claudetranscripts forforkSession,PATH); reproduced locally with the directory moved away and a bareHOME, fixed as described above.Matrix (
codex-cli 0.149.0,Claude Code 2.1.238/ Agent SDK 0.3.197,cursor-agent 2026.08.11, pi@earendil-works/pi-coding-agent 0.84.0, recorded 2026-08-21 throughscripts/bb-dev-app+ thebbCLI):fullmode; nothing asksAskUserQuestion)subAgentActivity)Agent)thread stopcursor-agentadvertises nosession/fork(0 events, pinned)/plancommand mention)claudeCodePermissionMode: plan, plan approval interaction)_processscope)webSearchitem)WebSearch+WebFetch)thread/archive; bridge unarchives and retries)failed to read session metadata) + bonusmissing-rolloutCODEX_HOMECLAUDE_CONFIG_DIRRedaction sweep (
node scripts/provider-recordings/redact.mjs packages/provider-bridge-protocol/recordings /tmp/redact-stack):diff -rqbetween the committed fixtures (recordings + current lanes) and that output is empty (idempotent). The only remaining occurrence of the username is the public skill namebb-global-skills:sawyer-voicein Claude's advertised skill catalog.Parity self-run (
pnpm parity --old . --new .):39 passed, 0 failed, 13 skipped (52 cells)— the 13 skips are the 3 process-scopedmodel-listcells (no thread events) and the 10 pi cells (in-process SDK, no child to replay; still pinned and assembled by the self-suite). Every PASS line reports identical old/new counts, e.g.codex/steer: old 87 events/3 rows, new 87 events/3 rows, unhandled 0→0, grammar drops 0→0. The cross-checkout path was exercised too:--old /home/sawyer/projects/bb(onmain, noprovider-paritypackage) loaded that checkout's@bb/agent-runtimecollector and bridge and matched on all 16 codex thread cells.Sizes: recordings 6.9 MB total including the 39 current lanes (6.2 MB without) — codex 1.6 MB, claude-code 1.2 MB, acp-cursor 932 KB, pi 2.5 MB recorded; largest cell
pi/turn-tools828 KB.Fixes none (step-0 PR of the provider-plugin migration).