Third-party canary: echo-provider exercises every provider-plugin capability

[SawyerHood]

What was wrong

The provider-plugin migration's core principle is zero first-party privilege: a marketplace plugin must be able to do everything codex and claude do, using only the public SDK. The smoke test proved the first-party plugins; the designated third-party canary, examples/plugins/echo-provider, did not. It emitted a bare agentMessage stream, declared no strings, reasoning levels, service tiers, env passthrough, provider options, or extension kinds, shipped no bb tool, and its README (and the bb-plugin-authoring skill) pointed at the private @bb/provider-bridge-protocol/conformance. So presentation, extension kinds, delegation, fileRead, search, planSteps, extension.state, and the settings → providerOptions → bridge round trip had never been exercised by non-first-party code.

This was written stacked on #2179. The whole chain (#2124 → #2136 → #2153 → #2148 → #2121 → #2164 → #2178 → #2179) squash-merged into main while it was in progress, so the branch is rebased onto main and the PR targets main.

What changed

examples/plugins/echo-provider (imports only @get-bb/plugin-sdk{,/provider-bridge,/host}, zod, node built-ins):

• server.ts: bb.settings.define({ shout }); bb.agents.registerTool({ name: "echo_stamp", experimental_presentation }); bb.providers.register with experimental_strings (signInHint, expiredHint, installUrl, brandPrefix, planModeCopy, iconTint), experimental_reasoningLevels, experimental_serviceTiers, full capabilities, composerActions: ["plan"], experimental_models.fallback, experimental_env.passthrough, experimental_deriveProviderOptions (reads the shout setting, the model, and the prompt mode), and experimental_extensionKinds — receipt (item) and mood (state), zod schemas.
• src/vocabulary.ts: the kinds, schemas, tool presentation, and every row presentation, shared by the declaration and the bridge.
• src/provider-bridge.ts: one scripted grammar v3 turn per prompt — command (+outputDelta), fileRead, search, delegation with a keyed child turn linked through parentRef, planSteps, a suppressed tool, the bb tool called over item/tool/call (providerNativeIds: true, turnId: null) and stamped server: "bb" with the definition's presentation, the extension item, extension.state, the echoed message (reporting the derived providerOptions and the passed-through env var), usage + contextWindow, turn.boundary. Presentation on every item.open and item.close. Handshake: grammarVersions [3,3], sessionRestore: true, steerMode: "queue", approvalEnforcedBy: "runtime". model/list returns one model. Directives: /noop (zero-work turn), malformed-receipt (schema-violating payload).
• Tests (all via @get-bb/plugin-sdk/provider-bridge/testing): provider-bridge.conformance.test.ts (12 scenarios incl. turn/settles-without-activity), provider-bridge.stream.test.ts (the whole turn through the kit's real assembler, the bridge's tool call answered), public-sdk-only.test.ts (fails on any @bb/* import anywhere in the package, and on imports outside the allowlist).
• README.md: the rule, a capability table, how each is proven.

apps/server: test/providers/echo-provider-canary.test.ts installs the plugin from its path, builds the real thread.start command, runs it on the real @bb/agent-runtime (now a server devDependency), posts every event through /internal/session/events, answers the tool call through /internal/session/tool-call, and reads the rows back. turbo.json: the plugin's sources join @bb/server#test inputs.

Docs: the bb-plugin-authoring skill's Conformance paragraph and docs/provider-bridge-protocol.md point at the public kit.

No wire change; HOST_DAEMON_PROTOCOL_VERSION untouched.

How you verified

Capability checklist → proof

Capability	Proof
experimental_strings incl. iconTint, experimental_reasoningLevels, experimental_serviceTiers, capabilities, composerActions, experimental_models.fallback	Live: GET /api/v1/system/providers for echo-agent projects all of them (strings, reasoningLevels with labels, serviceTiers, extensionKinds)
experimental_env.passthrough	Server test: command.bridgeLaunch.envPassthrough = ["BB_ECHO_PROVIDER_GREETING"]; the runtime forwards it and the bridge echoes BB_ECHO_PROVIDER_GREETING=hello from the daemon
bb.settings.define → experimental_deriveProviderOptions → providerOptions → bridge	Server test: updateSettings({shout:true}) → command.options.providerOptions = {shout:true, model:"echo-1", promptMode:null} → message echo: HELLO CANARY … providerOptions (server): shout=true. Live: bb plugin config echo-provider set shout true → echo: HELLO LIVE CANARY … shout=true
experimental_extensionKinds item kind, validated at ingest	Server test + live row: item/completed extension kind echo-provider/receipt payload {prompt, itemCount: 7, shouted: true}
Malformed extension payload → provider/unhandled	Server test: malformed-receipt turn persists two provider/unhandled rows, rawType: extension/item:echo-provider/receipt, no extension rows; the same turn's state row still persists
experimental_extensionKinds state kind	Server test + live row: thread/extensionState/updated kind echo-provider/mood
bb.agents.registerTool({experimental_presentation}) → dynamicTools[].presentation → bridge stamps server:"bb" + presentation	Server test: command.dynamicTools carries the presentation; persisted toolCall row server:"bb", result:"stamped: hello canary" (the plugin's own execute, via the real tool-call route), presentation = the declaration's. Live row identical
Presentation on every item.open/item.close	Stream test (≥18 item events) and server test (≥20 rows) assert every item carries label+icon; live: 0 item rows without presentation
command + outputDelta	Stream test, server test, live row
fileRead, search	Stream test, server test, live rows
delegation with child turn via parentRef	Stream/server tests: child turn/started.parentToolCallId === delegation.id, child message parentToolCallId, different turn id. Live: seq 17 turn/started parentToolCallId=da94261219-i4 = delegation id
planSteps	Stream test, server test, live row
Generic tool with presentation.suppress	Stream/server tests; live: hidden from the timeline as intended
item.textDelta/item.textClose + usage + contextWindow	Stream test (thread/tokenUsage/updated, thread/contextWindowUsage/updated), live rows
Handshake grammarVersions [3,3], steerMode, sessionRestore	Conformance handshake/initialize; live bridge accepted by the daemon
Zero-work turn settles	Conformance turn/settles-without-activity (/noop)
Only-public-SDK rule	public-sdk-only.test.ts (12 tests)

Commands

• pnpm exec turbo run typecheck test --filter=bb-plugin-echo-provider — 4 files, 17 tests pass. Conformance report: 12/12 pass (rpc/unknown-method, rpc/invalid-params, rpc/non-json-ignored, rpc/response-not-request, handshake/initialize, session/start-identity, turn/lifecycle, events/schema-valid, item/opens-before-delta, stop/release-not-interrupted, session/resume-id-uniqueness, turn/settles-without-activity).
• pnpm exec turbo run typecheck --filter=@bb/server — pass.
• pnpm exec turbo run test --filter=@bb/server — 1893 pass, 1 fail: internal-skill-trees mode 0644 vs 0664, the known local-umask failure unrelated to this change.
• Before this PR the new tests fail: the old bridge emitted none of the v3 shapes (stream test), had no tool/extension rows (server test), and the conformance fixture had no zero-work prompt.

Live run (layer C)

Dev instance bb-worktrees-env_7afqatsj6r-bb-52b06540aff5 on this branch. bb plugin install ./examples/plugins/echo-provider --yes (path install; status running, capability agent-tool echo_stamp, hasSettings). bb plugin config echo-provider set shout true. In the web app: New thread → picker → "E" (Echo) → model "1" → prompt hello live canary.

• Thread thr_nzk5zngiii, provider echo-agent, status idle.
• 37 persisted rows: item/started+item/completed for commandExecution, fileRead, search, delegation, planSteps, 2×toolCall, extension, 2×agentMessage; 2×turn/started (child parentToolCallId=da94261219-i4), 2×turn/completed, thread/extensionState/updated, thread/tokenUsage/updated, thread/contextWindowUsage/updated.
• Extension row: kind echo-provider/receipt, payload {prompt:"hello live canary", itemCount:7, shouted:true}, presentation Wrote receipt / PackageReceive / tint.
• echo_stamp row: server:"bb", result:"stamped: hello live canary", presentation Stamped receipt / Check / tint.
• Message: echo: HELLO LIVE CANARY · providerOptions (server): shout=true model=echo-1 promptMode=none · BB_ECHO_PROVIDER_GREETING=<unset> (the dev daemon's env has no such variable) · echo_stamp: stamped: hello live canary.
• SELECT count(*) … item_kind IS NOT NULL AND json_extract(data,'$.item.presentation') IS NULL → 0.
• Daemon log: Downloading host artifact … digest 9be6015218e303b…, then Using cached host artifact.
• Screenshots: $BB_THREAD_STORAGE/echo-live-timeline.png (collapsed) and $BB_THREAD_STORAGE/echo-live-timeline-expanded.png (expanded: Ran echo "hello live canary", Read README.md, Searched for …, Ran subagent: Echo "hello live canary" one more time, Ran tool bb:echo_stamp { text: … } with the result body, then the echoed message). Thread storage /home/sawyer/.bb/thread-storage/thr_u89sy372gv/.
• Dev app stopped afterwards (scripts/bb-dev-app stop).
• Re-run after the rebase onto main (810cf2e + this branch): the path install survived the restart (bb plugin list → running), bb plugin reload echo-provider, then bb thread spawn --project proj_personal --provider echo-agent --prompt "hello rebased canary" and bb thread wait --status idle (the CLI/agent path). Thread thr_28wgdzvcgc: the same 37 rows, extension payload {prompt:"hello rebased canary", itemCount:7, shouted:true}, mood state {mood:"cheerful", turnsEchoed:1}, message echo: HELLO REBASED CANARY … shout=true … echo_stamp: stamped: hello rebased canary, 0 item rows without presentation.

API gaps and findings

Gaps — things a third party needs that the public SDK does not expose:

1. The parity / recorded-replay harness is private and first-party-keyed. replayRecording, replayRecordedCells, compareParity, assembleRecordedEvents, checkRecordedCellReplay, RECORDED_CONFORMANCE_CELLS, and the recording readers (readBridgeRecording, listRecordedCells) are not re-exported from @get-bb/plugin-sdk/provider-bridge/testing; and even with the private import, resolveReplayProfile throws UnreplayableProviderError for any provider not in FIRST_PARTY_BRIDGE_MODULES. The recording side (experimental_isProviderBridgeRecording, experimental_recordProviderChildIo) is public, so a third party can record but cannot replay or compare. This PR therefore ships no parity self-run for the echo bridge; the stream test (real assembler, asserted events) is the closest public equivalent.
2. ThreadEvent is not exported by name from the testing kit. The kit's assembler returns canonical events but the type is only reachable as ReturnType<BridgeDeltaEventCollector["assembleMessage"]>[number] (see provider-bridge.stream.test.ts). A one-line export type { ThreadEvent } would fix it.
3. PluginProviderDeclaration.icon glyphs are dropped at projection. icon: "Zap" validates, but ProviderInfo only carries logoUrl (null for a glyph), so the picker shows the initial "E". A third party without an SVG asset cannot get an icon into the picker.

Observations — declared and projected correctly, but not consumed by the web client yet (WS2a/WS3 consumers, not API gaps):

4. Extension items do not render in the web timeline. The echo-provider/receipt row is persisted with full presentation but nothing in packages/thread-view projects item.type === "extension", so the row is invisible (the expanded screenshot shows every other row). The declarative base for extension kinds is WS3.
5. Presentation is not used for row labels. The echo_stamp row renders as the generic Ran tool bb:echo_stamp { text: … } instead of Stamped receipt with the Check glyph; same for the core rows, which use core verbs. Persistence is correct; the renderer is WS3.
6. reasoningLevels labels, serviceTiers labels, and strings.iconTint are projected on ProviderInfo but unused by the app. The picker showed Low / Medium / High rather than Whisper / Speak / Shout, a generic "Fast mode" switch, and no tint. useThreadCreationOptions.ts reads only brandPrefix, planModeCopy, installUrl; the usage settings read signInHint/expiredHint.
7. Anonymous text streams cannot carry presentation. item.textDelta/item.textClose have no presentation field, so a stream that opens itself (the claude/pi/acp channel-key pattern) synthesizes an agentMessage without one. The echo bridge opens its messages with item.open first and the close echoes the presentation; first-party bridges that stream anonymously will need the same change before presentation becomes required.

AGENT GENERATED: by Claude Opus 5

[SawyerHood]

Coordinator review — APPROVE as the canary; findings routed. (Draft is fine to keep until the G-A/G-C fixes have PRs; this PR's job was to find them.)

The third-party path is proven. I verified: zero @bb/* imports in non-test code with public-sdk-only.test.ts enforcing it; the bridge genuinely emits every capability (fileRead, search, delegation with a parentRef child turn, planSteps, suppressed tool, bb tool stamped server:"bb", extension item + extension.state, settings→providerOptions round trip); three proof layers including a live local-plugin install (37 rows, 0 items without presentation) and the malformed-payload → provider/unhandled rejection. This is the first time non-first-party code has exercised the v3 surface, and it works.

The three API gaps are real and are the most valuable output here:

1. Parity/replay harness is private + first-party-keyed — a third party can record but not replay/compare. That's half our promised regression oracle unreachable from outside. → its own PR: move replay/compare into @get-bb/plugin-sdk/provider-bridge/testing, key by providerId.
2. ThreadEvent not exported from the public kit → one-line fix, bundle with Fix env-daemon CI flakes #1.
3. Declaration icon glyph dropped at projection (picker shows "E") — first-party plugins masked this because they ship SVG assets: a privilege asymmetry, exactly what a canary is for. → WS2a follow-up: project {glyph} into ProviderInfo.

The four observations (extension items don't render; labels ignore presentation; reasoningLevels/serviceTiers/iconTint unused by the app) are known and assigned to WS3 — the data is persisted correctly; the client hasn't been taught to read it yet. The "anonymous text streams can't carry presentation" rule I'll document for bridge authors.

Rebasing onto main after the stack merged was the right call. Sawyer merges.

AGENT GENERATED: by Claude Opus 4.8