Export sinks on their doors; the cold export lane deleted - #488
Open
MattJackson wants to merge 26 commits into
Open
MattJackson wants to merge 26 commits into
MattJackson wants to merge 26 commits into
Conversation
… (THE DESIGN §11.11 R4, §11.12; ARCHITECT ruling Q-C1-FILE 2026-10-03)
The request-log file sink is door-only, and it asked the host for a disk lane that did not exist, so
every line was dropped with BUSBAR-7074, a break from 1.5.5. The host now serves the lane.
- contract: abi::host::service gains op DISK_APPEND = 22 (SERVICES 22 -> 23, may pend), with DiskAppendIn
{head, dest_key, bytes, result} -> DiskWritten {size, rotated, faults, written}. A FAILED answer names
its step in value (DISK_OPEN_FAILED / DISK_APPEND_FAILED) and still reports the rotation.
check_disk_append_in (host side) and check_disk_append (caller side), one RED per arm.
HostSlots.disk_append is appended (184 -> 192); the abi-layout golden gains the appended rows and the
new table size. SDK: Connector::disk_append, on the op's handle count. services: HostServices::disk_append,
DiskDest, DiskReport, DISK_KEEP (9), DISK_ROTATE_KEY ("rotate_mb", the host's rotation key).
- loader: an opener grants an instance the destinations its manifest declares (ExportRows::open,
Plugin::grant_destinations); open and refresh bind each one to the path the settings give it. The
disk_append slot maps the key through that binding (NO_DESTINATION otherwise) and writes the lane's report.
- kernel: host_disk::DiskLane: one thread, appends in submission order, bounded (LANE_BOUND). It writes
as 1.5.5 did (open for append, create if absent, write the bytes whole), after a rename rotation when one is due.
- root: LateServices and CredentialServices forward the call.
A delivery's host-service results and connections are stored under (ticket, n), with n counting from 0.
The flusher used ONE ticket for every batch, so the second batch picked up the first batch's stored
results: otlp got the first batch's establish ("the far end took no bytes": one export, then nothing),
and the file sink got the first batch's disk.append (only the first batch was written). Minting a
ticket per batch and recycling it afterwards forgets those results (worker recycle_now: completions,
services, conns).
…of 265), the design cited in words in the contract, every HostServices double answers disk_append
…'s export-doors axis (busbar_export_file::door) and its dropped-in cdylib is built with its dropped-in feature; plugin-loader's cold both-ways row for it goes, with the tests whose subject was its cold entry (the #11 file-sink suite, load_and_exercise_export_plugin) and export_tests drive an in-test cold boundary (owner NO-TEST-PLUGINS Q7, the CONF-SUITE-DEL treatment of ec6feb9); the webhook sink's cold suite stays
…lt distribution links the http framer; request-log-webhook moves to its door - Cargo.toml: export-file a0dfd7d (busbar-export-file#12: deliver appends through disk.append), export-otlp fc87b52 (#12), export-prometheus 4689954 (#12), export-webhook 631a1dc (#11). Each is a repin to 4abda5b using scripts/fleet/repin.sh, the same way busbar-auth-oidc#23 did it. otlp and webhook Needs now state keep_mode and the deny list. The coordinator replaced the fleet-pins merge with this. - busbar Cargo.toml: `transport-http-door` is linked on the connector-door axis, under scheme `http` (busbar_transport_http::door). This is ARCHITECT ruling Q-C1-OTLP: packaging only, so kernel, contract and loader source name no instance. `module: otlp` boots, and its spans reach the collector over the Connector's http framer. The `http` wire row is unchanged. - request-log-webhook: the dev head is door-only (export_door!, no cold entry), so it moves to the export-doors axis (entry busbar_export_webhook). Its need is framed by the `http` claim. - plugin-loader: the cold webhook both-ways row and its suite (export_conformance_tests) are removed, the CONF-SUITE-DEL treatment already applied to the file sink. Real plugins are the proofs, so the webhook joins export_axis_tests on both doors (probe, the 1.5.5 settings refusal, open). - busbar tests: export_request_log_file_1_5_5 (new). Two requests through the linked file sink each append the line the 1.5.5 golden cell export|request-log-file|jsonl recorded, byte for byte under the cell's masks. export_plugin_dropped_in_serves now states the door's Statement in the dropped-in manifest (common::plugins::state).
…ce); the linked-entry scan names an external entry by its crate (c1's 42bb141, the export-doors rows list no bare door); the catalogue reads both export axes and the page test runs on either (linked_axis_export_doors declared); the dropped-in export proof renders through the door; a stray doc block's empty line (clippy)
…nto the export kind's door registries (ARCHITECT ruling EXP2-PROOF 2026-10-03: the spec's working proof of the plugin model; deleting it was a gate weakening) Every test the file held at predev-all/b9 is ported to the real sinks on the memory ABI, each LINKED by its logic crate's door and DROPPED IN as its -plugin crate's cdylib (signed first-party, Statement rendering in its manifest), no test plugin (owner NO-TEST-PLUGINS): - envelope observations identical (#85), the observations the sink produced, one row + same fold, first-party series granted through either door and to nobody else, settings validation, host-written destination: the request-log FILE sink through disk.append (the host's acts served by crate::host's destination writes, rotating by rename); - declared diagnostics, host-carried outbound request, the need's egress class: the request-log WEBHOOK sink over the host's connection table (a recording table that honours the DeclaredConns contract); - the RED arm the_pre_envelope_path_loses_a_dropped_in_plugins_counters: the real file-sink cdylib staged by the loader, every op forwarded with its reply's envelope kept plugin-local; the host ingests nothing while the compiled-in build reports both rotations; - the pre-op RED arms (validate, check): a door whose table predates the op is refused at the load. The three cold-lane claims with no door form (the cold start op, 600 in flight past the blocking pool, the manifest egress-policy grant) are stated in the module doc with the reason. ExportRows::with_envelope: an opened instance's #85 envelope goes where the opener says (NoSink when unset, as before). plugin-loader dev-deps: busbar-export-file and busbar-export-file-plugin (dropped-in) at the workspace pin.
….5 behaviour frozen) WEBHOOK-TARGET: busbar-export-webhook repinned to f85762d (PR GetBusbar/busbar-export-webhook#11): its need's target comes from `settings.url`, so the host's connector no longer refuses it and every delivery is POSTed again. MAX-INFLIGHT: an export instance runs up to its in-flight bound of deliveries at once, each on a ticket of its own and awaited (no thread held while it pends), every batch started on one worker in queue order; while fewer lines are queued than may start, each travels alone. The bound is the instance's `max_inflight_deliveries` (contract export_calls::INFLIGHT_KEY; DEFAULT_INFLIGHT 64 when unset, 1.5.5's) within the plugin's declared max_inflight; the webhook declares no ceiling of its own. ENVELOPE (#85, owner-locked): the opener's default envelope sink is the host's observability (observe::EnvelopeObserver: a door's metrics named by their Statement family, its declared diagnostics by their id, into the installed PluginObserver); the plugin log sink also forwards a declared diagnostic downstream. Never NoSink for an opened export instance. EGRESS-GRANT (§5): the scan refuses a plugin that is not first-party whose stated Statement declares a need in a first-party egress class (provider, operator-infrastructure, loopback-allowed): skipped as RejectKind::EgressGrant, named in the grant's words; the export opener names a scan refusal. Tests: export_conformance_tests gains the 600-past-512 bound (door form), the third-party egress refusal (the OTLP collector), the envelope-to-observability arm and the span through both doors; traces_stream_both_doors: a third-party dropped collector is refused at load (--validate names it, a config naming it does not boot), the linked collector still receives the spans.
…erver through a bounded, non-blocking intake (ARCHITECT ruling ENVELOPE-ALL 2026-10-03; #85 owner-locked) - dispatch::Plugin::bind stands observe::EnvelopeObserver before whatever sink the binder gave, for every kind (plane, auth, store, hook, secret, transport, export): a metric of a family the Statement declares (named by family, kind and label keys) and a declared diagnostic reach the installed PluginObserver under the plugin's Statement name and kind word; log records stay with the plugin log; the binder's sink still receives every entry. ExportRows no longer builds its own observer (the bind does); the plugin log sink no longer forwards declared diagnostics. - observe: the intake is a bounded queue (INTAKE_BOUND 4096) drained by one observer thread; a plugin call hands its back-channel over with try_send and never waits. A full intake drops and counts; the count reaches PluginObserver::dropped, which the root wires to the kernel's new busbar_plugin_observations_dropped_total (in the root's HOST_SERIES; docs/observability.md). Cold envelopes (observe::fold) take the same intake. - Tests (observe_tests): one per kind through the real door where the crate's graph has one (auth admin-tokens, store-memory, secret-env, transport-tcp, export-file; plane and hook through their kinds' existing fixtures), a metric named by its family for every kind, and the full intake dropping, counting and never stalling the caller. RED-checked: without the bind's observer every per-kind arm fails.
…tract; moved mechanism paths
…d SDK glue, the host's cold acts and egress carrier, the root's cold exports axis); the export axis opens door rows only
…heus door renders them); the cold-linked check stays for the auth residue
…nt (kind::EXPORT => &[...]) reads as one keyed by its word
…ow is gone); the transport-error class is the auth residue's
…with its door (dropped-in)
…RABLE-BYPASS row's ledgered rename exemption, and its error-text intern a reviewed hold-escapes site (bounded by MAX_WORDS)
…i, door-only CARRIER), abi-freeze cold 2164 -> 1138; the dropped-in staging file is one per call; the 1.5.5 file-sink line test names its store
MattJackson
enabled auto-merge
October 5, 2026 16:17
promote into
|
| crate | test | step | first panic |
|---|---|---|---|
| `` | nextest xtask::cli::selftest_runs_every_registered_gates_red_proof |
test:workspace | |
transport_dropped_in_serves |
a_dropped_in_transport_registers_through_the_one_fold_and_serves |
test:dropped-in-tcp-transport | crates/busbar/tests/transport_dropped_in_serves.rs:168:13: busbar exited (ExitStatus(unix_wait_status(512))) before serving; log: |
-p busbar --test transport_dropped_in_serves |
test target failed |
test:dropped-in-tcp-transport |
DENY rows (14)
| gate | row | detail |
|---|---|---|
| construction | one-pick-site |
3 production call site(s) of 'pick_among(' (ceiling 2): crates/busbar-kernel-egress/src/walk.rs:298; crates/busbar-llm/src/engine/exhaustion/fallback.rs:120; crates/busbar-llm/src/engine/pipeline.rs:8 |
| reachability | reachability:unit-path:decision |
NO LIVE UNIT PATH: roster plane 'decision' has no kernel-loop runner registered for its capability key, no 'Units' impl its linked entry exports and no served door, so nothing 'fn main()' reaches driv |
| reachability | reachability:root-reach:decision |
NO UNIT PATH REACHED: no module a chain from crates/busbar/src/main.rs arrives at routes plane 'decision''s capability key onto a kernel-loop runner, its linked entry exports no 'Units' impl, and no s |
| kind-isolation | kind-isolation:deps |
4 finding(s), 78 shipped edge instance(s) over 28 class(es), 78 declaration(s); 62 '[[dep]]' row(s), 0 question(s): unlisted-dep-edge cleanliness -> plugin-tooling busbar-core-admin -> busbar-plugin-l |
| kind-isolation | kind-isolation:test-deps |
7 finding(s), 37 test edge instance(s) over 20 class(es), 37 declaration(s); 22 '[[dep]]' row(s), 0 question(s): unlisted-dep-edge cleanliness -> legacy busbar-core-admin -> busbar-a2a is a test edge |
| kind-isolation-ship | kind-isolation:deps |
16 finding(s) over 78 shipped edge(s): ship-edge cleanliness -> plugin-tooling busbar-core-admin -> busbar-plugin-loader is 'not-allowed': the architecture grants no cleanliness -> plugin-tooling edge |
| kind-isolation-ship | kind-isolation:test-deps |
15 finding(s) over 37 test edge(s): ship-edge cleanliness -> legacy busbar-core-admin -> busbar-a2a is 'not-allowed': the architecture grants no cleanliness -> legacy edge, and the ship criterion is t |
| kind-isolation-ship | kind-isolation:faces |
4 finding(s) over 31 crate(s): foreign-entry crates/busbar busbar is kind 'root' and implements 'Store' 2 time(s) in shipped source — the entry face of kind 'store'. A trait implementation is a claim |
| kind-isolation-ship | kind-isolation:testkit |
2 finding(s) over 12 crate(s): battery-ignored crates/busbar-plane-decisions busbar-plane-decisions carries a tests/conformance.rs whose every entry is '#[ignore]'d (or which has none). 'cargo test' |
| kind-isolation-ship | kind-isolation:legacy-drain |
5 finding(s): transitional-live qa/kind-isolation.toml 'busbar-a2a -> busbar-core-admin' (legacy drain: the retiring A2A engine's tests drive the admin surface that drained into the cleanliness tier) |
| kind-isolation-ship | kind-isolation:control-path |
77 finding(s) over 3 control surface(s): upstream crates/busbar-core-admin/src/admin_codec/meta.rs:84 busbar-core-admin names 'egress' — a control surface has no upstream to reach, so the vocabulary o |
| ship-ready | ship-ready:ship-twin |
'kind-isolation-ship' is not green: kind-isolation:deps (a dependency the architecture does not grant is still in the graph); kind-isolation:test-deps (a dependency the architecture does not grant is |
| instance-noun-neutrality | instance-noun-neutrality:voice |
tracked known-debt census — 8: crates/busbar-plane-llm/src/codec/gemini/handler.rs×7 [cross-plugin] | crates/busbar-plane-llm/src/codec/gemini/tests/handler_tests.rs×2 [cross-plugin] | crates/busbar |
| structure-lint | structure-lint:plane-dup:unledgered |
24 finding(s): PLANE-DUPLICATE (module): 'config.rs' — a2a:crates/busbar-a2a/src/a2a/config.rs decisions:crates/busbar-plane-decisions/src/config.rs mcp:crates/busbar-mcp/src/mcp/config.rs voice:crate |
Judged against base fb60d3435: 0 new red, 0 worse, 9 standing (excused).
tests passed: 24228, failed: 1. Run: https://github.com/GetBusbar/busbar/actions/runs/37339690309 . Artifact verdict-1ed5bd61c890d568b75562dc1ea481682dad9524 (failures.json, junit.xml, raw.log; 90 days).
github-merge-queue
Bot
removed this pull request from the merge queue due to a conflict with the base branch
Oct 5, 2026
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Q2 step 2 (ARCHITECT ruling: port the EXP2/disk.append fixes, re-pin, delete the cold export lane). Stacks on #482 (disk.append).
Re-pin: busbar-export-file at its merged dev PR head (GetBusbar/busbar-export-file#13: appends through disk.append, runs busbar's published conformance suite) and busbar-export-webhook at GetBusbar/busbar-export-webhook#12 (door behind dropped-in, the Need's keep mode and deny list, target from settings.url, published suite). Both on the export-doors axis; both cdylibs built with dropped-in.
Ported from lane-w3a/w3b (no salvage verdict overlaps p6-salvage #471): export-file door-only; EXP2 fixes (MAX-INFLIGHT: an instance's in-flight bound from its settings, DEFAULT_INFLIGHT 64; ENVELOPE: an opened instance's #85 envelope reaches the host's observability; EGRESS-GRANT: a third-party plugin declaring a first-party egress class is skipped at scan, named in the grant's words); the #11 export_conformance_tests restored on the door registries, its disk host on the kernel lane's rules.
Deleted (old-lane line count: abi/cold 2164 -> 1138): DynExport and ColdExport (plugin-loader export.rs, scrape.rs), the host's cold acts and egress carrier (host.rs trimmed to the rotate-by-rename the plugin log files use), the root's cold exports axis, LinkedExport, HostEgressCarrier and its collector policy (root otlp.rs; the door sinks' egress is judged by the connector's class), the contract's cold export wire (abi/cold/export.rs) and SDK glue (export_export_plugin!, ExportHandler, HostStep). MetricFamily/MetricSample move to abi::export (the prometheus door renders them). The JSON-lane loader tests run on the auth fixture. Ledgers: one-memory-abi and door-only rows struck; abi-freeze cold lowered.
Proof (Latchkey, debug): fmt clean; workspace clippy -D warnings; contract, loader (585), kernel lib (2641) green; busbar export tests (dropped-in serves, 1.5.5 file line byte for byte, traces both doors, scrape first-party, scrape boot window) green; --no-default-features tests green and every single-plane set builds; xtask gate --all at row level identical to f27bd58 (no new reds). ledger_identity needs the oracle checkout (network) on the runner, as before.