Skip to content

Experiment: Capsule Emit inference receipts as a plugin #1332

Description

@ndizazzo

Status: Draft prototype specification

Depends on #1331. Related umbrella: #1233. Prototype source and prior sidecar evidence: action-state-group/capsule-emit-mesh.

Summary

Build a native Rust, installable Capsule Emit plugin that produces signed, hash-chained Agent Action Capsules for OpenAI inference exchanges served by mesh-llm.

This is a prototype experiment to validate:

  • the plugin lifecycle contract in spec: expose OpenAI request lifecycle to plugins #1331;
  • a concrete inference-receipt field mapping;
  • local signing, verification, chaining, retrieval, and optional transparency anchoring;
  • non-streaming, streaming, error, denial, and cancellation behavior;
  • composition between mesh-llm inference receipts and agent/tool action capsules.

The plugin must consume host-provided request/route/response lifecycle data over plugin IPC. It must not listen as an HTTP reverse proxy, require clients to use a second port, replace an inference backend, or require a custom mesh-llm binary.

Experiment hypothesis

An installable plugin can create a useful, portable, independently verifiable record that says:

This collector observed mesh-llm accept this request commitment, select this claimed model/runtime route, and return this response commitment.

The prototype does not establish that the claimed model performed the computation correctly. A self-signed capsule can cryptographically seal a false runtime claim. Model artifact verification, native compute receipts, hardware-bound signing, TEE/GPU evidence, statistical fingerprints, and distributed stage receipts remain separate assurance layers.

The experiment succeeds if it proves the integration and evidence-envelope mechanics without overstating the resulting trust level.

Why a plugin

The existing Capsule Emit mesh experiment proved the capsule format against a real mesh-llm node and Goose session, but it had to be a sidecar because plugins could not receive mesh-llm's own OpenAI request envelope.

That architecture had unavoidable limitations:

  • clients had to target a second proxy port;
  • the collector sat outside the mesh-llm trust boundary;
  • streaming was buffered and re-synthesized;
  • client-compatibility transformations occurred in the sidecar;
  • it could not distinguish gateway, route-selection, and serving-host facts;
  • it could not prove whether a refusal happened before dispatch.

#1331 is intended to remove those limitations generically. This issue validates that work with a real consumer.

Scope

In scope

  • /v1/chat/completions;
  • streaming and non-streaming responses;
  • local-model serving for the primary acceptance run;
  • successful, invalid, denied, failed, timed-out, and cancelled exchanges;
  • host-provided route/model/runtime descriptors when available;
  • request and response commitments;
  • plugin-managed Ed25519 receipt signing with an owner-signed node/plugin delegation when the host owner identity is available;
  • inclusion and independent verification of the existing node ownership and release-build attestation evidence;
  • COSE_Sign1 signed statements;
  • performance instrumentation, adaptive evidence heuristics, and a comparative benchmark report;
  • per-plugin-instance hash chaining;
  • local JSONL ledger and signed-statement storage;
  • plugin HTTP resources for status, retrieval, and verification;
  • optional, explicitly enabled SCITT/transparency anchoring;
  • one live Goose integration run.

Stretch scope

  • /v1/responses and /v1/completions;
  • a request routed over a two-node private mesh;
  • linking an inference capsule to a Goose/MCP action capsule;
  • ingesting a native Skippy generation receipt as a typed evidence reference.

Out of scope

  • claiming proof of correct inference;
  • TEE, SEV-SNP, TDX, NVIDIA GPU, or TPM attestation;
  • claiming that a software owner-delegated plugin key is non-exportable or proves runtime integrity;
  • distributed per-stage activation receipts;
  • statistical model fingerprinting;
  • reputation or trust scoring;
  • arbitrary request/response mutation;
  • making Capsule Emit a required mesh-llm dependency;
  • replacing the sidecar repository before the experiment is evaluated.

Architecture

OpenAI client
    |
    v
mesh-llm OpenAI ingress
    |
    +-- request_received ------> capsule plugin
    |                              allocate receipt_id
    |                              commit request
    |                              return continue + response header reference
    |
    +-- backend_selected ------> capsule plugin
    |                              bind effective request and route claims
    |
    +-- normal inference path
    |
    +-- response byte tap -----> capsule plugin side stream
    |
    +-- exchange_finished -----> capsule plugin
                                   finalize response commitment
                                   build capsule
                                   sign COSE statement
                                   append chain/ledger
                                   optionally enqueue anchor

The plugin is out-of-process, like other mesh-llm plugins, but it is not a network sidecar. It uses the host control connection and negotiated side streams from #1331 and never occupies the OpenAI request path as an HTTP server.

The prototype must be a native Rust plugin using mesh-llm's public plugin author API. Python reference implementations are interoperability oracles used by development/conformance tests only; Python is not a runtime dependency, subprocess, embedded interpreter, packaging requirement, or fallback execution path.

Native Rust implementation requirements

The experiment should look and behave like an ordinary mesh-llm plugin:

  • use the mesh-llm-plugin crate, generated protocol types, plugin runtime, manifest builders/DSL, config schema, lifecycle hooks, and side-stream helpers;
  • use the plugin! authoring pattern where it fits instead of implementing a private control protocol;
  • declare the spec: expose OpenAI request lifecycle to plugins #1331 lifecycle capability through the generic manifest contract; do not special-case Capsule Emit in the host;
  • expose status, receipt lookup, issuer material, and verification through normal host-projected plugin HTTP/resource surfaces;
  • use Tokio-compatible async I/O and keep blocking filesystem, model hashing, and cryptographic preparation off control-plane tasks;
  • stream request/response commitments incrementally where possible rather than retaining whole SSE transcripts;
  • use typed Rust structures for the mesh evidence extension and capsule model, with explicit serde representations and schema/version constants;
  • keep AAC, canonicalization, hashing, COSE/CBOR, signing, ledger, and anchoring dependencies inside the plugin rather than adding them to mesh-llm core;
  • use reviewed Rust cryptography/serialization crates and pin behavior with golden vectors; exact dependency selection is an implementation decision subject to normal dependency review;
  • never invoke Python, shell out to the reference CLI, or use FFI in the serving/runtime path;
  • never read the owner keystore, node key, keychain, passphrase environment, or any host private-key file directly;
  • consume identity evidence and signing delegation only through the permissioned host contract in spec: expose OpenAI request lifecycle to plugins #1331;
  • package native per-platform plugin artifacts through the existing plugin installation/release conventions;
  • preserve the normal plugin health, startup, shutdown, configuration, upgrade, and state-directory behavior.

The Rust implementation must produce artifacts compatible with the upstream Python tooling. Development and CI may install capsule-emit, agent-action-capsule, and scitt-cose to verify interoperability, but a released plugin must run and verify local records without them.

Plugin declaration and grants

The plugin declares the mesh.openai.exchange.v1 hook with:

  • chat-completions request_received;
  • backend_selected;
  • exchange_finished;
  • buffered request body access;
  • streamed response body access;
  • access to only X-Capsule-Client-Nonce from request headers;
  • response metadata permission for X-Mesh-Receipt-Id;
  • read access to the public node identity/attestation bundle;
  • permission to request/renew the mesh.inference.capsule.sign.v1 signing delegation;
  • observe-only admission behavior.

The host configuration explicitly grants these scopes.

The prototype returns continue/abstain for every request. Admission decisions are tested with the exemplar policy plugin from #1331, not smuggled into Capsule Emit behavior.

The experiment runs best-effort/fail-open by default. If the plugin is unhealthy, inference continues and host status must show evidence unavailable. A separate test may exercise required/fail-closed mode, but that must be an operator decision.

Receipt lifecycle

Provisional receipt identity

For the observe-only Capsule Emit plugin, avoid adding a synchronous plugin round trip solely to allocate an identifier. Prefer the host-generated exchange_id as the provisional receipt_id and return it as response metadata:

X-Mesh-Receipt-Id: <opaque-id>

The plugin binds its pending state and final capsule to that identifier. If #1331 instead requires plugin-generated response metadata, benchmark that synchronous path independently and retain it only if it meets the pre-dispatch overhead budget below.

The header is a correlation reference, not the final capsule ID and not proof.

The plugin exposes a host-projected HTTP resource such as:

GET /plugins/capsule-emit/http/receipts/{receipt_id}

States:

  • pending;
  • complete;
  • failed;
  • incomplete;
  • not_found.

A complete result includes the final capsule_id, signed-statement reference, local verification result, anchor state, and evidence-strength labels. Prompt and completion bodies are never returned.

Request commitment

The plugin receives:

  • exact decoded request bytes;
  • host-computed SHA-256 of those bytes;
  • parsed OpenAI request;
  • endpoint, exchange ID, observation point, and sanitized nonce header.

It computes and records:

  1. an exact wire_request_sha256 over the bytes the host accepted;
  2. effect.request_digest using the Agent Action Capsule digest contract;
  3. the canonicalization/transformation identifier used for that digest;
  4. the client nonce and its source.

If the client sent X-Capsule-Client-Nonce, record client_supplied. Otherwise generate a fallback nonce and record plugin_generated_fallback. The fallback must never be described as client anti-replay evidence.

OpenAI requests commonly contain floating-point generation parameters. The implementation must publish test vectors showing exactly how JSON numbers enter the Agent Action Capsule digest. The previous experiment's decimal-string conversion may be reused only if the transformation is versioned and explicitly recorded. The exact wire digest remains available to remove ambiguity.

Raw prompts, messages, inline media, and tool arguments are discarded after the required commitments are computed. They are not written to the ledger or logs.

Route and model commitment

At backend_selected, the plugin binds available facts into the pending receipt:

  • requested model;
  • effective model;
  • local/remote target and observation point;
  • provider/backend kind;
  • model package or artifact digest;
  • tokenizer/template digest if available;
  • runtime/release artifact digest if available;
  • request attempt number;
  • transformations between original and effective requests;
  • source and verification state of every identity field.

Rules:

  • a display name, URL, or canonical model reference is not a model artifact digest;
  • unavailable artifact evidence remains unavailable;
  • a configured manifest fallback is labelled configured_claim, not host_verified;
  • release/build attestation is build provenance, not runtime integrity;
  • a remote route claim is not evidence from the remote compute boundary.

For the primary local-GGUF test, hash the actual resolved GGUF bytes or consume an equivalent host-provided verified artifact digest. For a Skippy package, prefer its canonical manifest/source/artifact digests.

Response commitment

For non-streaming responses, record:

  • exact client-facing response bytes and host-computed SHA-256;
  • semantic JSON response digest;
  • HTTP status and terminal category.

For streaming responses:

  • mesh-llm continues to stream original bytes directly to the client;
  • the host mirrors the final encoded SSE transcript to the plugin;
  • the plugin or host incrementally commits the ordered transcript;
  • the plugin may also build a bounded semantic reassembly for effect.response_digest;
  • the capsule records which representation each digest covers;
  • truncation, dropped chunks, cancellation, or observer overflow makes evidence incomplete.

Unlike the sidecar experiment, the plugin must not buffer and re-synthesize SSE, normalize tool calls, add tool-call IDs, remove mixed content, or otherwise alter the response.

A separate delivered_response_digest is required if the semantic response digest does not cover the exact emitted bytes.

Terminal semantics

Map host terminal states precisely:

Host state Capsule interpretation
completed confirmed/executed
policy denied before backend dispatch denied/no dispatch
invalid request rejected before dispatch
backend returned an error failed/errored
transport failure after dispatch failed/errored with transport context
client cancellation cancelled, including whether bytes were emitted
incomplete observer transcript evidence incomplete, never confirmed as complete
plugin absent/unhealthy no capsule; host reports evidence unavailable

A sidecar cannot know that a refusal occurred before dispatch; the plugin lifecycle can. The prototype must use denied only when the host explicitly confirms that no backend dispatch occurred.

Capsule field mapping

Use the existing proof-of-concept mapping as the starting point:

Receipt fact Agent Action Capsule mapping
request commitment effect.request_digest plus x-mesh-poc-v1.wire_request_sha256
response commitment effect.response_digest plus x-mesh-poc-v1.delivered_response_sha256
client nonce/source model_attestation.compute_attestation.x-mesh-poc-v1
model package digest/source model_attestation.compute_attestation.x-mesh-poc-v1
runtime/build digest/source model_attestation.compute_attestation.runtime and x-mesh-poc-v1
generation parameters x-mesh-poc-v1 as exact decimal strings
route/observation point x-mesh-poc-v1
later fingerprint/TEE evidence typed evidence references, empty in this prototype
node-local history chain.parent_capsule_id with relation confirms
whole statement signature COSE_Sign1 EdDSA

Fields not registered by the Agent Action Capsule specification remain under the explicit x-mesh-poc-v1 namespace. Do not present extension fields as standardized.

Publish the full sample schema and golden capsules as prototype artifacts.

Signing, node identity, and key management

The prototype should use mesh-llm's existing identity hierarchy rather than creating an unrelated trust root whenever an owner identity is available.

Existing identities remain separate

  • owner identity: the stable Ed25519 key created by mesh-llm auth init;
  • node identity: the QUIC/iroh endpoint key;
  • SignedNodeOwnership: the existing short-lived owner-signed binding from owner to node endpoint;
  • release build attestation: release-signer evidence for the packaged mesh-llm executable;
  • receipt key: a plugin-owned Ed25519 key used only for Capsule COSE statements.

The owner and node private keys remain host-owned. The plugin never reads or receives them.

Preferred owner-delegated flow

  1. On first start, the plugin generates an Ed25519 receipt key in its plugin state directory.
  2. It requests ReadIdentityBundle from spec: expose OpenAI request lifecycle to plugins #1331 and independently verifies all supplied public evidence.
  3. It sends only the receipt public key to DelegatePluginSigningKey with scope mesh.inference.capsule.sign.v1.
  4. The host authenticates plugin identity from the live connection and returns a fixed-shape, owner-signed PluginSigningDelegation bound to:
    • owner ID/public key;
    • current node endpoint ID;
    • current SignedNodeOwnership certificate ID;
    • plugin ID/version/artifact digest when available;
    • receipt public key;
    • signing scope;
    • issue/expiry time and delegation ID.
  5. The plugin verifies the delegation before becoming ready.
  6. Per-request COSE_Sign1 statements are signed by the receipt key, not the owner key.
  7. The proof bundle includes the receipt public key, PluginSigningDelegation, SignedNodeOwnership, and release build attestation/reference.
  8. Delegation renewal happens before expiry on a background lifecycle path, never per request.

This produces:

trusted owner key
    | signs
    +-- SignedNodeOwnership --------> node endpoint identity
    |
    +-- PluginSigningDelegation ----> Capsule receipt public key
                                           |
                                           | signs
                                           v
                                      COSE inference capsule

An offline verifier confirms that the same owner authorized both the node and the scoped receipt key, then applies expiry/revocation/trust policy.

Why not sign every capsule with the owner key

The owner key may represent several nodes and is a high-value, long-lived identity. Using it for every request would:

  • increase online exposure and signing volume;
  • blur which node produced a receipt;
  • couple inference latency to keystore/keychain availability;
  • turn the host into a high-frequency signing oracle;
  • complicate future hardware-backed receipt keys.

The owner key signs only short-lived delegation claims at startup/renewal.

Receipt key storage and rotation

  • private key mode 0600 where supported;
  • atomic generation/persistence and fsync policy consistent with plugin state handling;
  • secret bytes zeroized where practical;
  • no private key in source, packages, ledgers, metrics, examples, status, or logs;
  • stable issuer ID derived from the receipt public key;
  • explicit key-generation counter/version in local state;
  • rotate on operator request, suspected compromise, incompatible key format, or configured policy;
  • delegation is invalidated and reissued when receipt key, plugin artifact/version, node key, owner key, or node ownership certificate changes;
  • renewal retains the receipt key when safe so ledger continuity does not require key rotation;
  • key rotation creates an explicit chain-transition record signed by the old key when available and the new delegated key;
  • expired/missing/revoked delegation cannot be represented as owner-delegated.

A software receipt key remains exportable. Owner delegation establishes authorization and attribution, not that signing occurred only on that machine. A future host-managed TPM/TEE/non-exportable key should fit the same delegation/evidence shape.

Identity modes

Configuration supports:

  • self_attested: independent plugin key, accurately labelled;
  • owner_delegated: prefer owner delegation, fall back to self-attested with an explicit downgraded state;
  • owner_delegated_required: plugin is not ready for evidence if a valid delegation is unavailable;
  • hardware_delegated: reserved for future attested/non-exportable keys.

No fallback silently preserves the stronger assurance label.

Release attestation composition

The identity bundle may include mesh-llm's verified release build attestation. The capsule records its digest, verification status, signer, build ID, commit, target, and artifact digest as a distinct evidence reference.

Release attestation remains certified build provenance. It does not prove that the running process is unmodified, that the plugin artifact was measured, or that the claimed model executed the request.

Assurance labels

At minimum:

  • collector;
  • api_boundary_observation;
  • runtime_claimed;
  • identity_mode = self_attested | owner_delegated | hardware_delegated;
  • owner_binding_status;
  • node_ownership_status;
  • release_attestation_status;
  • delegation_id/expiry inside the proof bundle, never telemetry.

Ledger and chaining

Store:

  • capsules.jsonl;
  • signed-statements/{capsule_id}.cose;
  • receipt_id to capsule_id/status index;
  • anchor queue/status when enabled.

Requirements:

  • append serialization is single-writer and crash-safe enough not to create an invalid chain silently;
  • restart recovers the previous chain head;
  • a partially written line is detected and quarantined or repaired explicitly;
  • signed statements are written atomically;
  • a chain fork is reported, not silently selected;
  • retention is configurable;
  • raw prompts and outputs are never persisted;
  • verification can run without network access.

Hash chaining is per plugin issuer/instance. It is not evidence that the node could not maintain a second private chain. Transparency or independent witnessing is a separate assurance.

Anchoring

Anchoring is disabled by default.

When explicitly enabled:

  • submit only the intended digest/statement reference, never prompt or output bodies;
  • use a durable retry queue;
  • distinguish pending, submitted, anchored, rejected, and failed;
  • never report anchored merely because submission was requested;
  • expose the receipt and verification result;
  • make the anchor URL configurable;
  • apply bounded retries and backoff;
  • provide a way to disable or purge unsubmitted test entries.

The acceptance run should use offline verification. A live public-log write requires a deliberate operator action after reviewing the exact capsule.

Plugin resources and configuration

Suggested projected HTTP/resources:

  • GET status;
  • GET receipts/{receipt_id};
  • GET capsules/{capsule_id};
  • GET issuer;
  • POST verify for a local capsule/signed statement, if safely bounded;
  • GET anchor queue/status.

Suggested configuration:

  • ledger directory;
  • retention;
  • issuer/node label;
  • identity mode: self_attested, owner_delegated, or owner_delegated_required;
  • receipt key path or generated-key policy;
  • receipt key rotation policy;
  • delegation renewal window within host-capped validity;
  • anchoring enabled and service URL;
  • request/response semantic digest mode;
  • maximum semantic reassembly bytes;
  • evidence mode: full, digest_only, sampled, or disabled;
  • sampling rate for explicitly selected sampled mode;
  • observer queue capacity and high-water mark;
  • best-effort load-shedding thresholds;
  • ledger flush/group-commit policy;
  • whether configured model/runtime manifest fallbacks are allowed;
  • logging level.

Sensitive settings remain owner-only. The web UI is optional and not required for prototype success.

Privacy requirements

  • Prompt, completion, tool arguments, inline media, and authorization secrets are not persisted.
  • No raw body appears in normal logs, error messages, health details, or plugin status.
  • Digest-only records must warn that low-entropy values may be vulnerable to dictionary attacks.
  • The plugin keeps body material only for the minimum time needed to compute commitments.
  • Test fixtures use synthetic prompts and outputs.
  • Any permalink/bundle flow must be reviewed to ensure it contains only intended digest-level evidence.

Performance hypothesis, heuristics, and decision gate

Performance is part of the experiment's validity. Functional capsule generation is not sufficient if the observer materially worsens time-to-first-token, token cadence, throughput, memory use, or request reliability.

Hot-path hypothesis

The expected low-overhead shape is:

  • the host allocates exchange_id/receipt_id without waiting for the observe-only plugin;
  • owner/node identity evidence and receipt-key delegation are loaded/verified at startup and renewed outside request handling;
  • immutable model, package, runtime, delegation, and public-key digests are computed once and cached, never rehashed per request;
  • request bytes already buffered by OpenAI ingress are hashed once without an additional full copy;
  • response bytes are committed incrementally from the final emitter tap;
  • semantic response reconstruction is bounded and performed off the token-emission task;
  • capsule serialization, COSE signing, ledger append, and anchoring occur after the final client byte;
  • anchoring is always asynchronous and never on the inference or receipt-finalization critical path;
  • filesystem work uses a bounded worker and configurable group commit rather than fsync per token or chunk;
  • queues are bounded and expose pressure instead of growing with stream duration.

The benchmark must verify these assumptions rather than treating them as true by design.

Evidence modes

The prototype supports explicit, observable modes:

  • full: wire commitments, bounded semantic commitments, signing, and durable ledger for every request;
  • digest_only: incremental wire commitments and a signed record without full semantic reconstruction;
  • sampled: full evidence for a configured random sample; sampling must not use prompt text or prompt hashes;
  • disabled: no request body delivery or evidence work.

Mode is operator configuration. Adaptive load shedding is permitted only in best-effort mode and must be recorded per receipt as a downgrade or evidence-unavailable outcome.

In required/fail-closed mode, the plugin must never silently change full to digest_only or sampled. It either produces the required evidence or rejects before dispatch according to #1331.

Adaptive heuristics to evaluate

Implement and compare at least these heuristics:

  1. Cache immutable artifact/runtime digests and invalidate only on a verified model/runtime change.
  2. Use host-generated receipt IDs so observe-only mode has no identifier-allocation IPC round trip.
  3. Hash streaming response bytes incrementally and keep only a bounded semantic reconstruction window.
  4. Switch a best-effort request from full to digest_only when its semantic body limit is exceeded.
  5. Mark evidence incomplete when the observer queue crosses its hard limit; never claim a complete transcript after dropped bytes.
  6. Shed best-effort observation when plugin queue latency or pending receipt count exceeds configured high-water marks.
  7. Batch ledger flushes while preserving crash-detection and chain ordering.
  8. Keep transparency submission on a separate durable queue.
  9. Permit explicit sampling for low-value traffic while retaining full mode for clients that request evidence.
  10. Compare out-of-process plugin IPC/copy cost against a minimal in-host digest tap. If IPC dominates, reconsider which portion belongs in core versus the plugin.

Every heuristic decision is represented by a bounded mode/outcome/reason. No heuristic may be inferred later from a missing capsule.

Benchmark configurations

Measure the same build/model/settings in these configurations:

A. no plugin installed;
B. plugin installed with lifecycle/body grants disabled;
C. plugin active in digest_only mode with ledger persistence disabled;
D. plugin active in full mode with local signing and ledger persistence;
E. full mode with anchoring enabled but network submission isolated behind its queue;
F. the previous reverse-proxy sidecar, as an informative comparison rather than the target architecture.

This separates generic hook overhead, IPC/body mirroring, hashing, canonicalization/signing, persistence, and anchoring effects.

Workloads

Use both a deterministic fast mock backend, which makes fixed overhead visible, and a real local GGUF, which shows product impact.

At minimum:

  • short non-streaming request/response;
  • short prompt with long streaming output;
  • long-context request;
  • tool-calling request with nested schemas;
  • large multimodal-shaped request fixture without private data;
  • concurrency 1, 4, and 16 or the largest stable level on the test host;
  • successful, backend-error, and client-cancelled streams.

Use fixed prompts, seeds where supported, output limits, model artifact, runtime build, and machine state. Record warm-up policy, repetitions, sample counts, machine profile, and all plugin settings.

Measurements

Report absolute and relative deltas, distributions, and run-to-run variance—not only averages.

End-to-end:

  • admission-to-backend-dispatch delay;
  • time to first response byte/token;
  • inter-chunk/inter-token latency;
  • total response latency;
  • output tokens per second;
  • requests per second at concurrency;
  • error, cancellation, timeout, and incomplete-evidence rates.

Plugin/runtime:

  • lifecycle IPC round-trip or delivery duration by phase;
  • request hashing/canonicalization duration;
  • final-emitter tap lag;
  • observer queue depth/high-water mark;
  • dropped events and dropped bytes;
  • semantic reconstruction duration and peak bytes;
  • capsule build/canonicalization duration;
  • COSE signing and verification duration;
  • ledger append, flush, and recovery duration;
  • final-client-byte to receipt-complete delay;
  • pending receipt count;
  • anchor queue depth and attempt outcome;
  • plugin CPU time, RSS/high-water memory, bytes copied/allocated where measurable, and disk bytes/fsync rate.

Use p50, p95, and p99 for latency distributions, median and dispersion across repeated trials, and confidence intervals or tolerance-aware comparison where the sample permits.

Provisional performance budget

Agree the final budget before implementation measurements are reviewed. The prototype starts with these provisional gates:

  • deterministic mock: p95 admission-to-dispatch overhead no more than 5 ms in full mode and no more than 2 ms in digest_only mode;
  • real local model: p95 time-to-first-token and total-latency regression no more than 2%;
  • streaming: median token throughput regression no more than 2%, with no systematic p95 inter-token regression beyond baseline variance;
  • no observer drops, false-complete receipts, or added request failures within the declared supported concurrency;
  • memory remains bounded by configured queues/reassembly limits and does not grow with stream duration;
  • p95 receipt completion occurs within 250 ms of the final client byte with anchoring excluded.

A result outside budget is not waived because model execution is slow. Report both absolute and relative effects, and repeat borderline results.

Decision rule

  • Pass: full mode meets the agreed budget across the primary workloads.
  • Conditional: digest_only meets budget but full does not. Keep the plugin experiment, make full evidence explicit/high-assurance opt-in, and investigate canonicalization/persistence offloading.
  • Redesign: hook/IPC/body mirroring dominates. Move only the minimal byte commitment and correlation tap into host core, leaving capsule construction, signing, storage, and anchoring in the plugin.
  • Stop: even a minimal digest tap causes unacceptable regression or cannot preserve streaming correctness. Do not graduate the plugin architecture.

The findings report must identify the dominant cost center and recommend pass, conditional, redesign, or stop.

Metrics and telemetry privacy

Detailed benchmark artifacts remain local and contain measurements/configuration only—never request or response bodies.

If runtime metrics are exported through mesh-llm OTLP, export remains explicitly operator-enabled and metrics-only. There is no hard-coded collector. Proposed logical instruments include:

  • mesh_llm_capsule_receipt_total;
  • mesh_llm_capsule_hook_duration_ms;
  • mesh_llm_capsule_request_commit_duration_ms;
  • mesh_llm_capsule_response_tap_lag_ms;
  • mesh_llm_capsule_build_duration_ms;
  • mesh_llm_capsule_sign_duration_ms;
  • mesh_llm_capsule_ledger_append_duration_ms;
  • mesh_llm_capsule_receipt_completion_delay_ms;
  • mesh_llm_capsule_observer_queue_depth;
  • mesh_llm_capsule_observer_drop_total;
  • mesh_llm_capsule_anchor_queue_depth;
  • mesh_llm_capsule_anchor_attempt_total;
  • mesh_llm_capsule_identity_delegation_total;
  • mesh_llm_capsule_identity_delegation_duration_ms.

Allowed attributes are bounded enums only: phase, evidence_mode, identity_mode, delegation_outcome, terminal_outcome, completeness, stream_mode, pressure_outcome, downgrade_reason, and anchor_outcome.

Do not export prompts, completions, tool arguments, nonces, request/capsule/receipt IDs, raw or hashed prompt data, model paths, ledger paths, endpoint URLs, hostnames, or raw node/device identifiers. Metrics collection and export are bounded and non-blocking.

Any implementation that adds these metrics to mesh-llm's OTLP path must update TELEMETRY_ATTRIBUTE_ALLOWLIST, docs/plugins/telemetry.md, and focused privacy tests as required by the telemetry review contract.

Deliverables

  • A native Rust, external, installable mesh-llm plugin package; no mesh-llm fork, Python runtime, or HTTP reverse proxy.
  • Rust implementations of the required AAC data model, canonicalization contract, COSE_Sign1 producer/verifier, chaining, ledger, and optional anchor client.
  • Cross-language conformance fixtures proving compatibility with the upstream Python reference tools.
  • Reproducible development launch and package/install instructions.
  • Manifest and configuration schema using spec: expose OpenAI request lifecycle to plugins #1331.
  • Request/route/terminal lifecycle implementation.
  • Capsule creation and x-mesh-poc-v1 field mapping.
  • Ed25519 receipt-key generation, storage, renewal, rotation, and COSE_Sign1 signing.
  • Read/verify integration for SignedNodeOwnership and release build attestation.
  • Owner-signed PluginSigningDelegation request, verification, renewal, and proof-bundle inclusion.
  • Self-attested, owner-delegated, and owner-delegated-required policy modes with honest downgrade behavior.
  • Durable local ledger, chain recovery, key-transition handling, and receipt lookup.
  • Offline verification command and golden conformance artifacts.
  • Optional anchoring with accurate durable status.
  • Threat-model/assurance documentation using the required collector/self-attested labels.
  • Reproducible benchmark harness and machine-readable raw results.
  • Performance report comparing configurations A–F, including percentiles, variance, resource use, and a pass/conditional/redesign/stop recommendation.
  • Metrics inventory and telemetry privacy review covering every exported metric and attribute.
  • Comparison document showing what improved relative to the sidecar.

Test matrix

Contract tests

  • Request byte digest matches an independent SHA-256 implementation.
  • Response byte/SSE transcript digest matches an independent implementation.
  • JSON canonicalization vectors cover integer, decimal, exponent, Unicode, key order, nested tools, multimodal parts, and null/omitted fields.
  • COSE signature verification rejects a mutated capsule.
  • Capsule ID verification rejects mutated effect/model fields.
  • Restart preserves a valid chain head.
  • Partial ledger writes do not create a silently accepted chain.
  • No secret headers or raw bodies are persisted.
  • Upstream Python verification accepts Rust-produced capsules and COSE statements.
  • The Rust verifier accepts committed Python-produced golden capsules and statements.
  • Rust and Python implementations produce identical canonical bytes and digest results for the agreed conformance vectors.
  • The packaged plugin starts and completes the local verification path on every supported target without Python installed.
  • The plugin never opens or reads the owner keystore, node-key file, keychain, or passphrase material.
  • An owner-delegated capsule verifies through receipt key -> PluginSigningDelegation -> owner key and through SignedNodeOwnership -> the same owner/node.
  • Missing owner identity produces self-attested downgrade or not-ready according to configured identity mode.
  • Expired, revoked, mismatched-node, mismatched-plugin, wrong-scope, and bad-signature delegations are rejected.
  • Delegation renewal occurs outside request handling and does not change the receipt key unnecessarily.
  • Receipt-key rotation produces an explicit, verifiable chain transition.
  • Release attestation is verified and presented separately from node ownership/runtime claims.

Host integration tests

  • Non-streaming success emits one verifiable capsule.
  • Streaming success remains live passthrough and emits one verifiable capsule.
  • Backend error emits an errored capsule with the actual error commitment.
  • Pre-dispatch denial emits denied only when the host confirms no dispatch.
  • Client cancellation records cancellation and transcript completeness.
  • Plugin timeout in best-effort mode does not stop inference and reports evidence unavailable.
  • Plugin timeout in operator-selected required mode fails before dispatch.
  • Receipt ID header resolves pending then complete.
  • Plugin restart during an exchange does not produce a false complete capsule.

Performance acceptance

  • Benchmark no-plugin, disabled-grant, digest_only, full, queued-anchor, and sidecar configurations.
  • Run the deterministic mock workload and a real local-GGUF workload.
  • Cover streaming/non-streaming, short/long bodies, tool schemas, cancellation, errors, and declared concurrency.
  • Report p50/p95/p99 latency, token throughput, request throughput, resource use, queue pressure, and receipt completion delay.
  • Verify immutable artifact/runtime digests are cached rather than recomputed per request.
  • Verify owner delegation issuance/renewal, signing, ledger persistence, and anchoring do not run on token-emission tasks.
  • Verify memory stays within configured queue/reassembly bounds for long streams.
  • Verify every load-shed/downgrade event is explicit and no incomplete transcript becomes a complete capsule.
  • Meet the agreed performance budget or classify the result as conditional, redesign, or stop.
  • Confirm exported metrics pass prompt/completion/path/ID/URL privacy exclusion tests.

Live acceptance

  • Run against a real mesh-llm local GGUF, not only a mock.
  • Run a real Goose tool-call session through the normal mesh-llm port.
  • Produce independently verifiable inference and tool-action capsule streams.
  • Confirm the client receives the original mesh-llm streaming bytes without sidecar normalization.
  • Record the actual model artifact digest and its evidence source.
  • Keep the acceptance artifacts offline unless anchoring is explicitly approved.

Stretch acceptance

  • Route one request over a two-node private mesh and label gateway versus serving-host observations.
  • Link one inference receipt to one Goose/MCP action capsule by shared digest/reference.
  • Attach a Skippy native generation receipt as a typed evidence reference.

Success criteria

The experiment is successful when a stock mesh-llm build can install the native Rust plugin on a machine without Python, serve a real streaming request on its normal OpenAI port, return a receipt reference, and later provide a locally verifiable signed capsule whose request/response commitments match the host-emitted bytes. When the node has an owner identity, the proof must also verify through the scoped PluginSigningDelegation and existing SignedNodeOwnership without exposing either host private key.

Functional success is necessary but insufficient. Graduation also requires the full mode to meet the agreed performance budget, or an explicit conditional/redesign decision supported by the benchmark evidence.

Success does not depend on proving correct model execution or productionizing the public transparency service.

Evaluation and follow-up decision

At the end of the experiment, publish a short findings report answering:

  1. Is spec: expose OpenAI request lifecycle to plugins #1331 sufficient for audit and policy plugins without adding a generic unsafe middleware API?
  2. What are the measured p50/p95/p99 latency, token-throughput, CPU, memory, copying, queue, persistence, and receipt-completion costs of each evidence mode?
  3. Which heuristic or subsystem dominates overhead, and does the result support pass, conditional, redesign, or stop?
  4. Can streaming evidence be complete without meaningful latency or memory regression?
  5. Is the Agent Action Capsule profile expressive enough without abusing unregistered fields?
  6. Which mesh-specific extension fields should be proposed upstream?
  7. Does the Rust producer/verifier interoperate completely with the Python reference implementation, and which reusable pieces should be proposed upstream?
  8. What is required to link this API-boundary capsule to an authoritative native generation receipt?
  9. Does the owner -> node ownership plus owner -> scoped receipt-key delegation provide the right software identity chain, rotation, and revocation semantics?
  10. What is required to upgrade the delegated receipt key to host-managed or hardware-attested/non-exportable storage?
  11. Should the prototype graduate, remain experimental, or be retired?

Alternatives considered

Continue the sidecar

Useful as a compatibility demo and independent observer, but it cannot satisfy the in-host policy and provenance goals and changes streaming behavior.

Implement Capsule Emit directly in core

Would couple mesh-llm to one evidence format before the plugin contract and threat model are validated.

Use only native generation receipts

More authoritative about tokens, but does not commit to the original OpenAI request or exact client-visible response. The eventual design should link both layers.

Sign every capsule directly with the owner key

This gives a short verification chain but increases use of the long-lived owner key, obscures per-node attribution when one owner controls multiple nodes, and makes request completion depend on the owner keystore/keychain. Use owner signatures for short-lived delegation only.

Sign directly with the QUIC node transport key

This would bind receipts to the wire identity, but couples transport-key lifecycle to application evidence, expands the transport key's signing surface, and still requires owner/release evidence composition. Keep the node secret inside transport ownership and bind the receipt key through the existing owner/node certificate instead.

Give the plugin the owner or node private key

Rejected. Plugin compromise would become owner/node identity compromise, and it would bypass existing keystore, keychain, passphrase, rotation, and incident-response boundaries.

Use the Python reference implementation as the plugin runtime

This minimizes initial format work, but it would create a second plugin SDK/framing implementation, add interpreter and dependency packaging to every target, diverge from mesh-llm's native plugin patterns, and leave no credible production path. Python remains valuable as an independent conformance oracle, not as runtime architecture.

Embed or invoke Python behind a Rust wrapper

A native launcher around a Python worker would preserve the same deployment and supply-chain problems while adding IPC and failure modes. The prototype must implement the evidence path in Rust and use Python only in development/conformance tests.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions