You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Build a native Rust, installable Capsule Emit plugin that produces signed, hash-chained Agent Action Capsules for OpenAI inference exchanges served by mesh-llm.
local signing, verification, chaining, retrieval, and optional transparency anchoring;
non-streaming, streaming, error, denial, and cancellation behavior;
composition between mesh-llm inference receipts and agent/tool action capsules.
The plugin must consume host-provided request/route/response lifecycle data over plugin IPC. It must not listen as an HTTP reverse proxy, require clients to use a second port, replace an inference backend, or require a custom mesh-llm binary.
Experiment hypothesis
An installable plugin can create a useful, portable, independently verifiable record that says:
This collector observed mesh-llm accept this request commitment, select this claimed model/runtime route, and return this response commitment.
The prototype does not establish that the claimed model performed the computation correctly. A self-signed capsule can cryptographically seal a false runtime claim. Model artifact verification, native compute receipts, hardware-bound signing, TEE/GPU evidence, statistical fingerprints, and distributed stage receipts remain separate assurance layers.
The experiment succeeds if it proves the integration and evidence-envelope mechanics without overstating the resulting trust level.
Why a plugin
The existing Capsule Emit mesh experiment proved the capsule format against a real mesh-llm node and Goose session, but it had to be a sidecar because plugins could not receive mesh-llm's own OpenAI request envelope.
That architecture had unavoidable limitations:
clients had to target a second proxy port;
the collector sat outside the mesh-llm trust boundary;
streaming was buffered and re-synthesized;
client-compatibility transformations occurred in the sidecar;
it could not distinguish gateway, route-selection, and serving-host facts;
it could not prove whether a refusal happened before dispatch.
#1331 is intended to remove those limitations generically. This issue validates that work with a real consumer.
Scope
In scope
/v1/chat/completions;
streaming and non-streaming responses;
local-model serving for the primary acceptance run;
successful, invalid, denied, failed, timed-out, and cancelled exchanges;
host-provided route/model/runtime descriptors when available;
request and response commitments;
plugin-managed Ed25519 receipt signing with an owner-signed node/plugin delegation when the host owner identity is available;
inclusion and independent verification of the existing node ownership and release-build attestation evidence;
COSE_Sign1 signed statements;
performance instrumentation, adaptive evidence heuristics, and a comparative benchmark report;
per-plugin-instance hash chaining;
local JSONL ledger and signed-statement storage;
plugin HTTP resources for status, retrieval, and verification;
The plugin is out-of-process, like other mesh-llm plugins, but it is not a network sidecar. It uses the host control connection and negotiated side streams from #1331 and never occupies the OpenAI request path as an HTTP server.
The prototype must be a native Rust plugin using mesh-llm's public plugin author API. Python reference implementations are interoperability oracles used by development/conformance tests only; Python is not a runtime dependency, subprocess, embedded interpreter, packaging requirement, or fallback execution path.
Native Rust implementation requirements
The experiment should look and behave like an ordinary mesh-llm plugin:
use the mesh-llm-plugin crate, generated protocol types, plugin runtime, manifest builders/DSL, config schema, lifecycle hooks, and side-stream helpers;
use the plugin! authoring pattern where it fits instead of implementing a private control protocol;
expose status, receipt lookup, issuer material, and verification through normal host-projected plugin HTTP/resource surfaces;
use Tokio-compatible async I/O and keep blocking filesystem, model hashing, and cryptographic preparation off control-plane tasks;
stream request/response commitments incrementally where possible rather than retaining whole SSE transcripts;
use typed Rust structures for the mesh evidence extension and capsule model, with explicit serde representations and schema/version constants;
keep AAC, canonicalization, hashing, COSE/CBOR, signing, ledger, and anchoring dependencies inside the plugin rather than adding them to mesh-llm core;
use reviewed Rust cryptography/serialization crates and pin behavior with golden vectors; exact dependency selection is an implementation decision subject to normal dependency review;
never invoke Python, shell out to the reference CLI, or use FFI in the serving/runtime path;
never read the owner keystore, node key, keychain, passphrase environment, or any host private-key file directly;
package native per-platform plugin artifacts through the existing plugin installation/release conventions;
preserve the normal plugin health, startup, shutdown, configuration, upgrade, and state-directory behavior.
The Rust implementation must produce artifacts compatible with the upstream Python tooling. Development and CI may install capsule-emit, agent-action-capsule, and scitt-cose to verify interoperability, but a released plugin must run and verify local records without them.
Plugin declaration and grants
The plugin declares the mesh.openai.exchange.v1 hook with:
chat-completions request_received;
backend_selected;
exchange_finished;
buffered request body access;
streamed response body access;
access to only X-Capsule-Client-Nonce from request headers;
response metadata permission for X-Mesh-Receipt-Id;
read access to the public node identity/attestation bundle;
permission to request/renew the mesh.inference.capsule.sign.v1 signing delegation;
observe-only admission behavior.
The host configuration explicitly grants these scopes.
The prototype returns continue/abstain for every request. Admission decisions are tested with the exemplar policy plugin from #1331, not smuggled into Capsule Emit behavior.
The experiment runs best-effort/fail-open by default. If the plugin is unhealthy, inference continues and host status must show evidence unavailable. A separate test may exercise required/fail-closed mode, but that must be an operator decision.
Receipt lifecycle
Provisional receipt identity
For the observe-only Capsule Emit plugin, avoid adding a synchronous plugin round trip solely to allocate an identifier. Prefer the host-generated exchange_id as the provisional receipt_id and return it as response metadata:
X-Mesh-Receipt-Id: <opaque-id>
The plugin binds its pending state and final capsule to that identifier. If #1331 instead requires plugin-generated response metadata, benchmark that synchronous path independently and retain it only if it meets the pre-dispatch overhead budget below.
The header is a correlation reference, not the final capsule ID and not proof.
The plugin exposes a host-projected HTTP resource such as:
GET /plugins/capsule-emit/http/receipts/{receipt_id}
States:
pending;
complete;
failed;
incomplete;
not_found.
A complete result includes the final capsule_id, signed-statement reference, local verification result, anchor state, and evidence-strength labels. Prompt and completion bodies are never returned.
Request commitment
The plugin receives:
exact decoded request bytes;
host-computed SHA-256 of those bytes;
parsed OpenAI request;
endpoint, exchange ID, observation point, and sanitized nonce header.
It computes and records:
an exact wire_request_sha256 over the bytes the host accepted;
effect.request_digest using the Agent Action Capsule digest contract;
the canonicalization/transformation identifier used for that digest;
the client nonce and its source.
If the client sent X-Capsule-Client-Nonce, record client_supplied. Otherwise generate a fallback nonce and record plugin_generated_fallback. The fallback must never be described as client anti-replay evidence.
OpenAI requests commonly contain floating-point generation parameters. The implementation must publish test vectors showing exactly how JSON numbers enter the Agent Action Capsule digest. The previous experiment's decimal-string conversion may be reused only if the transformation is versioned and explicitly recorded. The exact wire digest remains available to remove ambiguity.
Raw prompts, messages, inline media, and tool arguments are discarded after the required commitments are computed. They are not written to the ledger or logs.
Route and model commitment
At backend_selected, the plugin binds available facts into the pending receipt:
requested model;
effective model;
local/remote target and observation point;
provider/backend kind;
model package or artifact digest;
tokenizer/template digest if available;
runtime/release artifact digest if available;
request attempt number;
transformations between original and effective requests;
source and verification state of every identity field.
Rules:
a display name, URL, or canonical model reference is not a model artifact digest;
a configured manifest fallback is labelled configured_claim, not host_verified;
release/build attestation is build provenance, not runtime integrity;
a remote route claim is not evidence from the remote compute boundary.
For the primary local-GGUF test, hash the actual resolved GGUF bytes or consume an equivalent host-provided verified artifact digest. For a Skippy package, prefer its canonical manifest/source/artifact digests.
Response commitment
For non-streaming responses, record:
exact client-facing response bytes and host-computed SHA-256;
semantic JSON response digest;
HTTP status and terminal category.
For streaming responses:
mesh-llm continues to stream original bytes directly to the client;
the host mirrors the final encoded SSE transcript to the plugin;
the plugin or host incrementally commits the ordered transcript;
the plugin may also build a bounded semantic reassembly for effect.response_digest;
the capsule records which representation each digest covers;
truncation, dropped chunks, cancellation, or observer overflow makes evidence incomplete.
Unlike the sidecar experiment, the plugin must not buffer and re-synthesize SSE, normalize tool calls, add tool-call IDs, remove mixed content, or otherwise alter the response.
A separate delivered_response_digest is required if the semantic response digest does not cover the exact emitted bytes.
Terminal semantics
Map host terminal states precisely:
Host state
Capsule interpretation
completed
confirmed/executed
policy denied before backend dispatch
denied/no dispatch
invalid request
rejected before dispatch
backend returned an error
failed/errored
transport failure after dispatch
failed/errored with transport context
client cancellation
cancelled, including whether bytes were emitted
incomplete observer transcript
evidence incomplete, never confirmed as complete
plugin absent/unhealthy
no capsule; host reports evidence unavailable
A sidecar cannot know that a refusal occurred before dispatch; the plugin lifecycle can. The prototype must use denied only when the host explicitly confirms that no backend dispatch occurred.
Capsule field mapping
Use the existing proof-of-concept mapping as the starting point:
Receipt fact
Agent Action Capsule mapping
request commitment
effect.request_digest plus x-mesh-poc-v1.wire_request_sha256
response commitment
effect.response_digest plus x-mesh-poc-v1.delivered_response_sha256
model_attestation.compute_attestation.runtime and x-mesh-poc-v1
generation parameters
x-mesh-poc-v1 as exact decimal strings
route/observation point
x-mesh-poc-v1
later fingerprint/TEE evidence
typed evidence references, empty in this prototype
node-local history
chain.parent_capsule_id with relation confirms
whole statement signature
COSE_Sign1 EdDSA
Fields not registered by the Agent Action Capsule specification remain under the explicit x-mesh-poc-v1 namespace. Do not present extension fields as standardized.
Publish the full sample schema and golden capsules as prototype artifacts.
Signing, node identity, and key management
The prototype should use mesh-llm's existing identity hierarchy rather than creating an unrelated trust root whenever an owner identity is available.
Existing identities remain separate
owner identity: the stable Ed25519 key created by mesh-llm auth init;
node identity: the QUIC/iroh endpoint key;
SignedNodeOwnership: the existing short-lived owner-signed binding from owner to node endpoint;
release build attestation: release-signer evidence for the packaged mesh-llm executable;
receipt key: a plugin-owned Ed25519 key used only for Capsule COSE statements.
The owner and node private keys remain host-owned. The plugin never reads or receives them.
Preferred owner-delegated flow
On first start, the plugin generates an Ed25519 receipt key in its plugin state directory.
An offline verifier confirms that the same owner authorized both the node and the scoped receipt key, then applies expiry/revocation/trust policy.
Why not sign every capsule with the owner key
The owner key may represent several nodes and is a high-value, long-lived identity. Using it for every request would:
increase online exposure and signing volume;
blur which node produced a receipt;
couple inference latency to keystore/keychain availability;
turn the host into a high-frequency signing oracle;
complicate future hardware-backed receipt keys.
The owner key signs only short-lived delegation claims at startup/renewal.
Receipt key storage and rotation
private key mode 0600 where supported;
atomic generation/persistence and fsync policy consistent with plugin state handling;
secret bytes zeroized where practical;
no private key in source, packages, ledgers, metrics, examples, status, or logs;
stable issuer ID derived from the receipt public key;
explicit key-generation counter/version in local state;
rotate on operator request, suspected compromise, incompatible key format, or configured policy;
delegation is invalidated and reissued when receipt key, plugin artifact/version, node key, owner key, or node ownership certificate changes;
renewal retains the receipt key when safe so ledger continuity does not require key rotation;
key rotation creates an explicit chain-transition record signed by the old key when available and the new delegated key;
expired/missing/revoked delegation cannot be represented as owner-delegated.
A software receipt key remains exportable. Owner delegation establishes authorization and attribution, not that signing occurred only on that machine. A future host-managed TPM/TEE/non-exportable key should fit the same delegation/evidence shape.
owner_delegated: prefer owner delegation, fall back to self-attested with an explicit downgraded state;
owner_delegated_required: plugin is not ready for evidence if a valid delegation is unavailable;
hardware_delegated: reserved for future attested/non-exportable keys.
No fallback silently preserves the stronger assurance label.
Release attestation composition
The identity bundle may include mesh-llm's verified release build attestation. The capsule records its digest, verification status, signer, build ID, commit, target, and artifact digest as a distinct evidence reference.
Release attestation remains certified build provenance. It does not prove that the running process is unmodified, that the plugin artifact was measured, or that the claimed model executed the request.
delegation_id/expiry inside the proof bundle, never telemetry.
Ledger and chaining
Store:
capsules.jsonl;
signed-statements/{capsule_id}.cose;
receipt_id to capsule_id/status index;
anchor queue/status when enabled.
Requirements:
append serialization is single-writer and crash-safe enough not to create an invalid chain silently;
restart recovers the previous chain head;
a partially written line is detected and quarantined or repaired explicitly;
signed statements are written atomically;
a chain fork is reported, not silently selected;
retention is configurable;
raw prompts and outputs are never persisted;
verification can run without network access.
Hash chaining is per plugin issuer/instance. It is not evidence that the node could not maintain a second private chain. Transparency or independent witnessing is a separate assurance.
Anchoring
Anchoring is disabled by default.
When explicitly enabled:
submit only the intended digest/statement reference, never prompt or output bodies;
use a durable retry queue;
distinguish pending, submitted, anchored, rejected, and failed;
never report anchored merely because submission was requested;
expose the receipt and verification result;
make the anchor URL configurable;
apply bounded retries and backoff;
provide a way to disable or purge unsubmitted test entries.
The acceptance run should use offline verification. A live public-log write requires a deliberate operator action after reviewing the exact capsule.
Plugin resources and configuration
Suggested projected HTTP/resources:
GET status;
GET receipts/{receipt_id};
GET capsules/{capsule_id};
GET issuer;
POST verify for a local capsule/signed statement, if safely bounded;
GET anchor queue/status.
Suggested configuration:
ledger directory;
retention;
issuer/node label;
identity mode: self_attested, owner_delegated, or owner_delegated_required;
receipt key path or generated-key policy;
receipt key rotation policy;
delegation renewal window within host-capped validity;
anchoring enabled and service URL;
request/response semantic digest mode;
maximum semantic reassembly bytes;
evidence mode: full, digest_only, sampled, or disabled;
sampling rate for explicitly selected sampled mode;
observer queue capacity and high-water mark;
best-effort load-shedding thresholds;
ledger flush/group-commit policy;
whether configured model/runtime manifest fallbacks are allowed;
logging level.
Sensitive settings remain owner-only. The web UI is optional and not required for prototype success.
Privacy requirements
Prompt, completion, tool arguments, inline media, and authorization secrets are not persisted.
No raw body appears in normal logs, error messages, health details, or plugin status.
Digest-only records must warn that low-entropy values may be vulnerable to dictionary attacks.
The plugin keeps body material only for the minimum time needed to compute commitments.
Test fixtures use synthetic prompts and outputs.
Any permalink/bundle flow must be reviewed to ensure it contains only intended digest-level evidence.
Performance hypothesis, heuristics, and decision gate
Performance is part of the experiment's validity. Functional capsule generation is not sufficient if the observer materially worsens time-to-first-token, token cadence, throughput, memory use, or request reliability.
Hot-path hypothesis
The expected low-overhead shape is:
the host allocates exchange_id/receipt_id without waiting for the observe-only plugin;
owner/node identity evidence and receipt-key delegation are loaded/verified at startup and renewed outside request handling;
immutable model, package, runtime, delegation, and public-key digests are computed once and cached, never rehashed per request;
request bytes already buffered by OpenAI ingress are hashed once without an additional full copy;
response bytes are committed incrementally from the final emitter tap;
semantic response reconstruction is bounded and performed off the token-emission task;
capsule serialization, COSE signing, ledger append, and anchoring occur after the final client byte;
anchoring is always asynchronous and never on the inference or receipt-finalization critical path;
filesystem work uses a bounded worker and configurable group commit rather than fsync per token or chunk;
queues are bounded and expose pressure instead of growing with stream duration.
The benchmark must verify these assumptions rather than treating them as true by design.
Evidence modes
The prototype supports explicit, observable modes:
full: wire commitments, bounded semantic commitments, signing, and durable ledger for every request;
digest_only: incremental wire commitments and a signed record without full semantic reconstruction;
sampled: full evidence for a configured random sample; sampling must not use prompt text or prompt hashes;
disabled: no request body delivery or evidence work.
Mode is operator configuration. Adaptive load shedding is permitted only in best-effort mode and must be recorded per receipt as a downgrade or evidence-unavailable outcome.
In required/fail-closed mode, the plugin must never silently change full to digest_only or sampled. It either produces the required evidence or rejects before dispatch according to #1331.
Adaptive heuristics to evaluate
Implement and compare at least these heuristics:
Cache immutable artifact/runtime digests and invalidate only on a verified model/runtime change.
Use host-generated receipt IDs so observe-only mode has no identifier-allocation IPC round trip.
Hash streaming response bytes incrementally and keep only a bounded semantic reconstruction window.
Switch a best-effort request from full to digest_only when its semantic body limit is exceeded.
Mark evidence incomplete when the observer queue crosses its hard limit; never claim a complete transcript after dropped bytes.
Shed best-effort observation when plugin queue latency or pending receipt count exceeds configured high-water marks.
Batch ledger flushes while preserving crash-detection and chain ordering.
Keep transparency submission on a separate durable queue.
Permit explicit sampling for low-value traffic while retaining full mode for clients that request evidence.
Compare out-of-process plugin IPC/copy cost against a minimal in-host digest tap. If IPC dominates, reconsider which portion belongs in core versus the plugin.
Every heuristic decision is represented by a bounded mode/outcome/reason. No heuristic may be inferred later from a missing capsule.
Benchmark configurations
Measure the same build/model/settings in these configurations:
A. no plugin installed;
B. plugin installed with lifecycle/body grants disabled;
C. plugin active in digest_only mode with ledger persistence disabled;
D. plugin active in full mode with local signing and ledger persistence;
E. full mode with anchoring enabled but network submission isolated behind its queue;
F. the previous reverse-proxy sidecar, as an informative comparison rather than the target architecture.
This separates generic hook overhead, IPC/body mirroring, hashing, canonicalization/signing, persistence, and anchoring effects.
Workloads
Use both a deterministic fast mock backend, which makes fixed overhead visible, and a real local GGUF, which shows product impact.
At minimum:
short non-streaming request/response;
short prompt with long streaming output;
long-context request;
tool-calling request with nested schemas;
large multimodal-shaped request fixture without private data;
concurrency 1, 4, and 16 or the largest stable level on the test host;
successful, backend-error, and client-cancelled streams.
Use fixed prompts, seeds where supported, output limits, model artifact, runtime build, and machine state. Record warm-up policy, repetitions, sample counts, machine profile, and all plugin settings.
Measurements
Report absolute and relative deltas, distributions, and run-to-run variance—not only averages.
End-to-end:
admission-to-backend-dispatch delay;
time to first response byte/token;
inter-chunk/inter-token latency;
total response latency;
output tokens per second;
requests per second at concurrency;
error, cancellation, timeout, and incomplete-evidence rates.
Plugin/runtime:
lifecycle IPC round-trip or delivery duration by phase;
request hashing/canonicalization duration;
final-emitter tap lag;
observer queue depth/high-water mark;
dropped events and dropped bytes;
semantic reconstruction duration and peak bytes;
capsule build/canonicalization duration;
COSE signing and verification duration;
ledger append, flush, and recovery duration;
final-client-byte to receipt-complete delay;
pending receipt count;
anchor queue depth and attempt outcome;
plugin CPU time, RSS/high-water memory, bytes copied/allocated where measurable, and disk bytes/fsync rate.
Use p50, p95, and p99 for latency distributions, median and dispersion across repeated trials, and confidence intervals or tolerance-aware comparison where the sample permits.
Provisional performance budget
Agree the final budget before implementation measurements are reviewed. The prototype starts with these provisional gates:
deterministic mock: p95 admission-to-dispatch overhead no more than 5 ms in full mode and no more than 2 ms in digest_only mode;
real local model: p95 time-to-first-token and total-latency regression no more than 2%;
streaming: median token throughput regression no more than 2%, with no systematic p95 inter-token regression beyond baseline variance;
no observer drops, false-complete receipts, or added request failures within the declared supported concurrency;
memory remains bounded by configured queues/reassembly limits and does not grow with stream duration;
p95 receipt completion occurs within 250 ms of the final client byte with anchoring excluded.
A result outside budget is not waived because model execution is slow. Report both absolute and relative effects, and repeat borderline results.
Decision rule
Pass: full mode meets the agreed budget across the primary workloads.
Conditional: digest_only meets budget but full does not. Keep the plugin experiment, make full evidence explicit/high-assurance opt-in, and investigate canonicalization/persistence offloading.
Redesign: hook/IPC/body mirroring dominates. Move only the minimal byte commitment and correlation tap into host core, leaving capsule construction, signing, storage, and anchoring in the plugin.
Stop: even a minimal digest tap causes unacceptable regression or cannot preserve streaming correctness. Do not graduate the plugin architecture.
The findings report must identify the dominant cost center and recommend pass, conditional, redesign, or stop.
Metrics and telemetry privacy
Detailed benchmark artifacts remain local and contain measurements/configuration only—never request or response bodies.
If runtime metrics are exported through mesh-llm OTLP, export remains explicitly operator-enabled and metrics-only. There is no hard-coded collector. Proposed logical instruments include:
mesh_llm_capsule_receipt_total;
mesh_llm_capsule_hook_duration_ms;
mesh_llm_capsule_request_commit_duration_ms;
mesh_llm_capsule_response_tap_lag_ms;
mesh_llm_capsule_build_duration_ms;
mesh_llm_capsule_sign_duration_ms;
mesh_llm_capsule_ledger_append_duration_ms;
mesh_llm_capsule_receipt_completion_delay_ms;
mesh_llm_capsule_observer_queue_depth;
mesh_llm_capsule_observer_drop_total;
mesh_llm_capsule_anchor_queue_depth;
mesh_llm_capsule_anchor_attempt_total;
mesh_llm_capsule_identity_delegation_total;
mesh_llm_capsule_identity_delegation_duration_ms.
Allowed attributes are bounded enums only: phase, evidence_mode, identity_mode, delegation_outcome, terminal_outcome, completeness, stream_mode, pressure_outcome, downgrade_reason, and anchor_outcome.
Do not export prompts, completions, tool arguments, nonces, request/capsule/receipt IDs, raw or hashed prompt data, model paths, ledger paths, endpoint URLs, hostnames, or raw node/device identifiers. Metrics collection and export are bounded and non-blocking.
Any implementation that adds these metrics to mesh-llm's OTLP path must update TELEMETRY_ATTRIBUTE_ALLOWLIST, docs/plugins/telemetry.md, and focused privacy tests as required by the telemetry review contract.
Deliverables
A native Rust, external, installable mesh-llm plugin package; no mesh-llm fork, Python runtime, or HTTP reverse proxy.
Rust implementations of the required AAC data model, canonicalization contract, COSE_Sign1 producer/verifier, chaining, ledger, and optional anchor client.
Cross-language conformance fixtures proving compatibility with the upstream Python reference tools.
Reproducible development launch and package/install instructions.
COSE signature verification rejects a mutated capsule.
Capsule ID verification rejects mutated effect/model fields.
Restart preserves a valid chain head.
Partial ledger writes do not create a silently accepted chain.
No secret headers or raw bodies are persisted.
Upstream Python verification accepts Rust-produced capsules and COSE statements.
The Rust verifier accepts committed Python-produced golden capsules and statements.
Rust and Python implementations produce identical canonical bytes and digest results for the agreed conformance vectors.
The packaged plugin starts and completes the local verification path on every supported target without Python installed.
The plugin never opens or reads the owner keystore, node-key file, keychain, or passphrase material.
An owner-delegated capsule verifies through receipt key -> PluginSigningDelegation -> owner key and through SignedNodeOwnership -> the same owner/node.
Missing owner identity produces self-attested downgrade or not-ready according to configured identity mode.
Expired, revoked, mismatched-node, mismatched-plugin, wrong-scope, and bad-signature delegations are rejected.
Delegation renewal occurs outside request handling and does not change the receipt key unnecessarily.
Receipt-key rotation produces an explicit, verifiable chain transition.
Release attestation is verified and presented separately from node ownership/runtime claims.
Host integration tests
Non-streaming success emits one verifiable capsule.
Streaming success remains live passthrough and emits one verifiable capsule.
Backend error emits an errored capsule with the actual error commitment.
Pre-dispatch denial emits denied only when the host confirms no dispatch.
Client cancellation records cancellation and transcript completeness.
Plugin timeout in best-effort mode does not stop inference and reports evidence unavailable.
Plugin timeout in operator-selected required mode fails before dispatch.
Receipt ID header resolves pending then complete.
Plugin restart during an exchange does not produce a false complete capsule.
Performance acceptance
Benchmark no-plugin, disabled-grant, digest_only, full, queued-anchor, and sidecar configurations.
Run the deterministic mock workload and a real local-GGUF workload.
Run against a real mesh-llm local GGUF, not only a mock.
Run a real Goose tool-call session through the normal mesh-llm port.
Produce independently verifiable inference and tool-action capsule streams.
Confirm the client receives the original mesh-llm streaming bytes without sidecar normalization.
Record the actual model artifact digest and its evidence source.
Keep the acceptance artifacts offline unless anchoring is explicitly approved.
Stretch acceptance
Route one request over a two-node private mesh and label gateway versus serving-host observations.
Link one inference receipt to one Goose/MCP action capsule by shared digest/reference.
Attach a Skippy native generation receipt as a typed evidence reference.
Success criteria
The experiment is successful when a stock mesh-llm build can install the native Rust plugin on a machine without Python, serve a real streaming request on its normal OpenAI port, return a receipt reference, and later provide a locally verifiable signed capsule whose request/response commitments match the host-emitted bytes. When the node has an owner identity, the proof must also verify through the scoped PluginSigningDelegation and existing SignedNodeOwnership without exposing either host private key.
Functional success is necessary but insufficient. Graduation also requires the full mode to meet the agreed performance budget, or an explicit conditional/redesign decision supported by the benchmark evidence.
Success does not depend on proving correct model execution or productionizing the public transparency service.
Evaluation and follow-up decision
At the end of the experiment, publish a short findings report answering:
What are the measured p50/p95/p99 latency, token-throughput, CPU, memory, copying, queue, persistence, and receipt-completion costs of each evidence mode?
Which heuristic or subsystem dominates overhead, and does the result support pass, conditional, redesign, or stop?
Can streaming evidence be complete without meaningful latency or memory regression?
Is the Agent Action Capsule profile expressive enough without abusing unregistered fields?
Which mesh-specific extension fields should be proposed upstream?
Does the Rust producer/verifier interoperate completely with the Python reference implementation, and which reusable pieces should be proposed upstream?
What is required to link this API-boundary capsule to an authoritative native generation receipt?
Does the owner -> node ownership plus owner -> scoped receipt-key delegation provide the right software identity chain, rotation, and revocation semantics?
What is required to upgrade the delegated receipt key to host-managed or hardware-attested/non-exportable storage?
Should the prototype graduate, remain experimental, or be retired?
Alternatives considered
Continue the sidecar
Useful as a compatibility demo and independent observer, but it cannot satisfy the in-host policy and provenance goals and changes streaming behavior.
Implement Capsule Emit directly in core
Would couple mesh-llm to one evidence format before the plugin contract and threat model are validated.
Use only native generation receipts
More authoritative about tokens, but does not commit to the original OpenAI request or exact client-visible response. The eventual design should link both layers.
Sign every capsule directly with the owner key
This gives a short verification chain but increases use of the long-lived owner key, obscures per-node attribution when one owner controls multiple nodes, and makes request completion depend on the owner keystore/keychain. Use owner signatures for short-lived delegation only.
Sign directly with the QUIC node transport key
This would bind receipts to the wire identity, but couples transport-key lifecycle to application evidence, expands the transport key's signing surface, and still requires owner/release evidence composition. Keep the node secret inside transport ownership and bind the receipt key through the existing owner/node certificate instead.
Give the plugin the owner or node private key
Rejected. Plugin compromise would become owner/node identity compromise, and it would bypass existing keystore, keychain, passphrase, rotation, and incident-response boundaries.
Use the Python reference implementation as the plugin runtime
This minimizes initial format work, but it would create a second plugin SDK/framing implementation, add interpreter and dependency packaging to every target, diverge from mesh-llm's native plugin patterns, and leave no credible production path. Python remains valuable as an independent conformance oracle, not as runtime architecture.
Embed or invoke Python behind a Rust wrapper
A native launcher around a Python worker would preserve the same deployment and supply-chain problems while adding IPC and failure modes. The prototype must implement the evidence path in Rust and use Python only in development/conformance tests.
Summary
Build a native Rust, installable Capsule Emit plugin that produces signed, hash-chained Agent Action Capsules for OpenAI inference exchanges served by mesh-llm.
This is a prototype experiment to validate:
The plugin must consume host-provided request/route/response lifecycle data over plugin IPC. It must not listen as an HTTP reverse proxy, require clients to use a second port, replace an inference backend, or require a custom mesh-llm binary.
Experiment hypothesis
An installable plugin can create a useful, portable, independently verifiable record that says:
The prototype does not establish that the claimed model performed the computation correctly. A self-signed capsule can cryptographically seal a false runtime claim. Model artifact verification, native compute receipts, hardware-bound signing, TEE/GPU evidence, statistical fingerprints, and distributed stage receipts remain separate assurance layers.
The experiment succeeds if it proves the integration and evidence-envelope mechanics without overstating the resulting trust level.
Why a plugin
The existing Capsule Emit mesh experiment proved the capsule format against a real mesh-llm node and Goose session, but it had to be a sidecar because plugins could not receive mesh-llm's own OpenAI request envelope.
That architecture had unavoidable limitations:
#1331 is intended to remove those limitations generically. This issue validates that work with a real consumer.
Scope
In scope
Stretch scope
Out of scope
Architecture
The plugin is out-of-process, like other mesh-llm plugins, but it is not a network sidecar. It uses the host control connection and negotiated side streams from #1331 and never occupies the OpenAI request path as an HTTP server.
The prototype must be a native Rust plugin using mesh-llm's public plugin author API. Python reference implementations are interoperability oracles used by development/conformance tests only; Python is not a runtime dependency, subprocess, embedded interpreter, packaging requirement, or fallback execution path.
Native Rust implementation requirements
The experiment should look and behave like an ordinary mesh-llm plugin:
The Rust implementation must produce artifacts compatible with the upstream Python tooling. Development and CI may install capsule-emit, agent-action-capsule, and scitt-cose to verify interoperability, but a released plugin must run and verify local records without them.
Plugin declaration and grants
The plugin declares the mesh.openai.exchange.v1 hook with:
The host configuration explicitly grants these scopes.
The prototype returns continue/abstain for every request. Admission decisions are tested with the exemplar policy plugin from #1331, not smuggled into Capsule Emit behavior.
The experiment runs best-effort/fail-open by default. If the plugin is unhealthy, inference continues and host status must show evidence unavailable. A separate test may exercise required/fail-closed mode, but that must be an operator decision.
Receipt lifecycle
Provisional receipt identity
For the observe-only Capsule Emit plugin, avoid adding a synchronous plugin round trip solely to allocate an identifier. Prefer the host-generated exchange_id as the provisional receipt_id and return it as response metadata:
The plugin binds its pending state and final capsule to that identifier. If #1331 instead requires plugin-generated response metadata, benchmark that synchronous path independently and retain it only if it meets the pre-dispatch overhead budget below.
The header is a correlation reference, not the final capsule ID and not proof.
The plugin exposes a host-projected HTTP resource such as:
States:
A complete result includes the final capsule_id, signed-statement reference, local verification result, anchor state, and evidence-strength labels. Prompt and completion bodies are never returned.
Request commitment
The plugin receives:
It computes and records:
If the client sent X-Capsule-Client-Nonce, record client_supplied. Otherwise generate a fallback nonce and record plugin_generated_fallback. The fallback must never be described as client anti-replay evidence.
OpenAI requests commonly contain floating-point generation parameters. The implementation must publish test vectors showing exactly how JSON numbers enter the Agent Action Capsule digest. The previous experiment's decimal-string conversion may be reused only if the transformation is versioned and explicitly recorded. The exact wire digest remains available to remove ambiguity.
Raw prompts, messages, inline media, and tool arguments are discarded after the required commitments are computed. They are not written to the ledger or logs.
Route and model commitment
At backend_selected, the plugin binds available facts into the pending receipt:
Rules:
For the primary local-GGUF test, hash the actual resolved GGUF bytes or consume an equivalent host-provided verified artifact digest. For a Skippy package, prefer its canonical manifest/source/artifact digests.
Response commitment
For non-streaming responses, record:
For streaming responses:
Unlike the sidecar experiment, the plugin must not buffer and re-synthesize SSE, normalize tool calls, add tool-call IDs, remove mixed content, or otherwise alter the response.
A separate delivered_response_digest is required if the semantic response digest does not cover the exact emitted bytes.
Terminal semantics
Map host terminal states precisely:
A sidecar cannot know that a refusal occurred before dispatch; the plugin lifecycle can. The prototype must use denied only when the host explicitly confirms that no backend dispatch occurred.
Capsule field mapping
Use the existing proof-of-concept mapping as the starting point:
Fields not registered by the Agent Action Capsule specification remain under the explicit x-mesh-poc-v1 namespace. Do not present extension fields as standardized.
Publish the full sample schema and golden capsules as prototype artifacts.
Signing, node identity, and key management
The prototype should use mesh-llm's existing identity hierarchy rather than creating an unrelated trust root whenever an owner identity is available.
Existing identities remain separate
The owner and node private keys remain host-owned. The plugin never reads or receives them.
Preferred owner-delegated flow
This produces:
An offline verifier confirms that the same owner authorized both the node and the scoped receipt key, then applies expiry/revocation/trust policy.
Why not sign every capsule with the owner key
The owner key may represent several nodes and is a high-value, long-lived identity. Using it for every request would:
The owner key signs only short-lived delegation claims at startup/renewal.
Receipt key storage and rotation
A software receipt key remains exportable. Owner delegation establishes authorization and attribution, not that signing occurred only on that machine. A future host-managed TPM/TEE/non-exportable key should fit the same delegation/evidence shape.
Identity modes
Configuration supports:
No fallback silently preserves the stronger assurance label.
Release attestation composition
The identity bundle may include mesh-llm's verified release build attestation. The capsule records its digest, verification status, signer, build ID, commit, target, and artifact digest as a distinct evidence reference.
Release attestation remains certified build provenance. It does not prove that the running process is unmodified, that the plugin artifact was measured, or that the claimed model executed the request.
Assurance labels
At minimum:
Ledger and chaining
Store:
Requirements:
Hash chaining is per plugin issuer/instance. It is not evidence that the node could not maintain a second private chain. Transparency or independent witnessing is a separate assurance.
Anchoring
Anchoring is disabled by default.
When explicitly enabled:
The acceptance run should use offline verification. A live public-log write requires a deliberate operator action after reviewing the exact capsule.
Plugin resources and configuration
Suggested projected HTTP/resources:
Suggested configuration:
Sensitive settings remain owner-only. The web UI is optional and not required for prototype success.
Privacy requirements
Performance hypothesis, heuristics, and decision gate
Performance is part of the experiment's validity. Functional capsule generation is not sufficient if the observer materially worsens time-to-first-token, token cadence, throughput, memory use, or request reliability.
Hot-path hypothesis
The expected low-overhead shape is:
The benchmark must verify these assumptions rather than treating them as true by design.
Evidence modes
The prototype supports explicit, observable modes:
Mode is operator configuration. Adaptive load shedding is permitted only in best-effort mode and must be recorded per receipt as a downgrade or evidence-unavailable outcome.
In required/fail-closed mode, the plugin must never silently change full to digest_only or sampled. It either produces the required evidence or rejects before dispatch according to #1331.
Adaptive heuristics to evaluate
Implement and compare at least these heuristics:
Every heuristic decision is represented by a bounded mode/outcome/reason. No heuristic may be inferred later from a missing capsule.
Benchmark configurations
Measure the same build/model/settings in these configurations:
A. no plugin installed;
B. plugin installed with lifecycle/body grants disabled;
C. plugin active in digest_only mode with ledger persistence disabled;
D. plugin active in full mode with local signing and ledger persistence;
E. full mode with anchoring enabled but network submission isolated behind its queue;
F. the previous reverse-proxy sidecar, as an informative comparison rather than the target architecture.
This separates generic hook overhead, IPC/body mirroring, hashing, canonicalization/signing, persistence, and anchoring effects.
Workloads
Use both a deterministic fast mock backend, which makes fixed overhead visible, and a real local GGUF, which shows product impact.
At minimum:
Use fixed prompts, seeds where supported, output limits, model artifact, runtime build, and machine state. Record warm-up policy, repetitions, sample counts, machine profile, and all plugin settings.
Measurements
Report absolute and relative deltas, distributions, and run-to-run variance—not only averages.
End-to-end:
Plugin/runtime:
Use p50, p95, and p99 for latency distributions, median and dispersion across repeated trials, and confidence intervals or tolerance-aware comparison where the sample permits.
Provisional performance budget
Agree the final budget before implementation measurements are reviewed. The prototype starts with these provisional gates:
A result outside budget is not waived because model execution is slow. Report both absolute and relative effects, and repeat borderline results.
Decision rule
The findings report must identify the dominant cost center and recommend pass, conditional, redesign, or stop.
Metrics and telemetry privacy
Detailed benchmark artifacts remain local and contain measurements/configuration only—never request or response bodies.
If runtime metrics are exported through mesh-llm OTLP, export remains explicitly operator-enabled and metrics-only. There is no hard-coded collector. Proposed logical instruments include:
Allowed attributes are bounded enums only: phase, evidence_mode, identity_mode, delegation_outcome, terminal_outcome, completeness, stream_mode, pressure_outcome, downgrade_reason, and anchor_outcome.
Do not export prompts, completions, tool arguments, nonces, request/capsule/receipt IDs, raw or hashed prompt data, model paths, ledger paths, endpoint URLs, hostnames, or raw node/device identifiers. Metrics collection and export are bounded and non-blocking.
Any implementation that adds these metrics to mesh-llm's OTLP path must update TELEMETRY_ATTRIBUTE_ALLOWLIST, docs/plugins/telemetry.md, and focused privacy tests as required by the telemetry review contract.
Deliverables
Test matrix
Contract tests
Host integration tests
Performance acceptance
Live acceptance
Stretch acceptance
Success criteria
The experiment is successful when a stock mesh-llm build can install the native Rust plugin on a machine without Python, serve a real streaming request on its normal OpenAI port, return a receipt reference, and later provide a locally verifiable signed capsule whose request/response commitments match the host-emitted bytes. When the node has an owner identity, the proof must also verify through the scoped PluginSigningDelegation and existing SignedNodeOwnership without exposing either host private key.
Functional success is necessary but insufficient. Graduation also requires the full mode to meet the agreed performance budget, or an explicit conditional/redesign decision supported by the benchmark evidence.
Success does not depend on proving correct model execution or productionizing the public transparency service.
Evaluation and follow-up decision
At the end of the experiment, publish a short findings report answering:
Alternatives considered
Continue the sidecar
Useful as a compatibility demo and independent observer, but it cannot satisfy the in-host policy and provenance goals and changes streaming behavior.
Implement Capsule Emit directly in core
Would couple mesh-llm to one evidence format before the plugin contract and threat model are validated.
Use only native generation receipts
More authoritative about tokens, but does not commit to the original OpenAI request or exact client-visible response. The eventual design should link both layers.
Sign every capsule directly with the owner key
This gives a short verification chain but increases use of the long-lived owner key, obscures per-node attribution when one owner controls multiple nodes, and makes request completion depend on the owner keystore/keychain. Use owner signatures for short-lived delegation only.
Sign directly with the QUIC node transport key
This would bind receipts to the wire identity, but couples transport-key lifecycle to application evidence, expands the transport key's signing surface, and still requires owner/release evidence composition. Keep the node secret inside transport ownership and bind the receipt key through the existing owner/node certificate instead.
Give the plugin the owner or node private key
Rejected. Plugin compromise would become owner/node identity compromise, and it would bypass existing keystore, keychain, passphrase, rotation, and incident-response boundaries.
Use the Python reference implementation as the plugin runtime
This minimizes initial format work, but it would create a second plugin SDK/framing implementation, add interpreter and dependency packaging to every target, diverge from mesh-llm's native plugin patterns, and leave no credible production path. Python remains valuable as an independent conformance oracle, not as runtime architecture.
Embed or invoke Python behind a Rust wrapper
A native launcher around a Python worker would preserve the same deployment and supply-chain problems while adding IPC and failure modes. The prototype must implement the evidence path in Rust and use Python only in development/conformance tests.