Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -178,7 +178,7 @@ catalog_path: catalog.json # approved tool catalog

listen_addr: "127.0.0.1:8443" # tokenless dev mode is loopback-only; set CMCP_BEARER_TOKEN before binding wider
max_response_size_bytes: 2097152 # 2 MB default
policy_reload_interval_seconds: 0 # leave at 0; above 0 is a trap in production, see docs/spec/policy-hot-reload.md
policy_reload_interval_seconds: 0 # >0 with a pinned CMCP_POLICY_HASH refuses to start, see docs/spec/policy-hot-reload.md
```

Environment variables:
Expand Down
2 changes: 1 addition & 1 deletion STATUS.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ picture is stated once. Developer Preview: interfaces may change before v1.0.
| Transparency-log anchoring for TRACE Claims | v0.2 | Write and lookup. |
| Server-side (provider) attestation | Not yet (Phase 2) | Phase 1 attests the gateway boundary only. |
| Server provenance checking | Shipped | Consumes [server-provenance-v1](https://github.com/agentrust-io/trace-spec/blob/main/spec/server-provenance-v1.md) records: verifies the signature against a configured publisher key, then compares the record's tool-catalog hash against the tools the server advertises **to this gateway**. Five outcomes reach the audit chain and none of them is silent: `verified`, `catalog-mismatch` (the document is fine and the server is not), `invalid`, `unchecked` (verified but the tool list was unavailable, so the comparison that matters never ran), `absent`. Absence is recorded and non-fatal by default, because almost no MCP server has a record and a gateway that refuses to route without one gets disabled on first contact. Set `attestation.required_provenance_kind` for a floor. |
| Real-time policy update without enclave restart | Not yet, and the knob that looks like it is a trap | `policy_reload_interval_seconds` defaults to `0`, and a policy change requires a restart. Setting it above `0` does **not** buy hot-reload in production: the reload re-validates the new bundle against the hash pinned at startup, so any genuinely changed bundle fails and the old policy keeps being enforced. Worse, the failure path does not advance the interval, so every subsequent tool call re-reads and re-hashes the whole bundle on the enforcement path. Leave it at `0`. See [policy-hot-reload.md](docs/spec/policy-hot-reload.md) for the measurements and the options. |
| Real-time policy update without enclave restart | Not yet; the configuration that pretended to offer it is now refused | `policy_reload_interval_seconds` defaults to `0` and a policy change requires a restart. Setting it above `0` alongside a pinned `CMCP_POLICY_HASH` used to look enabled and do nothing: every reload was re-validated against the startup hash, so a changed bundle was always rejected while the old policy kept being enforced, and the failed reload re-read and re-hashed the whole bundle on every subsequent tool call. That pairing now aborts startup (`POLICY_RELOAD_PINNED_HASH`), and a failing reload costs one attempt per interval rather than one per request. Automatic reload works only under `CMCP_DEV_MODE=1`, where no hash is pinned. The agreed fix for production is a pinned signing key rather than a pinned artifact hash, which is designed but not built: see [policy-hot-reload.md](docs/spec/policy-hot-reload.md). |
| AARM R4 five decision types | Shipped, with caveats | ALLOW, DENY, MODIFY, STEP_UP, DEFER are recorded in the audit chain. MODIFY is recorded as `redact`, DEFER is classified but not asynchronously enforced, and the TRACE Claim still carries the pre-AARM vocabulary. See [LIMITATIONS.md](LIMITATIONS.md). |
| AARM R8 telemetry export | Shipped | OpenTelemetry spans mirroring audit entries. Opt in with `CMCP_OTEL_ENABLED=1` and `pip install cmcp-runtime[otel]`; a no-op otherwise. Exports digests, never payloads. The audit chain stays authoritative. |
| AARM R2/R3 declared intent | Not implemented | cMCP takes no declared-intent input, so the intent-alignment half of R2 and R3 is unmet. Adding one changes the MCP-facing surface. |
Expand Down
2 changes: 1 addition & 1 deletion docs/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -105,7 +105,7 @@ All fields are optional as a group. If `path` is set, `trust_anchor_path` must a
| `catalog_path` | string | `catalog.json` | Path to the JSON tool catalog. Path traversal (`..` components) is rejected. |
| `listen_addr` | string | `0.0.0.0:8443` | Address and port the gateway binds to. Default is `127.0.0.1:8443` in tokenless `CMCP_DEV_MODE=1`, otherwise `0.0.0.0:8443`. Tokenless dev mode requires loopback (e.g., `127.0.0.1:8443`, `localhost:8443`, `[::1]:8443`). Wildcard, LAN, public, and non-loopback hostname binds require `CMCP_BEARER_TOKEN`. |
| `max_response_size_bytes` | integer | `2097152` | Maximum tool response size in bytes (2MB). Must be a positive integer. Responses exceeding this limit are rejected before inspection. |
| `policy_reload_interval_seconds` | integer | `0` | Interval in seconds between automatic Cedar bundle reloads. `0` disables automatic reload, and **`0` is the only value to use today.** Above `0` the reload re-validates the new bundle against the hash pinned at startup (`CMCP_POLICY_HASH`), so a bundle that actually changed is rejected and the old policy stays in force; the failed reload also does not advance the interval, making every later tool call re-read and re-hash the bundle on the enforcement path. It works only under `CMCP_DEV_MODE=1`, where no hash is pinned. See [Policy Hot-Reload](spec/policy-hot-reload.md). |
| `policy_reload_interval_seconds` | integer | `0` | Interval in seconds between automatic Cedar bundle reloads. `0` disables automatic reload. **Setting it above `0` together with a pinned `CMCP_POLICY_HASH` aborts startup** (`POLICY_RELOAD_PINNED_HASH`): the reload re-validates against that pinned hash, so a bundle that actually changed would always be rejected and the old policy would stay in force. The two are alternatives, not layers — pin a hash for a policy that must not change, or use `CMCP_DEV_MODE=1` where reload works because nothing is pinned. Production hot-reload needs a pinned signing key instead, which is designed but not built: see [Policy Hot-Reload](spec/policy-hot-reload.md). |

## Environment variables

Expand Down
1 change: 1 addition & 0 deletions docs/spec/error-codes.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ This is the normative registry for all error codes used across the cMCP Runtime.
| `ATTESTATION_PROVIDER_UNSUPPORTED` | 500 | FATAL | No supported TEE provider detected and `CMCP_DEV_MODE` is not set | [attestation.md §1.1](attestation.md) |
| `ATTESTATION_PROVIDER_NOT_IMPLEMENTED` | 501 | FATAL | A recognized provider was explicitly selected but is not yet implemented (e.g. `opaque`) | [attestation.md §1.1](attestation.md) |
| `POLICY_HASH_MISMATCH` | 500 | FATAL | Measured policy bundle hash does not match deployment manifest | [failure-modes.md FM-4](failure-modes.md) |
| `POLICY_RELOAD_PINNED_HASH` | 500 | FATAL | `policy_reload_interval_seconds > 0` configured alongside a pinned `CMCP_POLICY_HASH`. Every reload is validated against that hash, so a changed bundle could never be installed; refused at startup rather than appearing to work | [policy-hot-reload.md](policy-hot-reload.md) |
| `CATALOG_HASH_MISMATCH` | 500 | FATAL | Measured catalog hash does not match deployment manifest | [attestation.md §5](attestation.md) |
| `AGENT_MANIFEST_BINDING_FAILED` | 500 | FATAL | Signed Agent Manifest signature, authenticated subject, policy hash, or catalog hash did not match the runtime session inputs | [session-policy.md](session-policy.md) |
| `TOOL_NOT_IN_CATALOG` | 403 | WARN | Agent requested a tool not present in the attested catalog | [cedar-policy.md](cedar-policy.md) |
Expand Down
95 changes: 73 additions & 22 deletions docs/spec/policy-hot-reload.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
# Policy Hot-Reload

**Document status:** Design proposal, nothing built
**Document status:** Direction decided (option A, signing key); the two
direction-independent fixes have landed, the signing-key model has not been built
**Applies to:** cMCP Runtime gateway (`PolicyStore`, `startup`)
**Related config:** `policy_reload_interval_seconds`

Expand All @@ -11,8 +12,12 @@
Hot-reload is not missing. It is implemented in `PolicyStore.reload_if_stale`, wired
into `PolicyEvaluator`, and documented as a supported knob — and **it cannot swap a
policy in any production configuration.** This document records why, measures what
the current code does instead, and lays out the options for fixing it. No code
changes accompany it.
the current code does instead, and lays out the options for fixing it.

The direction is now decided (option A, a pinned signing key) and the two fixes
that did not depend on the direction have landed: the guaranteed-inert
configuration is refused at startup, and a failing reload no longer re-reads the
bundle on every request. The signing-key model itself is not built.

`STATUS.md` says real-time policy update is "Not yet" because
`policy_reload_interval_seconds` is `0`. That reads as "unimplemented, default off".
Expand Down Expand Up @@ -201,25 +206,71 @@ config error** rather than a warning at request time.
restart, which for a confidential gateway means an attestation cycle. That is a
real operational burden and the reason this item is on the list at all.

## Recommendation is not the point yet, but two things are not optional

Whichever direction is chosen, two fixes stand on their own and do not depend on
it:

1. **A configuration that cannot work must not start.** `expected_hash` set
together with `reload_interval_seconds > 0` is, today, guaranteed-inert. Under
option E that pairing is the config error; under A/B/C/D it stops existing.
Either way it must never again be a thing an operator can switch on and believe.
2. **The interval must be honoured on failure too.** `_last_reload_at` has to
advance whether or not the reload succeeded, or a failing reload turns into
per-request bundle I/O on the enforcement path. This is worth fixing even if
hot-reload is removed, because the same shape will reappear in the next thing
that polls.

And the test gap generalises past this feature: a code path whose only tests
construct it in the configuration production never uses is a path with no tests.
The reload tests should be parameterised over pinned and unpinned, so whichever
option lands is exercised the way it will actually be deployed.
## Decision (2026-08-10): option A, pin a signing key

**Direction chosen: A.** Runtime policy change gets its authority from a pinned
signing key rather than a pinned artifact hash. Nothing about it is implemented
yet; what follows is what the decision commits us to, so the implementation issue
starts from a settled model rather than reopening the choice.

What it means concretely:

- The bundle manifest gains a signature over its own canonical contents. The
manifest already carries `author_identity` and `commit_sha`, both currently
unsigned assertions; signing turns them into claims someone is accountable for.
- The gateway pins a **public key** (a new setting, not `CMCP_POLICY_HASH`). A
reload verifies the new bundle's signature against it.
- The manifest's `version` must increase monotonically across reloads, and a
bundle whose version does not is refused. Without this a validly signed *older*
bundle is a downgrade attack: an attacker who can write the bundle directory
replays yesterday's more permissive policy, and every signature still checks out.
- `CMCP_POLICY_HASH` keeps its current meaning and stays the right choice for a
deployment that wants exactly one policy for the life of the process. It is a
pin on an artifact, so it remains incompatible with reload — the two are
alternatives, not layers, and configuring both is refused at startup (see below).
- Evidence: a TRACE claim records the bundle hash it evaluated under **plus** the
signer identity and the bundle version, so a verifier can answer both "what
policy ran" and "who authorised it" for a process whose policy changed mid-life.

Open sub-questions the implementation issue has to answer, none of which reopen
the direction:

- Where the pinned key comes from, and whether it can be rotated without a
restart. A key that can only change on restart is fine and is probably right,
since key rotation is rarer than policy change.
- Revocation. A signing key that is compromised needs a way to stop being trusted
that is faster than a fleet restart, or the model's advantage over hash pinning
shrinks.
- Whether the signature covers the bundle hash or the full canonical bundle. The
first is smaller and reuses `_canonical_bundle_hash`; the second is
self-contained.
- What happens to a session already admitted under the previous bundle, which is
listed under Not in scope below and now needs an answer.

## Two things that were not optional, and are now done

Both stood on their own, independent of the direction, and both have landed:

1. **A configuration that cannot work does not start.** `CMCP_POLICY_HASH`
together with `policy_reload_interval_seconds > 0` now aborts startup
(`POLICY_RELOAD_PINNED_HASH`). Under option A the two remain alternatives
rather than layers, so this refusal is durable rather than a stopgap: a pin on
an exact artifact and a policy that may change are contradictory whatever the
reload path is authorised by. Dev mode pins no hash and is unaffected, which is
the one configuration where reload actually works and is tested as such.
2. **The interval is honoured on failure.** `_last_reload_at` is stamped *before*
the attempt, so an exception cannot skip it. A failing reload now costs one
attempt per interval instead of one full bundle read plus hash per request.
Worth doing regardless of what happens to hot-reload, because the same shape
reappears in anything else that polls on the enforcement path.

The test gap generalised past this feature and was closed with it: a code path
whose only tests construct it in the configuration production never uses is a path
with no tests. The reload tests now cover pinned and unpinned side by side, one
asserting that a pinned hash cannot install a changed bundle and one asserting
that without the pin it can, so the difference the pin makes is visible in the
suite rather than discovered later. The load bound has a test above it that counts
the reads.

## Not in scope

Expand Down
17 changes: 15 additions & 2 deletions src/cmcp_runtime/policy/bundle.py
Original file line number Diff line number Diff line change
Expand Up @@ -211,19 +211,32 @@ def reload_if_stale(self) -> bool:
Returns True if a reload attempt was made (regardless of whether the
bundle hash changed). Thread-safe; uses an RLock so nested calls from
the same thread are safe.

This runs on the enforcement path, once per policy evaluation, so the
interval is a load bound and not only a freshness knob: the attempt is
timestamped whether it succeeded or failed. Advancing it only on success
meant a failing reload left the staleness check true, so every subsequent
tool call re-read every policy file and recomputed the bundle hash. A
reload that keeps failing must cost one attempt per interval, not one per
request.
"""
if self._reload_interval <= 0:
return False
with self._lock:
if time.monotonic() - self._last_reload_at < self._reload_interval:
return False
# Stamped before the attempt, so an exception cannot skip it.
self._last_reload_at = time.monotonic()
try:
new_bundle = load_policy_bundle(self._bundle_path, self._expected_hash)
if new_bundle.bundle_hash != self._bundle.bundle_hash:
self._bundle = new_bundle
logger.info("Policy bundle reloaded: hash=%s", new_bundle.bundle_hash)
self._last_reload_at = time.monotonic()
return True
except Exception as exc:
logger.warning("Policy bundle reload failed (keeping current): %s", exc)
logger.warning(
"Policy bundle reload failed (keeping current, retrying in %ds): %s",
self._reload_interval,
exc,
)
return False
30 changes: 30 additions & 0 deletions src/cmcp_runtime/startup.py
Original file line number Diff line number Diff line change
Expand Up @@ -345,6 +345,36 @@ def run_startup(config_path: str) -> RuntimeContext:
"Set CMCP_DEV_MODE=1 only in development to skip this check.",
)
sys.exit(1)

# POLICY-003: a pinned hash and automatic reload cannot both be satisfied.
# The pin says "the policy is exactly this artifact, decided before the
# process started"; reload says "the policy may change while it runs". The
# reload path re-validates against this same pinned hash, so a bundle that
# actually changed is rejected and the old policy keeps being enforced --
# silently, at WARNING, while the failed reload re-reads and re-hashes the
# whole bundle on every subsequent tool call because the interval is only
# advanced on success.
#
# That combination could never do what an operator setting it intends, so it
# is refused here rather than discovered in production. Runtime policy change
# needs a trust anchor that survives the bundle changing: a pinned signing
# key, not a pinned artifact hash. See docs/spec/policy-hot-reload.md.
if policy_expected_hash is not None and config.policy_reload_interval_seconds > 0:
_fatal(
"POLICY_RELOAD_PINNED_HASH",
"policy_reload_interval_seconds > 0 cannot be combined with a pinned "
"CMCP_POLICY_HASH: every reload is validated against that hash, so a "
"changed bundle is always rejected and the policy never updates.",
detail=(
f"policy_reload_interval_seconds={config.policy_reload_interval_seconds}, "
"CMCP_POLICY_HASH is set. Set policy_reload_interval_seconds to 0. "
"Automatic reload currently works only under CMCP_DEV_MODE=1, where no "
"hash is pinned; see docs/spec/policy-hot-reload.md."
),
action="startup_aborted",
)
sys.exit(1)

try:
policy_bundle = load_policy_bundle(config.policy_bundle_path, expected_hash=policy_expected_hash)
except PolicyHashMismatch as exc:
Expand Down
Loading