docs(policy): hot-reload is implemented and inert in production

[imran-siddique]

Design doc only, no code, per the "design first" call on this item.

The plan item was based on a drifted status file

The item read "cMCP live policy reload — policy_reload_interval_seconds is 0, design first", which reads as "unimplemented, default off". STATUS.md said the same.

Hot-reload is already implemented: PolicyStore.reload_if_stale polls, swaps atomically under an RLock, and keeps the current bundle on failure. PolicyEvaluator.evaluate calls it. It is documented as a supported config knob.

It cannot swap a policy in any production configuration.

Why

Production requires CMCP_POLICY_HASH (POLICY-001/#137). That pinned hash is handed to PolicyStore and re-used on every reload:

new_bundle = load_policy_bundle(self._bundle_path, self._expected_hash)

So the reload asks a question that answers itself: has the bundle changed, and does it still hash to the value it had before it changed? A changed bundle always raises PolicyHashMismatch. The line that would install it —

if new_bundle.bundle_hash != self._bundle.bundle_hash:

— is unreachable whenever a hash is pinned.

Measured, not inferred

An operator edit from allow-all to deny-all, with the pin set:

startup bundle hash : sha256:fdb2e39e...
new on-disk hash    : sha256:e1426d71...
reload_if_stale()   : False
active bundle hash  : sha256:fdb2e39e...   <-- unchanged

The gateway keeps enforcing allow-all. The only signal is a WARNING whose text is gateway will not start — a startup message, emitted during steady-state operation.

Second defect: it becomes a load amplifier

_last_reload_at only advances on success. The failure path returns without touching it, so the staleness check stays true. Since the reload always fails in production, the interval stops being an interval.

Measured: 50 policy evaluations after the first interval elapsed → 50 full bundle re-reads from disk, each reading every policy file and computing a SHA-256, on the enforcement hot path.

The production effect of policy_reload_interval_seconds: 60 is therefore not "policy updates every minute". It is "after one minute, every tool call performs full bundle I/O and hashing, and the policy never changes" — reached by following the documented configuration.

Why no test caught it

Every reload test constructs PolicyStore without expected_hash — the dev-mode configuration, where reload genuinely works. test_policy_store_bundle_swap_on_hash_change passes and correctly proves the swap logic, in the one configuration production never uses.

What the doc contains

The real design question, which a fix has to answer before any code: if not a hash fixed at startup, what authorises a new bundle? Plus a second question specific to cMCP: a TRACE claim names the bundle hash a call was evaluated under, so what does a mid-process reload do to evidence?

Five options with their costs: pin a signing key (A), pin a set of hashes (B), re-read the pin from a trusted source (C), operator-triggered reload (D), or keep it dev-only and make that a startup config error (E).

Two fixes that stand regardless of direction: a configuration that cannot work must not start, and the interval must be honoured on failure.

Also corrected

STATUS.md, docs/configuration.md and the README all described the knob as merely disabled. They now say it is a trap and to leave it at 0.

Not addressed

CMCP_CATALOG_HASH has the identical pin and load_catalog the identical shape. Flagged in the doc as needing a deliberate decision rather than a copy of whatever lands here.

🤖 Generated with Claude Code