feat(baseline): tag the baseline so a rewrite is detectable - #64
Merged
Conversation
The baseline is what every drift comparison is made against, and it was an unsigned plain file. Anything able to write ~/.claude/agentrust could add a rogue skill to the approved set and the check would report "nothing added, nothing subtracted" from then on, indefinitely and quietly. The evidence shared a fate with the adversary, which is the failure this project argues against elsewhere. baseline.json now carries an HMAC-SHA256 tag over a canonical digest of its content. approve writes it, and hook and verify check it. A broken tag is reported ahead of the drift section, because if the baseline was altered then a reassuring "nothing changed" below it is worse than no result at all. HMAC rather than the existing Ed25519 signing key: verifying a signature would pull the crypto packages into the SessionStart hook, and the hook is deliberately standard-library only. The tag secret is separate, 32 random bytes, chmod 600. Three verdicts, not two. A baseline written before tagging existed, or one whose secret has been deleted, reports as untagged rather than broken. Neither is evidence of tampering, and raising a tamper alarm on a benign state teaches the user to dismiss the real one. On the limit, stated plainly in the code, the report, and the README rather than left for a reader to discover: the secret sits in the same directory as the baseline, so this stops someone who can write the file without reading the directory. It does not stop someone who owns the directory, because they can retag whatever they rewrite. Calling that tamper-proof would make it theatre. What does survive that adversary is off-box comparison, so approve now prints a baseline digest and verify prints the digest of the baseline it read. A silent re-baseline changes the digest even when the attacker can forge a valid tag. Anchoring the digest in an append-only log would automate the comparison and is the natural next step. 15 new tests: a rewritten baseline is detected, a forged tag is detected, a stripped tag and a missing secret read as untagged, the digest excludes the block carrying it, and the report states the limit alongside the claim. Suite: 41 passed, up from 26. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Imran Siddique <imran.siddique@opaque.co>
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
CodeQL flagged the stored tag secret as clear-text storage of sensitive information, and the flag was worth more than a suppression. Working through it: the only adversary an HMAC defeats here is one who can WRITE ~/.claude/agentrust without being able to READ it. On a developer machine that adversary barely exists, because anything that can write your home directory can read it and would simply retag. So the secret bought almost no coverage while adding a credential to leak, a file to manage, and a claim inviting a reader to assume more protection than exists. A bare SHA-256 digest over the baseline content gives the same real coverage with nothing to steal: corruption, truncation and a hand-edit that does not recompute it are all caught, and neither scheme catches an attacker who owns the directory. The control that does survive that attacker is unchanged and is where the security actually lives: approve prints the digest, verify prints the digest of the baseline it read, and a human who recorded the first sees a silent re-baseline. There is now a test that makes this limit executable rather than prose, asserting that a resealed rewrite passes the local check while its digest no longer matches what was approved. Removes the hmac and secrets imports and writes no credential at all; a test asserts the state directory contains only baseline.json. INTEGRITY_UNTAGGED is kept as an alias of INTEGRITY_UNSEALED so an older caller keeps working. Suite: 42 passed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Imran Siddique <imran.siddique@opaque.co>
2 tasks
imran-siddique
added a commit
that referenced
this pull request
Aug 1, 2026
…lision (#69) Two independent changes. Splitting them across PRs would have meant two CI rounds for work that touches the same test files, so they are separate commits here. ## Baseline sealing for the two engines that lacked it claude-code has sealed its baseline since #64. Codex and scheduled-agents did not, so anything able to write their state could add a component to the APPROVED set and the check would report "nothing added, nothing subtracted" from then on. That gap mattered most for scheduled-agents, whose whole subject is the things that run when nobody is watching. Both now seal on write via core.save_baseline and check on read. A broken seal is reported AHEAD of any drift result, in the hook message and in verify, because if the baseline was altered then the comparison against it means nothing and a reassuring "no changes" underneath would be worse than no result at all. Both commands print the baseline digest with the instruction to record it off the machine, which is the only part that survives an attacker who owns the state directory. Sealing writes without checking on read would have been decorative, so there are tests for the check, not just the write: an edited baseline is detected, verify states integrity before drift, and the scheduled-agents hook warns instead of reporting no drift when the baseline was tampered with. That last test plants a PreToolUse hook command in the baseline, which is the realistic attack on that surface. ## Test module and file collisions Four engines each define a module named `capture`, and three of their suites were named test_capture.py. Two separate problems, both fixed. sys.path insertion meant `import capture` resolved to whichever suite pytest collected first, so a root-level run tested each suite against another engine's code. Each suite now loads its engine by path under a unique module name. The shared test_capture.py basename then still broke collection, since pytest could not derive distinct module names. Renamed to test_claude_code_capture.py, test_codex_capture.py and test_scheduled_capture.py, matching the copilot suite. A single root-level pytest run now works for the first time: 184 passed. Per-suite runs are unchanged, which is how CI invokes them. Suites: claude-code 59, codex 29, scheduled-agents 26, copilot 25, core 45. The one codex failure is the pre-existing TRACE conformance check that fix/codex-trace-pins addresses. Signed-off-by: Imran Siddique <imran.siddique@opaque.co> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The gap
The baseline is what every drift comparison is made against, and it was an unsigned plain file next to its own signing key. Anything able to write
~/.claude/agentrustcould add a rogue skill to the approved set and the check would report "nothing added, nothing subtracted" from then on, quietly and indefinitely.That is the evidence sharing a fate with the adversary, which is the failure this project argues against elsewhere.
What changed
baseline.jsonnow carries an HMAC-SHA256 tag over a canonical digest of its content.approvewrites it;hookandverifycheck it.Demonstrated end to end, adding a skill to the approved set after approval:
Reported before the drift section, on purpose. If the baseline was altered, a reassuring "nothing changed" underneath it is worse than no result at all. There is a test asserting the ordering.
HMAC rather than the existing Ed25519 key. Verifying a signature would pull the crypto packages into the SessionStart hook, and the hook is deliberately standard-library only. The tag secret is separate, 32 random bytes,
chmod 600.Three verdicts, not two. A baseline written before tagging existed, or one whose secret was deleted, reports as
untaggedrather thanbroken. Neither is evidence of tampering, and a tamper alarm on a benign state teaches the user to dismiss the real one.The limit, stated rather than buried
The secret lives in the same directory as the baseline. So this catches corruption, a hand-edit, and anyone who can write the file without reading that directory. It does not catch someone who owns the directory, because they can retag whatever they rewrote. Calling it tamper-proof would make it theatre, so the code, the report output, and the README all say so.
What does survive that adversary is off-box comparison.
approvenow prints a baseline digest andverifyprints the digest of the baseline it read, so a silent re-baseline is visible to a human who wrote the first one down, even against an attacker who can forge a valid tag. In the demo above the digest moved from6a50c8aftoa0161d69.Anchoring that digest in an append-only log would automate the comparison. That is the natural next step and deliberately not in this PR, since it introduces a network dependency and a service decision.
Test plan
claude-code/tests: 41 passed, up from 26. 15 new.untagged, notbroken.Relationship to #63
Independent. #63 touches
_skills/snapshot/diff; this touches_save/_load/cmd_approve/cmd_verifyand adds a section torender_report. Both add test classes toclaude-code/tests/test_capture.py, so expect a one-place conflict there whichever lands second, resolved by keeping both sides. I have done that resolution twice already for #63 against main.