feat: seal codex and scheduled-agents baselines, and fix the test collision - #69
Merged
Merged
Conversation
…lision Two independent changes. Splitting them across PRs would have meant two CI rounds for work that touches the same test files, so they are separate commits here. ## Baseline sealing for the two engines that lacked it claude-code has sealed its baseline since #64. Codex and scheduled-agents did not, so anything able to write their state could add a component to the APPROVED set and the check would report "nothing added, nothing subtracted" from then on. That gap mattered most for scheduled-agents, whose whole subject is the things that run when nobody is watching. Both now seal on write via core.save_baseline and check on read. A broken seal is reported AHEAD of any drift result, in the hook message and in verify, because if the baseline was altered then the comparison against it means nothing and a reassuring "no changes" underneath would be worse than no result at all. Both commands print the baseline digest with the instruction to record it off the machine, which is the only part that survives an attacker who owns the state directory. Sealing writes without checking on read would have been decorative, so there are tests for the check, not just the write: an edited baseline is detected, verify states integrity before drift, and the scheduled-agents hook warns instead of reporting no drift when the baseline was tampered with. That last test plants a PreToolUse hook command in the baseline, which is the realistic attack on that surface. ## Test module and file collisions Four engines each define a module named `capture`, and three of their suites were named test_capture.py. Two separate problems, both fixed. sys.path insertion meant `import capture` resolved to whichever suite pytest collected first, so a root-level run tested each suite against another engine's code. Each suite now loads its engine by path under a unique module name. The shared test_capture.py basename then still broke collection, since pytest could not derive distinct module names. Renamed to test_claude_code_capture.py, test_codex_capture.py and test_scheduled_capture.py, matching the copilot suite. A single root-level pytest run now works for the first time: 184 passed. Per-suite runs are unchanged, which is how CI invokes them. Suites: claude-code 59, codex 29, scheduled-agents 26, copilot 25, core 45. The one codex failure is the pre-existing TRACE conformance check that fix/codex-trace-pins addresses. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Imran Siddique <imran.siddique@opaque.co>
2 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two independent changes, one per commit. Splitting them across PRs would have meant two CI rounds for work that touches the same test files, so they are here together and reviewable separately.
1. Baseline sealing for the two engines that lacked it
claude-code has sealed its baseline since #64. Codex and scheduled-agents did not, so anything able to write their state could add a component to the approved set and the check would report "nothing added, nothing subtracted" from then on, indefinitely and quietly.
That gap mattered most for scheduled-agents, whose entire subject is the things that run when nobody is watching.
Both now seal on write via
core.save_baselineand check on read. A broken seal is reported ahead of any drift result, in both the hook message andverify: if the baseline was altered then the comparison against it means nothing, and a reassuring "no changes" underneath would be worse than no result at all.Both
approvecommands now print the baseline digest with instructions to record it off the machine. That is the only part of this that survives an attacker who owns the state directory, since anyone who can write the file can also reseal it.Sealing writes without checking reads would have been decorative, so the tests cover the check rather than the wiring: an edited baseline is detected,
verifystates integrity before drift, and the scheduled-agents hook warns instead of reporting no drift when the baseline was tampered with. That last test plants aPreToolUsehook command into the baseline, which is the realistic attack on that surface.2. Two test collisions, one root cause each
Four engines each define a module named
capture, and three of their suites were namedtest_capture.py.Module name.
sys.pathinsertion meantimport captureresolved to whichever suite pytest collected first, so a root-level run tested each suite against another engine's code. Each suite now loads its engine by path under a unique module name.File basename. With that fixed, collection still failed because pytest could not derive distinct module names from three identical filenames. Renamed to
test_claude_code_capture.py,test_codex_capture.pyandtest_scheduled_capture.py, matching the copilot suite.A single root-level
pytestnow works for the first time in this repo: 184 passed. Per-suite runs are unchanged, which is how CI invokes them.Test plan
ruff check --target-version py39clean across every path this PR touches. The 21 errors a repo-wide run reports are all insentinel,comply54anddecisionassure, untouched here.The one codex failure is the pre-existing TRACE conformance check that
fix/codex-trace-pinsaddresses.What is left after this
TRACE emission for the Copilot check, so it earns an
integration.yamlhonestly, and publishingagentrust-capture-coreto PyPI. The second needs credentials and is yours; nothing breaks until it happens, because every engine carries the vendored fallback.