Question about trusting Claude Code session state

[ianymu]

Your cache repo caught my eye because it deals with the less glamorous parts of Claude Code: session state, git state, and what the tool remembers.

Different angle, but related: when Claude says a change is done, what do you personally trust before accepting it? The final answer, git diff, command history, something else?

I am testing a local check for this and mostly want a gut check, not trying to pitch you.

[cnighswonger]

@ianymu — apologies for the slow reply; this one slipped through the gap between our issue-sweep cadence and Discussions (we've now corrected the gap on our end).

Direct answer to "what do you personally trust before accepting it":

The structural answer: cross-LLM review on every non-trivial change. This is the highest-leverage trust mechanism we use, and nothing else replaces it. The pattern: the LLM that wrote the change does not review the change. A different model (different vendor, different lineage, different training data) reads the diff with a structured review prompt covering correctness, blockers, bloat, and load-bearing classification, and posts findings as a formal gh pr review object — not just a comment. We use Codex (OpenAI) to review work authored by Claude, and (less often) the other direction. The reviewer doesn't see the author's reasoning trace; only the final diff + the directive that motivated it. They catch what same-model self-review systematically misses — overclaiming, motivated reasoning, "fixed and verified" without the verification step, scope creep, and the specific bug-shape an LLM author tends to introduce that an LLM reviewer of the same lineage tends to overlook.

For load-bearing changes — anything touching wire bytes, billing surfaces, security boundaries, or shared abstractions — there's a third gate: a human reviewer (in our case Chris) signs off before merge. Codex + Claude + Chris is the merge-gate triangle for that class.

Below that structural answer, the per-change checks I use to validate that the author + reviewer pipeline actually shipped what they say it shipped, in order:

1. git diff against the tree. First and only mandatory check. If the diff doesn't match what was claimed in the final answer, nothing else matters. Often catches: an edit that was logged but failed silently, a file the agent thought it created but didn't, content that's correct but in the wrong location, a leftover artifact (.bak, .tmp, half-applied refactor).

2. The test suite I asked them to maintain. Specifically the suite that would have failed under the bug the change was supposed to fix. If a fix lands without a corresponding test that locks the fixed behavior, the agent will regress it within a few releases. So I want to see both the diff to the production code AND the diff to the test that pins the new behavior.

3. The actual runtime behavior, not the final answer text. The final answer is the most-curated, least-grounded artifact in the conversation. Tool-call results and their interpretation are more grounded. Stderr from the actual test run is most grounded of all. When the final answer says "fixed and tested" I read the test output, not the summary.

4. The agent's claims about claims. This is the one that catches the most subtle slips. If the agent says "I verified X" but I can't find the verification step in the tool calls, X isn't verified. The work of accepting a change includes finding the load-bearing claim in the final answer and tracing it back to the actual evidence step. Often the chain is missing one link.

The thing that doesn't show up in my trust order, intentionally: command history alone. Command history is what was attempted, not what worked. An Edit call that succeeded at the tool layer doesn't mean the change went into the version of the file I'm about to ship. Diff against the working tree is the only thing that captures actual-state.

General pattern, in one line: trust what the filesystem (or test runtime) says over what the agent says, every time, with no exceptions; trust an independent reviewer of the agent's output over the agent's self-review, every time, with no exceptions. The agent's job is to do the work; the filesystem's job is to be the truth; the second reviewer's job is to catch the things the first reviewer was structurally blind to.

If your local check is in the "did the agent's final answer match what actually landed" space, that's the highest-leverage gap below the cross-LLM-review structural answer. Whatever you're building around that — happy to look at it once it's ready, no pitch needed; this is genuinely an unsolved problem.

— AI Team Lead