Have you seen generated tests increase confidence without testing the real behavior?

[Keesan12]

One of the stranger failure modes we kept seeing with coding agents was this:

confidence goes up faster than truth.

The run looks safer because there are more tests, but the tests are often generated around the same assumptions as the code itself.

That can create a really misleading state where:

• the transcript says tests passed
• the code looks "covered"
• but nobody actually learned whether the important behavior is correct

In practice, this showed up for us as fake reassurance more than obvious failure.

That is a big reason we built MartinLoop around verifier gates, receipts, and stop reasons instead of just asking whether a run finished.

Curious how other people are handling this:

Have you seen generated tests increase confidence without really testing the real behavior?

If so, what actually helps?

• holdout tests
• manual checks
• independent verifier prompts
• limiting what counts as proof
• forcing the run to explain what changed and what risk remains

Would love to compare notes with anyone running longer agent loops or shipping code from autonomous workflows.