What happened
On PR #1823, the fullsend-ai-review agent's third review run found a HIGH-severity issue: the new _statement_subject_matches_image rule at policy/release/lib/attestations.rego:187-190 references input.image.digest, but this field does not appear in the acceptance sample (acceptance/samples/policy-input-golden-container.json), is not used anywhere else in the codebase (0 grep matches), and diverges from the established pattern image.parse(input.image.ref).digest. The tests mock the field with with input.image.digest as "sha256:abc123", concealing the potential runtime issue. The agent DISMISSED its prior approval due to this finding, but the PR was merged after a human approval ("Lgtm") ~15 hours later without the finding being addressed.
Critically, the same code was present in all three review runs (introduced in commit 81c889b), but the HIGH finding was only surfaced on the third pass — the first two runs APPROVED without catching it. This demonstrates that field-access contract violations are not reliably detected by the review agent without explicit guidance.
What could go better
The review agent eventually caught this issue, but only on its 3rd attempt reviewing the same code. The inconsistency stems from a lack of documented knowledge about what fields are available on input.image — the agent had to independently discover the discrepancy by grepping the codebase and checking the acceptance sample, which it only did in one of three runs.
If AGENTS.md documented the input.image object shape and the standard patterns for accessing image metadata (especially image.parse(input.image.ref).digest for digest comparison), the review agent could reliably flag deviations on the first pass. This would also help human reviewers, who may not have noticed the field-access divergence.
Confidence: Moderate. I am confident the documentation would improve review consistency for this class of issue. I am less certain whether input.image.digest is actually absent at runtime — the EC CLI might populate it even though it's not in the acceptance sample. The team should verify this independently.
Proposed change
Add an "Input Contract" or "Runtime Input Shape" subsection to AGENTS.md documenting the input.image object. Specifically:
- Available fields:
input.image provides config, parent, ref, signatures, source (as seen in the acceptance sample at acceptance/samples/policy-input-golden-container.json).
- Image digest access pattern: To get the image digest, use
image.parse(input.image.ref).digest (the image.parse helper in policy/lib/image/). Do not access input.image.digest directly — this field is not part of the established contract.
- Reference file: The acceptance sample at
acceptance/samples/policy-input-golden-container.json is the canonical reference for input.image shape.
This complements existing issue #1820 (which documents test attestation predicate fields) by covering the image input contract — a different part of the evaluation input.
Validation criteria
On the next 3 PRs that add or modify rules accessing input.image fields in policy/release/ or policy/lib/, the review agent should flag any direct access to undocumented input.image fields (i.e., fields not listed in the AGENTS.md contract) on the first review run, not requiring multiple passes to detect the pattern. If the agent references the AGENTS.md guidance when flagging the issue, the documentation is working as intended.
Generated by retro agent from #1823
What happened
On PR #1823, the fullsend-ai-review agent's third review run found a HIGH-severity issue: the new
_statement_subject_matches_imagerule atpolicy/release/lib/attestations.rego:187-190referencesinput.image.digest, but this field does not appear in the acceptance sample (acceptance/samples/policy-input-golden-container.json), is not used anywhere else in the codebase (0 grep matches), and diverges from the established patternimage.parse(input.image.ref).digest. The tests mock the field withwith input.image.digest as "sha256:abc123", concealing the potential runtime issue. The agent DISMISSED its prior approval due to this finding, but the PR was merged after a human approval ("Lgtm") ~15 hours later without the finding being addressed.Critically, the same code was present in all three review runs (introduced in commit
81c889b), but the HIGH finding was only surfaced on the third pass — the first two runs APPROVED without catching it. This demonstrates that field-access contract violations are not reliably detected by the review agent without explicit guidance.What could go better
The review agent eventually caught this issue, but only on its 3rd attempt reviewing the same code. The inconsistency stems from a lack of documented knowledge about what fields are available on
input.image— the agent had to independently discover the discrepancy by grepping the codebase and checking the acceptance sample, which it only did in one of three runs.If AGENTS.md documented the
input.imageobject shape and the standard patterns for accessing image metadata (especiallyimage.parse(input.image.ref).digestfor digest comparison), the review agent could reliably flag deviations on the first pass. This would also help human reviewers, who may not have noticed the field-access divergence.Confidence: Moderate. I am confident the documentation would improve review consistency for this class of issue. I am less certain whether
input.image.digestis actually absent at runtime — the EC CLI might populate it even though it's not in the acceptance sample. The team should verify this independently.Proposed change
Add an "Input Contract" or "Runtime Input Shape" subsection to AGENTS.md documenting the
input.imageobject. Specifically:input.imageprovidesconfig,parent,ref,signatures,source(as seen in the acceptance sample atacceptance/samples/policy-input-golden-container.json).image.parse(input.image.ref).digest(theimage.parsehelper inpolicy/lib/image/). Do not accessinput.image.digestdirectly — this field is not part of the established contract.acceptance/samples/policy-input-golden-container.jsonis the canonical reference forinput.imageshape.This complements existing issue #1820 (which documents test attestation predicate fields) by covering the image input contract — a different part of the evaluation input.
Validation criteria
On the next 3 PRs that add or modify rules accessing
input.imagefields inpolicy/release/orpolicy/lib/, the review agent should flag any direct access to undocumentedinput.imagefields (i.e., fields not listed in the AGENTS.md contract) on the first review run, not requiring multiple passes to detect the pattern. If the agent references the AGENTS.md guidance when flagging the issue, the documentation is working as intended.Generated by retro agent from #1823