v1.91.156.0 fix(evals): office-hours marathon fixture loads the formatter's imports; validator checks the observed cap fixes (#3246)

[garrytan]

Closes #3246.

The weekly marathon's office-hours-section-loading case has been red since #3164 (Oct 13), because of two test defects. Neither is a product defect.

1. The fixture didn't copy every module the formatter imports. The test copied bin/gstack-office-hours-review from a hand-kept list (lib/office-hours-review.ts, lib/fs-atomic.ts). #3164 added lib/spec-review-cap.ts to the formatter's imports, so in marathon run 39032308051 every prepare/finalize failed with Cannot find module '../lib/spec-review-cap.ts'. The agent correctly declared the review UNREVIEWED and saved the failure to a file, and the validator then failed with "unreviewed report lacks a saved failure reason". test/helpers/office-hours-formatter-fixture.ts now copies the formatter together with its relative-import closure. A free test in test/office-hours-review.test.ts runs the copied CLI and fails on any missing module; the old three-file list reproduces the exact marathon error. The case's touchfiles gain lib/spec-review-cap.ts and the helper.

2. The validator never received the --fixes list. Once the module was restored, a focused paid run reached the cap path. The agent applied the two remaining blocking fixes and ran finalize --fixes, and the formatter recorded "issues verified: 2 of 2". But the marathon test never passed fixes to validateOfficeHoursReviewArtifacts (the field #3164 added), so the validator recomputed "2 fixes unconfirmed" and failed with "Spec Review does not match the complete saved evidence and computed metrics". observedSpecFixes() returns the last valid fixes list the agent saved with Write, and the marathon test passes it in. A free test in test/office-hours-completion.test.ts drives the real finalize --fixes at the cap and checks three cases: the observed list validates, no list reproduces the marathon mismatch, and a list naming a different finding still fails.

Gate integrity. Gate edits: the marathon test's fixture copy and the evidence it hands the existing validator. No assertion, threshold, trial count, case kind, judge or budget changed. Negative controls are in both free tests: the missing-module error, the missing-fixes mismatch, and the wrong-finding mismatch.

Validation.

• Focused paid runs of office-hours-section-loading (marathon, Opus 5.5), one trial each:
  • after repair 1: failed on defect 2;
  • rerun: failed on defect 2 again;
  • after repair 2: PASS in 430 s through the --fixes cap path ($1.90).
• The first focused run also showed a separate model slip: the round-2 reviewer subagent used Edit after a seal error, although its prompt says "Do not use Edit". It did not recur in the next two runs, so it is logged as flaky (CREW rule 16).
• Free: office-hours, touchfiles, module-size and spawnSync-tripwire tests (325 pass); typecheck and typecheck:test clean; remote pregate on Ubicloud: 268 files, 9040 tests, 0 failed.

[trunk-io]

😎 This pull request was merged.

[capy-ai]

flaky: office-hours-section-loading, failed in local focused run capy-census-red-office-hours-section-loading-1792435886 (round-2 reviewer subagent used Edit on round-2.json after a seal error, despite the prompt's "Do not use Edit"; validator: "round 2 artifact differs from its observed Write"), passed on rerun capy-census-red-ohsl-repair2-1792437571, logged for the daily flaky-fix pass

[github-actions]

E2E Evals: ✅ PASS (reused: main moved, R2)

No paid case ran on ebc2c96b0ac6f84f2636c19a1ce3dae4b2c1d7e7. It carries the GREEN verdict of run 39314917806 at 6eeeea492e86a4bb997aa6f76df2ba27157a1bac, because main moved (afac8df -> 0c87a55) and the PR's own diff from its merge-base, excluding stamp files, is byte-identical to 6eeeea4's (sha256 29ae77628943).

OWN DIFF: identical sha256=29ae776289434fe8474f281f4106fc44a189c18f040e963d9012ce840a404e19 (merge-base afac8dfd873a7c9c8d8b14dac8ebc30dfcc50e2a -> 0c87a553c75c42c3c112eb66f203d6785ef2b8a6)
base: afac8dfd873a7c9c8d8b14dac8ebc30dfcc50e2a -> 0c87a553c75c42c3c112eb66f203d6785ef2b8a6
selection: d44f8e46a0c5f5b09372ac070739036cc5270ff4507306f10dd9fdbd0f9f37db (identical to the reused gate)

---

Verdict reuse (R2a restamp-only, R2 main moved): a changed own diff, a non-stamp byte, a changed selection, a newer red or unbound gate, or a missing binding or receipt runs the full gate; the evals-fresh label opts out. Free Tests and every other check still ran on this head.