feat(github): cache bare PR reads and reviews, mirroring the head-SHA sync-state pattern

[JSONbored]

Summary

Closes #2537.

GET /pulls/{n} (PR state/mergeable_state) and GET /pulls/{n}/reviews were the largest and third-largest GitHub REST contributors observed in a live rate-limit trough — larger than the PR-files route the earlier fix (#2527) addressed, and deliberately left uncached by that fix. The bare-PR-state read was implemented as four near-identical helper functions (fetchLivePullRequestMergeState, fetchLivePullRequestState, fetchLivePullRequestHeadSha, fetchLivePullRequest) with no caching or cross-call coalescing, called from several independent read sites (readiness checks, the maintenance planner, duplicate-sibling reconciliation, a gate-override command). Reviews were explicitly excluded from #2527 ("reviews are more volatile than files" — true at a fixed head, but reviews only actually change on a pull_request_review webhook, not on every sweep tick).

What changed

• Schema: extends pull_request_detail_sync_state (migration 0093) with prMergeableState/prState/prStateFetchedAt, and reuses the already-present (previously-unused) reviewsSyncedAt column as the review cache's freshness marker — no new table.
• PR-state cache (src/github/backfill.ts): new cachedFetchLivePullRequestMergeState / cachedFetchLivePullRequestState / cachedFetchLivePullRequestHeadSha, each a read-through wrapper over the durable row with a 5-minute safety-net TTL (PR_STATE_CACHE_MAX_AGE_MS) so a missed webhook self-heals within one sweep tick. primeDurablePrStateCache lets an already-paid-for full-payload fetch (the sweep resync) prime the cache for other readers instead of them re-fetching moments later. Wired into every read-only call site (readiness, duplicate-sibling reconciliation, the gate-override command).
• Reviews cache (fetchAndStorePullRequestDetails): gated purely on reviewsSyncedAt being set — no head-SHA comparison, since reviews are independent of the head (unlike files). Only stamps reviewsSyncedAt when the review fetch actually succeeded that pass (a failed sync must not be cached as if it succeeded).
• Webhook invalidation (src/queue/processors.ts): invalidatePrStateCache on pull_request synchronize/closed/reopened; invalidatePrReviewsCache on pull_request_review submitted/edited/dismissed (already the exact 3 actions shouldProcessPullRequestPublicSurface restricts that event to). Both explicitly null the relevant columns (PARTIAL-UPDATE CONTRACT: omitted = unchanged, explicit null = cleared).
• Metrics: gittensory_pr_state_cache_total{field,result} and gittensory_pr_reviews_cache_total{result}, mirroring the existing gittensory_github_pull_request_files_fetch_total bounded-label style.
• Act-boundary preserved, unchanged: the pre-merge verdict thread (runAgentMaintenancePlanAndExecute's disposition input) and its unified-comment mirror — both already documented against the feat(mcp): add gittensory_explain_gate_disposition tool (#2234) #4220 contradiction — continue to call the raw, uncached fetchLivePullRequestMergeState via refreshLiveMergeState, which is untouched by this diff. They must never be routed through the new cache: a base-conflicting PR reading a stale clean here while the disposition reads live dirty is exactly the bug feat(mcp): add gittensory_explain_gate_disposition tool (#2234) #4220 fixed.

Correctness verification

I adversarially verified the act-boundary claim with actual mutation testing, not just a read of the code: temporarily rerouted refreshLiveMergeState through the new cache, confirmed a pre-existing, unrelated test (#audit-rate-headroom: the per-PR re-review refreshes merge state and CI...) failed as expected, then confirmed the two purpose-built #4220 regression tests in this diff did not originally catch it — they were confounded by other same-request cache-priming/invalidation side effects that happened to clear the stale seed before the disposition ran, for unrelated reasons. Fixed both tests to no-op those confounding calls and assert directly (via pass-through spies) that the act-boundary's own read goes through the raw fetcher and never the cached one — then re-ran the same mutation to confirm both now fail correctly. Reverted the mutation; both pass cleanly on the real code.

Also added dedicated tests for primeDurablePrStateCache (previously exercised only indirectly, with no assertions on its own output), covering its early-exit, nullish-fallback, and PARTIAL-UPDATE CONTRACT branches.

Known limitation

invalidatePrStateCache fires on synchronize/closed/reopened only (matching the issue's own scope), not on an edited action that retargets a PR's base branch — a rare case where GitHub could recompute mergeable_state for the new base without one of those three actions firing. Bounded by the 5-minute safety-net TTL; not a correctness issue for the act-boundary path (which never reads this cache). Left as a documented gap rather than expanding scope.

Tests

• test/unit/pr-detail-durable-cache.test.ts (new): cache miss/hit/expiry for all three PR-state fields, fail-open on a write hiccup, invalidatePrStateCache, the write-through status-preservation regression, and primeDurablePrStateCache's three branches.
• test/unit/backfill-file-hydration-scoping.test.ts: cold-cache fetch, nullish-state fallback, head-independent reuse (contrasted with the files cache's head-scoped behavior), and invalidated-cache refetch for reviews.
• test/unit/queue.test.ts: webhook invalidation for all 3 PR-state actions and all 3 review actions (it.each), non-invalidating actions left alone, sibling-event non-invalidation, and the two mutation-tested #4220 regression tests described above.

Validation

• npm run typecheck
• npx vitest run test/unit/pr-detail-durable-cache.test.ts test/unit/backfill-file-hydration-scoping.test.ts test/unit/queue.test.ts
• npm run test:changed
• npm run test:coverage (unsharded, full suite — 100% line and branch coverage on every changed line in src/** per coverage/lcov.info, cross-checked programmatically against the diff)
• npm run test:workers
• npm run db:migrations:check
• npm run test:ci (the full local gate, exit 0)
• npm audit --audit-level=moderate
• git diff --check
• Manual mutation testing of the act-boundary regression tests (see Correctness verification above)

Scope

• Change is narrow and limited to the stated problem
• No secrets, wallets, hotkeys, trust scores, or reward values added anywhere
• No edits to site/, CNAME, **/lovable/**, or CHANGELOG.md

Safety

• No auth/session/CORS surface touched
• The act-boundary merge/close decision is unchanged and verified via mutation testing to still force-refetch live, uncached

[loopover-orb]

Warning

🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨

⏸️ Gittensory review result - manual review recommended

_{Review updated: 2026-07-02 22:06:33 UTC}

12 files · 1 AI reviewer · 1 blocker · readiness 98/100 · CI green · blocked

⏸️ Suggested Action - Manual Review

• AI reviewers agree on a likely critical defect: src/github/backfill.ts:713 and src/github/backfill.ts:2017 refresh `reviewsSyncedAt` on a review-cache hit because `reviewSyncedThisPass` only checks for a warning, so a skipped review fetch extends the TTL without observing GitHub and a missed invalidation can pin stale review data indefinitely. — Resolve the flagged defect, or override if the AI reviewers are mistaken, then re-run the gate.

Review summary
This change adds durable caching for bare PR state and PR reviews, with schema/migration parity and broad regression coverage around PR-state invalidation, partial-update semantics, and repair of existing review markers. The PR-state side is careful about stale-vs-live act boundaries, but the review-cache freshness safety net is undermined because cache-hit syncs refresh `reviewsSyncedAt` even when no reviews were fetched. That makes a missed review-cache invalidation stay fresh forever under normal repeated sync traffic, which is exactly the failure mode the TTL was added to bound.

Blockers

• src/github/backfill.ts:713 and src/github/backfill.ts:2017 refresh `reviewsSyncedAt` on a review-cache hit because `reviewSyncedThisPass` only checks for a warning, so a skipped review fetch extends the TTL without observing GitHub and a missed invalidation can pin stale review data indefinitely.

Nits — 5 non-blocking

• nit: src/github/backfill.ts:2011 reuses `options.forceFiles` to force review-cache misses too, which works but makes the option name misleading now that it controls both files and reviews.
• In src/github/backfill.ts, make `fetchAndStorePullRequestDetails` report whether reviews were actually fetched successfully, and only set `reviewsSyncedAt` in the outer sync-state upserts when that is true rather than when there was merely no warning.
• Add a regression test in test/unit/backfill-file-hydration-scoping.test.ts where a fresh `reviewsSyncedAt` cache hit is followed by repeated `refreshPullRequestDetails` calls for more than 24 hours; the review route should be called once the original marker ages out, not kept alive by cache-hit stamps.
• Rename or split `forceFiles` in src/github/backfill.ts if the intended contract is now “force detail refresh,” so future call sites do not accidentally force or skip review refreshes based on a file-only name.
• PR author also opened the linked issue — Link an issue that was opened by a different contributor, or provide a rationale for why this self-authored issue represents genuine discovery work.

Concerns raised — review before merging

• src/github/backfill.ts:713 and src/github/backfill.ts:2017 refresh `reviewsSyncedAt` on a review-cache hit because `reviewSyncedThisPass` only checks for a warning, so a skipped review fetch extends the TTL without observing GitHub and a missed invalidation can pin stale review data indefinitely.

Signal	Result	Evidence
Code review	❌ 1 blocker	1 reviewer
Linked issue	✅ Linked	#2537
Related work	✅ No active overlap found	No same-issue or scoped active PR overlap found.
Change scope	✅ 20/20	Low review scope from cached public metadata (size label size:L; 1 linked issue).
Validation posture	✅ 25/25	PR body includes validation/test evidence.
Contributor workload	✅ 10/10	Author activity: 64 registered-repo PR(s), 55 merged, 522 issue(s).
Contributor context	✅ Confirmed Gittensor contributor	JSONbored; Gittensor profile; 64 PR(s), 522 issue(s).
Gate result	❌ Blocking	Repo-configured hard blocker found.

Review context

• Author: JSONbored
• Role context: owner (maintainer lane)
• Public audience mode: oss maintainer
• Lane context: Repository registration is not available in the local Gittensory cache.
• Public profile languages: not available
• Official Gittensor activity: 64 PR(s), 522 issue(s).
• PR-specific overlap: none found.

Contributor next steps

• Treat this as maintainer-lane context rather than normal contributor-lane activity.
• No action.

Signal definitions

• Related work = same linked issue, overlapping active PRs, or title/path similarity.
• Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
• Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
• Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
• Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.

_{🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed}

---

💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.

• Re-run Gittensory review

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 96.08%. Comparing base (bcc19ea) to head (dfbbbbb).
✅ All tests successful. No failed tests found.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2595      +/-   ##
==========================================
+ Coverage   96.05%   96.08%   +0.03%     
==========================================
  Files         234      234              
  Lines       26280    26357      +77     
  Branches     9531     9564      +33     
==========================================
+ Hits        25244    25326      +82     
+ Misses        425      424       -1     
+ Partials      611      607       -4     

Files with missing lines	Coverage Δ
src/db/repositories.ts	96.54% <ø> (ø)
src/db/schema.ts	69.46% <ø> (ø)
src/github/backfill.ts	96.98% <100.00%> (+0.21%)	⬆️
src/queue/processors.ts	92.50% <100.00%> (+0.22%)	⬆️

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[JSONbored]

Fixed the CI failure: migrations/0093_pull_request_detail_sync_pr_state.sql collided with migrations/0093_gate_require_fresh_rebase_window.sql (added by #2616, merged after this branch was cut). Rebased onto current main and renumbered this migration to 0094 — npm run db:migrations:check now reports a clean, contiguous 0001..0094 with no new duplicates. Full local gate (test:ci, npm audit) green; no other changes.

[JSONbored]

Fixed both flagged blockers:

1. Cross-field cache-freshness bug (real defect). cachedFetchLivePullRequestMergeState / cachedFetchLivePullRequestState / cachedFetchLivePullRequestHeadSha each wrote through only the one field they cared about, but all three share a single prStateFetchedAt stamp — so a live fetch from any one of them made the OTHER two fields look "fresh" to a later reader despite never having been fetched, silently returning undefined for a field that was simply never populated. Since all three narrow fetchers already hit the exact same GET /pulls/{n} endpoint (no extra API cost), a new fetchAndCachePrStateFields helper now fetches the full payload once per cache miss and writes mergeable_state/state/headSha through together — headSha still omitted (not nulled) when absent, preserving the existing PARTIAL-UPDATE CONTRACT so a prior value the files cache depends on is never cleared. It also only writes on an actual fetch success, so a transient failure no longer poisons the cache with a false "confirmed fresh" stamp. The three public, uncached narrow fetchers used by the act-boundary/gate-override callers are untouched. Added a direct regression test proving a fetch from one reader now warms the other two, plus updated/added tests for the corrected headSha behavior.

2. Possible leaked secret (false positive, confirmed via the actual scanner). { token: "installation-token" } in six of this PR's own new test fixtures matched the deterministic generic_secret_assignment heuristic (a keyword-shaped pattern-match, not a real credential format — the value itself is an established, harmless test placeholder used hundreds of times elsewhere in this same file, just never previously part of a scanned diff). Ran the actual scanForSecrets function from src/review/secrets-scan.ts directly against the diff to confirm both that this was the only trigger and that renaming to "fake-installation-token" (recognized by the scanner's own placeholder-value allowlist) clears it — verified {"found":false,"kinds":[]} before pushing. No change to the shared scanner itself.

Also rebased twice more to keep pace with main (renumbered the migration 0093→0094, unrelated to this PR's own logic). Full local gate (test:ci, npm audit) green; 389 tests passing across the three affected test files.

[JSONbored]

Fixed a third, separate defect from the latest review pass — a real backward-compatibility issue:

Legacy reviews_synced_at markers from failed syncs. Confirmed by diffing against `origin/main`: every pre-existing writer of `reviews_synced_at` (`backfillOpenPullRequestDetails` / `refreshPullRequestDetails` / `backfillRepository`) stamped it unconditionally on every sync pass, success or failure — there was no gate on it at all before this PR. This PR's new durable review cache treats any non-null `reviews_synced_at` as "fully synced, never refetch" until a `pull_request_review` webhook invalidates it. So an existing row whose review sync previously failed (rate limit, transient error) already carries a timestamp from that failed attempt, and the new cache would trust that incomplete/stale review data indefinitely for any PR that doesn't happen to get a new review action after this deploys.

Added migration `0095`: a one-time `UPDATE` clearing `reviews_synced_at` on every existing row. There's no reliable way to tell a trustworthy stamp from an untrustworthy one from the row alone (`status` reflects the aggregate files/reviews/checks outcome for a pass, not reviews specifically), so it clears unconditionally rather than guessing — the cost is one redundant review refetch per already-correctly-synced PR, which is cheap and bounded. Pinned the exact migration behavior with a dedicated test that applies the real migration file against a raw in-memory SQLite DB (mirroring the existing `digest-subscription-normalization.test.ts` pattern), seeding legacy-shaped rows and asserting only `reviews_synced_at` clears while every other column is untouched.

Rebased again to keep pace with `main` (two more unrelated merges landed). Full local gate (`test:ci`, `npm audit`, `db:migrations:check`) green.

[JSONbored]

Superseded by #2633, which merged independently while this PR was still being iterated on.

#2633 delivers a hardened, complete implementation of the review-caching half of #2537 — including two things this PR's own fix history was still working through: making fetchAndStorePullRequestDetails report the real per-pass sync outcome instead of inferring it from warning strings (closing a TOCTOU race between the cache snapshot read and the final write, not just the "cache hit re-stamps the TTL" bug this PR caught), and a targeted legacy-data migration keyed on sync status rather than a blanket reset. It went through its own multiple gate-review rounds and is the better implementation for that half.

This PR's bare PR-state caching (mergeable_state/state/head-SHA, scoped to the non-mutating freshness-guard/readiness/dup-winner reads while keeping every act-boundary read — merge/close, draft-dodge, reopen-reclose, gate-override — live) is the half #2633 explicitly deferred as its own follow-up, and isn't touched by it. That work is tested and gate-verified in this branch's history but conflicts too deeply with #2633's now-merged review-cache implementation to rebase cleanly; closing here rather than force a risky reconciliation. Worth a fresh, narrowly-scoped PR against current main if that caching is still wanted — the design and tests already exist in this branch for reference.