meta: adversarial audit 2026-07-27 — index, method, and cross-cutting patterns

[JSONbored]

Summary

Index for the adversarial audit run on 2026-07-27 against origin/main @ 776c4147a, plus live inspection of the production ORB server edge-nl-01.

Method: seven parallel deep sweeps (webhook/relay/dedup, queue/locks/concurrency, rate limiting, review lifecycle, self-host runtime/resources, AI pipeline, newest features + maintainability), each instructed to check recent commit history before reporting so that already-fixed issues were filtered out. Every finding cited below was then re-verified by reading the cited lines at HEAD before filing; findings that did not survive verification were dropped.

Epics

Epic	Theme	Severity
#9461	Four verified paths to a wrong auto-verdict	Critical
#9475	Data retention and DB correctness	Critical (confirmed production data loss)
#9495	Rate limiting — inbound DoS and outbound backoff clearing	High (incl. security)
#9466	Per-PR lock lifecycle	High
#9484	States a PR can enter and never leave	High
#9480	AI pipeline reliability, verdict fidelity, cost	High
#9488	Runtime, deployment and observability	Medium
#9492	Maintainability — the structures that generate these bugs	Medium

Standalone: #9471 (relay silent event loss).

The three findings to act on first

1. gate: AI blocker silently collapses to a clean pass when its text trips the public-safe filter (false merge, live config) #9460 — an AI blocker silently becomes a clean pass. In the single combine strategy (what this deployment runs), a blocker whose text contains an ordinary word like score or ranking collapses to "no defect" and the PR merges. fix(review): filter the AI narrative assessment per sentence, not all-or-nothing #9445, two commits before the audit, widened its reachability.
2. retention: dedupeSignalSnapshots keeps the STALE row on Postgres (MAX(rowid)→MAX(ctid)) — confirmed deleting fresh snapshots in production #9470 — dedupeSignalSnapshots deletes the fresh row and keeps the stale one on Postgres. MAX(rowid) becomes MAX(ctid) — physical position, not recency. Measured on the production database: ~344 keys across dedupe-eligible types currently resolve to the wrong row, and the dedupe has been running daily (8,570 rows deleted on 2026-07-27 alone).
3. security(rate-limit): webhook bucket keyed on unverified installation.id — 10 unauthenticated req/min silences a target installation's webhooks #9469 — an unauthenticated party can silence an installation's webhooks with 10 requests/minute, because the rate-limit bucket is keyed on an installation.id read from the body before signature verification.

What the audit says about the codebase

The engine is in good shape on the axes that get attention. The sweeps explicitly probed and cleared: signature verification (timing-safe, fail-closed), the broker/token identity path, truncated-JSON handling at the output-token limit (cannot produce a false merge or close), the consensus combine branch, AI-output→comment sanitization, inline-comment anchoring, subprocess credential hygiene, the queue state machine's status-guarded transitions, fairness and starvation escapes, CI-resolution edge cases, the collector's crash-safe cursor, private-config merge semantics, and the ledger's signing crypto. Details are recorded in the relevant epics so they are not re-audited.

The defects that remain are almost all silent ones. That is the through-line: 7-day production error volume is genuinely low (3 dead jobs, 2 export failures), because none of these failures raise errors. They fail into "looks fine".

Two patterns generate most of them:

• "Cannot determine" resolving to "nothing found." gate: AI blocker silently collapses to a clean pass when its text trips the public-safe filter (false merge, live config) #9460, gate(visual): capture-retry marker never cleared + pending marker skips instead of holds → screenshot gate silently disabled (false merge) #9462 and gate(visual): browserless render failure is swallowed, not deferred → screenshot gate false-closes legitimate PRs (#9030/#9207 unpatched sibling) #9464 are three instances in three subsystems. pre-merge-checks.ts already models the correct behaviour (unresolvable ⇒ NEUTRAL ⇒ hold, explicitly to prevent auto-merge bypass); the other gates do not follow it.
• Guards added to N instances when there were N+1. drift: the #9399 sibling-guard pattern is live in four more places — fix the four, then remove the structure that generates them #9491 documents four live cases of the exact pattern queue: generate-tests checkbox and text-command both missing the pr.state !== "open" guard #9020 added to their retrigger sibling #9399 fixed. The structural cause is measured in maintainability: processors.ts (16.2k lines, 437 touches) is the structural cause of this audit's drift and determinism gaps; plus a confirmed dead wire #9492: processors.ts is 16,200 lines with 437 touches since 2026-07-01, and most of the sibling pairs that drift apart are inside it.

A third, narrower pattern worth naming: SQLite-shaped SQL silently changing meaning under the Postgres dialect shim (#9470), which no SQLite-only test can catch.

Scope notes

• Review policy on this deployment lives in the private server-side config on edge-nl-01, not the public repo .loopover.yml. Config-side items (ops: two misfiring alerts, a blind spot on visual capture, a 44% false-positive gate, and three hardening gaps (from live edge-nl-01 inspection) #9487) are flagged as such.
• The deployed image at audit time was orb-v3.5.0-beta.10, which predates fix(retention): bound the five unpruned tables that filled the hosted D1 #9415 — so retention: #9415's five tables prune via full seq scan + sort, hourly — no PK mapping, no indexes, no migration #9472's hourly full-scan prune lands when the next build deploys, and should be fixed before then.

[linear-code]

JSO-6560