fix(review): stop publishing a reproducibility freeze point that commits to an empty corpus

`/v1/public/stats` reported a `latestBacktestRun.corpusChecksum` of
4f53cda1…, which is `checksumCases([])` — SHA-256 over the canonicalized
empty case list. The fairness page rendered it as a "Reproducibility freeze
point", and `/v1/public/eval-scores` published two records committed to it
with `trust.tier: "reproducible"` alongside decided=460/confirmed=287. A hash
over zero cases is the same 32 bytes for every rule, every window and every
deployment, so it points at nothing a skeptic can re-derive anything from.

Treat it the same way as a missing run: `loadPublicRulePrecision` now reports
`latestBacktestRun: null` for it, which clears the freeze point from the
fairness page and, through the builder's existing guard, empties the
eval-scores response. The record builder additionally refuses the value on its
own, since it is the code that stamps the `reproducible` tier and should not
depend on its caller to be honest.

The scores are unaffected — they come from a different dataset
(`signal.human_override:*` audit events), so an empty corpus never means the
numbers are zero, only that they are uncommitted, which is exactly the state
the record spec says must not be published.

Docs: the walkthrough's step 4 referenced a bare `/v1/public/stats`, which
resolves against loopover.ai and 404s — only the API host serves it. Fetch it
absolutely, matching the convention in what-you-can-verify.mdx. Step 1 and the
track-record command shell out to `wrangler d1 execute --remote` against the
deployment's own database, so they need that deployment's Cloudflare
credentials and are not runnable by a third party; each step is now marked
with who can actually run it instead of the page claiming nothing needs a key.
The freeze point covers whichever single rule the latest run backtested, not
every rule on the report, and is described that way now.

Author: JSONbored

apps/loopover-ui/content/docs/verify-this-review.mdx

@@ -11,16 +11,27 @@ website only build trust if a skeptic can check them without asking anyone's per
 page is the end-to-end walkthrough: export the same corpus snapshot the numbers come from, verify
 its checksum, replay the same scorer over it, and compare what you get against what is published.
 
-Everything below runs read-only against a database export and pure functions from
-`@loopover/engine`. Nothing posts anywhere, nothing needs an API key.
+Everything below runs read-only against a corpus export and pure functions from `@loopover/engine`.
+Nothing posts anywhere, and nothing needs a LoopOver API key.
+
+<Callout variant="warn">
+  **Step 1 is not yet runnable by a stranger.** Exporting the corpus reads the deployment's own
+  database — with `--remote` that means the operator's Cloudflare D1, which needs *their* Cloudflare
+  credentials, not a LoopOver key. So today steps 1–3 are reproducible by an operator or by anyone
+  running a self-host deployment against their own data, and step 4 is the part a third party can
+  check unauthenticated. No anonymous corpus download exists yet; until one does, this page marks
+  each step with who can actually run it rather than implying everyone can run all of them.
+</Callout>
 
 For the wider contract — every claim, its artifact, and its trust assumption, including the ones you
 cannot check — see [what you can verify](/docs/what-you-can-verify).
 
-## 1. Export the corpus snapshot
+## 1. Export the corpus snapshot *(operator / self-host)*
 
 Every rule's fired/override history exports as a versioned, checksummed JSON snapshot
-([backtest &amp; calibration](/docs/backtest-calibration) explains how that history is recorded):
+([backtest &amp; calibration](/docs/backtest-calibration) explains how that history is recorded).
+`--remote` shells out to `wrangler d1 execute … --remote`, so it reads the deployed database and
+requires that deployment's own Cloudflare credentials:
 
 ```bash
 npx tsx scripts/backtest-corpus-export.ts --rule-id linked_issue_scope_mismatch --output corpus.json --remote
@@ -34,10 +45,12 @@ npx tsx scripts/backtest-corpus-export.ts --rule-id linked_issue_scope_mismatch
 
 The snapshot's `checksum` field is a SHA-256 over the canonicalized cases (keys sorted, so
 property order can never change the hash). The fairness report's *reproducibility freeze point*
-shows the checksum of the corpus behind the latest persisted backtest run — an export of the same
-window reproduces the same checksum, byte for byte.
+shows that checksum for the **most recent persisted backtest run**, whichever single rule that run
+covered — re-exporting that rule over the same window reproduces the same checksum, byte for byte.
+It is one run's freeze point, not a per-rule commitment for every rule on the report, so compare it
+against an export of the rule the run actually covered.
 
-## 2. Verify the checksum
+## 2. Verify the checksum *(anyone, given a snapshot)*
 
 The manifest is self-verifying: recompute the hash over its own `cases` array and compare it to
 the recorded `checksum`. The canonicalization lives in `scripts/backtest-corpus-export-core.ts`
@@ -53,7 +66,7 @@ console.log(recomputed.checksum === saved.checksum ? "checksum OK" : "CHECKSUM M
 '
 ```
 
-## 3. Replay the scorer
+## 3. Replay the scorer *(anyone, given a snapshot)*
 
 The published precision comes from the same pure functions any Node script can import:
 `scoreBacktest` replays a classifier over the labeled cases, and `compareBacktestScores` applies
@@ -75,19 +88,42 @@ console.log(report);
 - **`null` is never `0`.** Precision and recall stay `null` below the decided-sample floor;
   the fairness report renders that as *insufficient data*, never as a zero.
 
-## 4. Compare against the published numbers
+## 4. Compare against the published numbers *(anyone)*
+
+The [fairness report](/fairness) renders each rule's decided-case count and measured precision from
+the public stats endpoint's `rulePrecision` block. That endpoint is served by the API host, not by
+this site, so fetch it absolutely — a bare `/v1/public/stats` resolves against `loopover.ai` and
+404s:
+
+```bash
+curl -s "https://api.loopover.ai/v1/public/stats" | jq '.rulePrecision'
+```
+
+The same per-rule numbers are also published as digest-committed
+[EvalScoreRecords](/docs/what-you-can-verify), each independently re-derivable without trusting the
+transport — recompute `recordDigest` over the record's own remaining fields and compare:
+
+```bash
+curl -s "https://api.loopover.ai/v1/public/eval-scores" | jq '.records'
+```
+
+That array is empty whenever the latest backtest run's corpus is empty. A checksum over zero cases
+is byte-identical for every rule and every window, so it commits to nothing a reader could re-derive
+the scores from — publishing a record against it would assert a reproducibility that does not exist.
+An empty `records` array therefore means *the numbers are not currently committed to a corpus*, never
+that the numbers are zero.
 
-The [fairness report](/fairness) renders each rule's decided-case count and measured precision
-from the public stats endpoint (`/v1/public/stats`, the `rulePrecision` block). The aggregated
-run history is also readable directly:
+The aggregated run history is readable directly too, again against the deployment's own database
+(operator / self-host):
 
 ```bash
 npx tsx scripts/backtest-track-record.ts --db loopover --remote
 ```
 
 Your replayed `confirmed / decided` for a rule should match the published precision for the same
-window; the freeze-point checksum ties the published numbers to the exact corpus you just
-verified.
+window. The freeze-point checksum ties the published numbers to the corpus you just verified only
+for the rule the latest run actually backtested — for any other rule it is a timestamped pointer to
+a different run, not a commitment to that rule's own cases.
 
 ## 5. Verify an attested run (when a run carries one)
 

apps/loopover-ui/content/docs/what-you-can-verify.mdx

@@ -176,7 +176,10 @@ replayable history — not hand-entered numbers.
 
 This is the walkthrough in [Verify this review](/docs/verify-this-review): export the checksummed
 corpus, verify its checksum, re-run the same public scoring functions from `@loopover/engine`, and
-compare. **Anyone** can do this for public repositories.
+compare. The checksum verification and the scorer replay are pure functions **anyone** can run over
+a snapshot they hold, and the published numbers themselves are fetchable unauthenticated — but the
+export step that produces the snapshot reads the deployment's own database and needs that
+deployment's credentials, so it is an operator / self-host step today rather than an anonymous one.
 
 <Callout variant="warn">
   **Private repositories are the exception.** A hosted tenant's review history cannot be published

src/review/eval-score-records.ts

@@ -4,7 +4,7 @@
 // It adds no new scoring and no new trust -- the numbers are the same ones `/v1/public/stats` already
 // publishes, just committed to a corpus checksum and made independently re-derivable per-record.
 import { canonicalJson, contentDigest } from "./decision-record";
-import type { PublicRulePrecision } from "./public-rule-precision";
+import { EMPTY_CORPUS_CHECKSUM, type PublicRulePrecision } from "./public-rule-precision";
 
 export const EVAL_SCORE_RECORD_SCHEMA_VERSION = 1 as const;
 
@@ -76,6 +76,14 @@ async function finalizeRecord(input: EvalScoreRecordDigestInput): Promise<EvalSc
  * a record whose commitments cannot be independently re-derived (no corpus checksum to point at) is not
  * publishable, so this deliberately emits nothing rather than a record with a placeholder commitment.
  *
+ * Also returns an empty array when the run's checksum is {@link EMPTY_CORPUS_CHECKSUM}. A hash over zero
+ * cases is the same 32 bytes for every rule, every window, and every deployment, so it points at nothing a
+ * consumer could re-derive the scores from -- it is a placeholder commitment wearing a real hash's clothes,
+ * and pairing it with a `reproducible` trust tier claims a reproducibility the artifact cannot support. The
+ * scores themselves come from a different dataset (live human-override events) and are unaffected by whether
+ * a corpus was exported, so an empty corpus never means the numbers are zero -- it means they are
+ * uncommitted, which is exactly the state #9215 says must not be published.
+ *
  * `recall` and `abstained` do not apply to this work-unit kind: ORB's gate rules fire deterministically (no
  * agent choosing to abstain) and this data measures precision, not a false-negative rate. `recall` is
  * `null` (genuinely inapplicable, never a misleading `0`); `abstained` is `0` (there is no abstention
@@ -85,6 +93,7 @@ async function finalizeRecord(input: EvalScoreRecordDigestInput): Promise<EvalSc
 export async function buildEvalScoreRecordsFromRulePrecision(precision: PublicRulePrecision, issuedAt: string): Promise<EvalScoreRecord[]> {
   if (!precision.latestBacktestRun) return [];
   const { corpusChecksum } = precision.latestBacktestRun;
+  if (corpusChecksum === EMPTY_CORPUS_CHECKSUM) return [];
   const windowStart = new Date(Date.parse(issuedAt) - precision.windowDays * 24 * 60 * 60 * 1000).toISOString();
 
   const records = await Promise.all(
@@ -145,3 +154,4 @@ export async function verifyEvalScoreRecordDigest(record: EvalScoreRecord): Prom
 // Re-exported so callers that only import this module never need a second import from decision-record.ts
 // just to canonicalize something alongside a record (e.g. logging, or a future signed-bundle wrapper).
 export { canonicalJson, contentDigest };
+export { EMPTY_CORPUS_CHECKSUM };

src/review/public-rule-precision.ts

@@ -23,6 +23,15 @@ export const PUBLIC_PRECISION_MIN_DECIDED = 10;
 // duplication rule-calibration-trend.ts documents for its identical queries.
 const HUMAN_OVERRIDE_EVENT_TYPE_PREFIX = "signal.human_override:";
 
+/** `checksumCases([])` — SHA-256 over the canonicalized empty case list (the two-byte string `"[]"`), i.e. what
+ *  a corpus export produces for a rule with no labeled cases at all. A hash over zero cases is the same 32
+ *  bytes for every rule, every window and every deployment, so it is not the "independently-verifiable freeze
+ *  point" {@link PublicRulePrecision.latestBacktestRun} claims to be — it points at nothing a skeptic could
+ *  re-derive anything from. Hard-coded because the canonicalization that produces it
+ *  (`scripts/backtest-corpus-export-core.ts`) runs on `node:crypto` and is unimportable from the Workers
+ *  runtime; this module's own test re-derives it from `sha256Hex("[]")` so it can never drift. */
+export const EMPTY_CORPUS_CHECKSUM = "4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945";
+
 export type PublicRulePrecisionRow = {
   ruleId: string;
   decided: number;
@@ -40,7 +49,9 @@ export type PublicRulePrecision = {
   /** All three reversal shapes counted over the window — the "counted against ourselves" number. */
   reversals: { reopened: number; reverted: number; superseded: number };
   /** The latest persisted backtest run carrying a corpus checksum — the independently-verifiable freeze
-   *  point — or null when no run has been recorded yet. */
+   *  point — or null when no run has been recorded yet, or when the latest run's corpus was empty (see
+   *  {@link EMPTY_CORPUS_CHECKSUM}: a commitment to nothing is not a freeze point, so it is reported as
+   *  absent rather than published as though it were verifiable). */
   latestBacktestRun: { corpusChecksum: string; at: string } | null;
 };
 
@@ -102,6 +113,9 @@ export async function loadPublicRulePrecision(env: Env, nowMs: number = Date.now
       reverted: reversalCount("reversal_reverted"),
       superseded: reversalCount("reversal_superseded"),
     },
-    latestBacktestRun: latest && typeof latest.checksum === "string" && latest.checksum !== "" ? { corpusChecksum: latest.checksum, at: latest.created_at } : null,
+    latestBacktestRun:
+      latest && typeof latest.checksum === "string" && latest.checksum !== "" && latest.checksum !== EMPTY_CORPUS_CHECKSUM
+        ? { corpusChecksum: latest.checksum, at: latest.created_at }
+        : null,
   };
 }

test/unit/eval-score-records.test.ts

@@ -6,9 +6,10 @@ import {
   ORB_GATE_SUBJECT_ID,
   OUTCOME_CONFIRMED_PRECISION_SCORING_RULE_VERSION,
   verifyEvalScoreRecordDigest,
+  EMPTY_CORPUS_CHECKSUM,
   type EvalScoreRecord,
 } from "../../src/review/eval-score-records";
-import { contentDigest } from "../../src/review/decision-record";
+import { contentDigest, sha256Hex } from "../../src/review/decision-record";
 import type { PublicRulePrecision } from "../../src/review/public-rule-precision";
 
 const ISSUED_AT = "2026-07-27T12:00:00.000Z";
@@ -29,6 +30,25 @@ describe("buildEvalScoreRecordsFromRulePrecision (#9266)", () => {
     expect(records).toEqual([]);
   });
 
+  it("refuses to publish records whose freeze point commits to an empty corpus", async () => {
+    // Regression: production published decided=460/confirmed=287 alongside sha256("[]") -- a hash that is
+    // byte-identical for every rule and every window, so it committed to nothing a consumer could re-derive.
+    const records = await buildEvalScoreRecordsFromRulePrecision(
+      { ...PRECISION_WITH_FREEZE_POINT, latestBacktestRun: { corpusChecksum: EMPTY_CORPUS_CHECKSUM, at: "2026-07-27T10:00:00.000Z" } },
+      ISSUED_AT,
+    );
+    expect(records).toEqual([]);
+  });
+
+  it("EMPTY_CORPUS_CHECKSUM is the exporter's own checksum over zero cases", async () => {
+    // scripts/backtest-corpus-export-core.ts hashes `JSON.stringify(cases.map(canonicalizeCase))`, which for
+    // an empty list is the two-byte string "[]" -- re-derived here so the hard-coded constant cannot drift
+    // from the exporter that produces the value it guards against. (That exporter's canonicalization and
+    // canonicalJson coincide ONLY on the empty case, so this hashes the literal preimage rather than
+    // round-tripping [] through either one.)
+    expect(EMPTY_CORPUS_CHECKSUM).toBe(await sha256Hex("[]"));
+  });
+
   it("builds one record per rule, committed to the freeze point's corpus checksum", async () => {
     const records = await buildEvalScoreRecordsFromRulePrecision(PRECISION_WITH_FREEZE_POINT, ISSUED_AT);
     expect(records).toHaveLength(2);

test/unit/public-rule-precision.test.ts

@@ -3,7 +3,9 @@ import {
   loadPublicRulePrecision,
   PUBLIC_PRECISION_MIN_DECIDED,
   PUBLIC_PRECISION_WINDOW_DAYS,
+  EMPTY_CORPUS_CHECKSUM,
 } from "../../src/review/public-rule-precision";
+import { sha256Hex } from "../../src/review/decision-record";
 import { recordAuditEvent } from "../../src/db/repositories";
 import { createSignalStore } from "../../src/review/signal-tracking-wire";
 import { createTestEnv } from "../helpers/d1";
@@ -95,6 +97,33 @@ describe("loadPublicRulePrecision (#8230)", () => {
     expect(block.latestBacktestRun).toEqual({ corpusChecksum: "newest111", at: new Date(NOW - 30_000).toISOString() });
   });
 
+  it("reports no freeze point when the latest run's corpus was empty", async () => {
+    // Regression: production's latest run carried sha256("[]") -- the checksum of ZERO cases -- and the
+    // fairness page rendered it as a "reproducibility freeze point" while /v1/public/eval-scores published
+    // records committed to it. A hash over no cases is identical everywhere, so it verifies nothing.
+    const env = createTestEnv();
+    await seedVerdicts(env, "ai_consensus_defect", 15, 5);
+    await recordAuditEvent(env, {
+      eventType: "calibration.logic_backtest_run",
+      targetKey: "rule",
+      outcome: "completed",
+      metadata: { corpusChecksum: EMPTY_CORPUS_CHECKSUM, comparison: {} },
+      createdAt: new Date(NOW - 1000).toISOString(),
+    });
+
+    const block = await loadPublicRulePrecision(env, NOW);
+    expect(block.latestBacktestRun).toBeNull();
+    // The scores come from a different dataset (human-override events) and are unaffected -- an empty corpus
+    // means the numbers are uncommitted, never that they are zero.
+    expect(block.rules).toEqual([{ ruleId: "ai_consensus_defect", decided: 20, confirmed: 15, precision: 0.75 }]);
+  });
+
+  it("EMPTY_CORPUS_CHECKSUM is the exporter's own checksum over zero cases", async () => {
+    // scripts/backtest-corpus-export-core.ts hashes `JSON.stringify(cases.map(canonicalizeCase))`, which for
+    // an empty list is the two-byte string "[]" -- re-derived here so the constant cannot drift from it.
+    expect(EMPTY_CORPUS_CHECKSUM).toBe(await sha256Hex("[]"));
+  });
+
   it("degrades fail-safe on a broken store and reports null freeze point on a fresh ledger", async () => {
     const empty = await loadPublicRulePrecision(createTestEnv(), NOW);
     expect(empty).toEqual({