feat(review): scope the live auto-tune breaker to miner-originated PRs independently (#5061)

Extends the existing self-tightening precision circuit-breaker
(src/review/auto-tune.ts) so a miner fleet's own self-review accuracy
trips the SAME safety breaker independently of the maintainer's
overall (mixed) review-stack accuracy, without ever blanket-penalizing
human-submitted PRs to the same repo.

Issue #2352's own premise (scope via computeGateEval's existing
`source` filter) does not hold: `source` only ever distinguishes WHICH
REVIEW ENGINE decided (currently always 'gittensory-native'), not who
authored the PR. The real signal -- `confirmedContributor`, an
official-Gittensor-miner check via the live subnet API -- was already
computed in processors.ts right where gate_decision rows get written,
but never threaded through.

- migrations/0144: adds `review_audit.miner_authored` -- a coarse,
  non-identifying boolean category (NOT a login), preserving
  review_audit's own deliberate "no actor-identifying data" design
  (it feeds the anonymized cross-instance export).
- parity-wire.ts: recordNativeGateDecision takes an optional
  `minerAuthored` field, written alongside the existing (unchanged)
  `source` column.
- processors.ts: threads the already-in-scope `confirmedContributor`
  through at the one gate_decision write site.
- parity.ts: computeGateEval gains an optional `minerOnly` filter,
  orthogonal to `source`. Omitted (every pre-#2352 caller) is
  byte-identical to before.
- outcomes-wire.ts: runSelfTuneBreaker now runs a SECOND,
  miner-scoped computeGateEval pass alongside the existing one, then
  re-keys its rows with a `:miner` suffix before running them through
  the SAME applyAutoTune/applyCloseAutoTune/maybeAutoClear* primitives
  used for the existing pass. Every one of those (plus createFlagStore
  and listEngagedProjectScopes) is already fully generic over an
  opaque `project` string, so the distinct flag scope
  (holdonly:<project>:miner) falls out naturally with ZERO changes to
  auto-tune.ts itself. Refactored the existing engage+log+autoclear
  sequence into a shared runBreakerPassForReport helper, run once per
  scope, so the human/mixed pass's behavior is preserved byte-for-byte
  (same event names, same log shape) while the miner pass reuses the
  identical logic under a "miner_" event prefix.

IMPORTANT, discovered while testing: the existing/unscoped pass is NOT
disjoint from miner-authored data -- it has no miner_authored filter,
so it counts every prediction for a project, miner-authored or not.
This preserves that pass's existing meaning (overall accuracy,
unchanged) but means a project's miner-authored rows are counted in
BOTH the mixed population and the miner-only subset.

Regression-tested per the issue's own deliverable: the miner-scoped
breaker fires independently of the existing one in both directions,
the CLOSE-side mirror does too, and clearing one flag (via cooldown +
precision recovery) never clears the other -- both merge- and
close-side, both directions.

Author: JSONbored

migrations/0144_review_audit_miner_authored.sql

@@ -0,0 +1,16 @@
+-- #2352: extend the live auto-tune circuit-breaker (src/review/auto-tune.ts) to consider miner-originated PRs
+-- as a distinguishable population, so a miner fleet's own self-review accuracy trips the SAME safety breaker
+-- independently of the maintainer's overall (mixed) review-stack accuracy.
+--
+-- review_audit (migration 0049) deliberately carries NO actor-identifying data (it feeds the anonymized
+-- cross-instance export in src/selfhost/orb-collector.ts -- see that migration's own "Privacy: ... No actor
+-- logins" comment). This column preserves that: it is a coarse, non-identifying miner/non-miner CATEGORY, not
+-- a login -- it reveals no more than "was this PR's author, at decision time, a confirmed official Gittensor
+-- miner" (src/queue/processors.ts's `confirmedContributor`, itself an aggregate boolean from the live
+-- Gittensor subnet API, not a stored identity).
+--
+-- Written by src/review/parity-wire.ts's recordNativeGateDecision alongside the existing `source` column (that
+-- write is UNCHANGED -- this is a strictly additive column, so every existing read of review_audit, including
+-- computeGateEval's current unscoped/source-scoped passes, is byte-identical unless it opts into the new
+-- `minerOnly` filter). Read by src/review/parity.ts's computeGateEval when its new `minerOnly` option is set.
+ALTER TABLE review_audit ADD COLUMN miner_authored INTEGER NOT NULL DEFAULT 0;

src/queue/processors.ts

@@ -9200,6 +9200,9 @@ async function maybePublishPrPublicSurface(
         headSha: pr.headSha,
         conclusion: gateEvaluation.conclusion,
         reasonCode,
+        // #2352: lets the live auto-tune breaker (src/review/outcomes-wire.ts's runSelfTuneBreaker) scope a
+        // SEPARATE precision read to miner-originated PRs, independently of the maintainer's overall accuracy.
+        minerAuthored: confirmedContributor,
       });
       // #2349 (PR 1): additive per-contributor calibration data, mirroring recordNativeGateDecision's own
       // action derivation above so both writers agree on whether this conclusion is a comparable decision --

src/review/outcomes-wire.ts

@@ -513,6 +513,91 @@ const BREAKER_EVAL_WINDOW_DAYS = 90;
  * break the cron). With no pr_outcome history the eval reads neutral → nothing engages → byte-identical. The
  * close breaker is INERT until selftune is enabled AND close-outcome data is present, exactly like its merge twin.
  */
+// #2352: the flag-scope suffix that makes a miner-originated project's breaker flags (holdonly:<project>:miner
+// / closehold:<project>:miner) DISTINCT from the same project's human/mixed-population flags. Every downstream
+// primitive that keys on `project` -- applyAutoTune/applyCloseAutoTune/maybeAutoClear* (auto-tune.ts),
+// createFlagStore, listEngagedProjectScopes -- is already fully generic over that opaque string, so re-keying
+// a report's rows with this suffix is the ENTIRE mechanism; none of those primitives needed to change.
+const MINER_BREAKER_SCOPE_SUFFIX = ":miner";
+
+function minerBreakerScope(project: string): string {
+  return `${project}${MINER_BREAKER_SCOPE_SUFFIX}`;
+}
+
+/** Run the full engage + auto-clear sequence for one {@link GateEvalReport} (either the plain project-keyed
+ *  report or a miner-rescoped one). `eventPrefix` namespaces the emitted log events (`""` for the existing
+ *  human/mixed pass, `"miner_"` for the #2352 miner-scoped pass) so an operator can tell which population
+ *  triggered a given line. `engagedHoldonly`/`engagedClosehold` are this SAME scope's already-engaged flags
+ *  (the caller pre-splits {@link listEngagedProjectScopes}'s result by scope) -- passing the WRONG scope's
+ *  engaged list here would auto-clear a flag using the other population's precision, which is exactly the
+ *  cross-scope leak #2352 exists to prevent. */
+async function runBreakerPassForReport(
+  flags: FlagStore,
+  report: GateEvalReport,
+  engagedHoldonly: readonly string[],
+  engagedClosehold: readonly string[],
+  nowMs: number,
+  eventPrefix: string,
+): Promise<void> {
+  const engaged = await applyAutoTune(flags, report);
+  for (const action of engaged) {
+    console.error(
+      JSON.stringify({
+        level: "error",
+        event: `${eventPrefix}breaker_engaged`,
+        project: action.project,
+        mergePrecision: action.mergePrecision,
+        decided: action.decided,
+        floor: AUTOTUNE_MERGE_PRECISION_FLOOR,
+      }),
+    );
+  }
+  // CLOSE-side breaker: engage closehold for any repo whose close precision dropped below the floor.
+  const closeEngaged = await applyCloseAutoTune(flags, report);
+  for (const action of closeEngaged) {
+    console.error(
+      JSON.stringify({
+        level: "error",
+        event: `${eventPrefix}close_breaker_engaged`,
+        project: action.project,
+        closePrecision: action.closePrecision,
+        decided: action.decided,
+        floor: AUTOTUNE_CLOSE_PRECISION_FLOOR,
+      }),
+    );
+  }
+  // OBSERVABILITY: a single summary line of the engaged close-hold backlog so a human can see, at a glance,
+  // how many (and which) repos are currently holding would-closes for review. Only emitted when ≥1 engaged.
+  if (closeEngaged.length > 0) {
+    console.error(
+      JSON.stringify({
+        level: "error",
+        event: `${eventPrefix}closehold_backlog`,
+        count: closeEngaged.length,
+        projects: closeEngaged.map((a) => a.project),
+      }),
+    );
+  }
+  // Auto-clear any auto-engaged breaker (merge AND close) that has cooled down + recovered. Candidates are the
+  // UNION of report.rows (projects with a fresh decided sample) and every project currently holding a
+  // per-project flag IN THIS SCOPE (#autoclear-deadlock) — a project whose breaker is suppressing 100% of its
+  // merges/closes stops producing new decided samples for THAT action class and can drop out of report.rows
+  // entirely, which would otherwise strand its flag engaged forever regardless of how long the cooldown has
+  // elapsed.
+  const mergeClearCandidates = new Set([...report.rows.map((row) => row.project), ...engagedHoldonly]);
+  const closeClearCandidates = new Set([...report.rows.map((row) => row.project), ...engagedClosehold]);
+  for (const project of mergeClearCandidates) {
+    if (await maybeAutoClearHoldOnly(flags, report, project, nowMs)) {
+      console.log(JSON.stringify({ event: `${eventPrefix}breaker_auto_cleared`, project }));
+    }
+  }
+  for (const project of closeClearCandidates) {
+    if (await maybeAutoClearCloseHoldOnly(flags, report, project, nowMs)) {
+      console.log(JSON.stringify({ event: `${eventPrefix}close_breaker_auto_cleared`, project }));
+    }
+  }
+}
+
 export async function runSelfTuneBreaker(env: Env): Promise<void> {
   try {
     const nowMs = Date.now();
@@ -521,64 +606,41 @@ export async function runSelfTuneBreaker(env: Env): Promise<void> {
       nowMs,
       source: GITTENSORY_NATIVE_SOURCE,
     });
+    // #2352: a SEPARATE, miner-scoped pass so a miner fleet's own self-review accuracy trips the SAME breaker
+    // independently of the maintainer's overall (mixed) accuracy. Re-keying every row's `project` with the
+    // `:miner` suffix (see MINER_BREAKER_SCOPE_SUFFIX's own doc comment) is what makes every downstream
+    // primitive naturally produce a DISTINCT flag, with zero changes to auto-tune.ts itself.
+    const minerReportRaw = await computeGateEval(env, {
+      days: BREAKER_EVAL_WINDOW_DAYS,
+      nowMs,
+      source: GITTENSORY_NATIVE_SOURCE,
+      minerOnly: true,
+    });
+    const minerReport: GateEvalReport = {
+      hasSignal: minerReportRaw.hasSignal,
+      rows: minerReportRaw.rows.map((row) => ({ ...row, project: minerBreakerScope(row.project) })),
+    };
+
     const flags = createFlagStore(env);
-    const engaged = await applyAutoTune(flags, report);
-    for (const action of engaged) {
-      console.error(
-        JSON.stringify({
-          level: "error",
-          event: "breaker_engaged",
-          project: action.project,
-          mergePrecision: action.mergePrecision,
-          decided: action.decided,
-          floor: AUTOTUNE_MERGE_PRECISION_FLOOR,
-        }),
-      );
-    }
-    // CLOSE-side breaker: engage closehold for any repo whose close precision dropped below the floor.
-    const closeEngaged = await applyCloseAutoTune(flags, report);
-    for (const action of closeEngaged) {
-      console.error(
-        JSON.stringify({
-          level: "error",
-          event: "close_breaker_engaged",
-          project: action.project,
-          closePrecision: action.closePrecision,
-          decided: action.decided,
-          floor: AUTOTUNE_CLOSE_PRECISION_FLOOR,
-        }),
-      );
-    }
-    // OBSERVABILITY: a single summary line of the engaged close-hold backlog so a human can see, at a glance,
-    // how many (and which) repos are currently holding would-closes for review. Only emitted when ≥1 engaged.
-    if (closeEngaged.length > 0) {
-      console.error(
-        JSON.stringify({
-          level: "error",
-          event: "closehold_backlog",
-          count: closeEngaged.length,
-          projects: closeEngaged.map((a) => a.project),
-        }),
-      );
-    }
-    // Auto-clear any auto-engaged breaker (merge AND close) that has cooled down + recovered. Candidates are the
-    // UNION of report.rows (projects with a fresh decided sample) and every project currently holding a
-    // per-project flag (#autoclear-deadlock) — a project whose breaker is suppressing 100% of its merges/closes
-    // stops producing new decided samples for THAT action class and can drop out of report.rows entirely, which
-    // would otherwise strand its flag engaged forever regardless of how long the cooldown has elapsed.
     const engagedScopes = await listEngagedProjectScopes(env);
-    const mergeClearCandidates = new Set([...report.rows.map((row) => row.project), ...engagedScopes.holdonly]);
-    const closeClearCandidates = new Set([...report.rows.map((row) => row.project), ...engagedScopes.closehold]);
-    for (const project of mergeClearCandidates) {
-      if (await maybeAutoClearHoldOnly(flags, report, project, nowMs)) {
-        console.log(JSON.stringify({ event: "breaker_auto_cleared", project }));
-      }
-    }
-    for (const project of closeClearCandidates) {
-      if (await maybeAutoClearCloseHoldOnly(flags, report, project, nowMs)) {
-        console.log(JSON.stringify({ event: "close_breaker_auto_cleared", project }));
-      }
-    }
+    const isMinerScope = (project: string): boolean => project.endsWith(MINER_BREAKER_SCOPE_SUFFIX);
+
+    await runBreakerPassForReport(
+      flags,
+      report,
+      engagedScopes.holdonly.filter((project) => !isMinerScope(project)),
+      engagedScopes.closehold.filter((project) => !isMinerScope(project)),
+      nowMs,
+      "",
+    );
+    await runBreakerPassForReport(
+      flags,
+      minerReport,
+      engagedScopes.holdonly.filter(isMinerScope),
+      engagedScopes.closehold.filter(isMinerScope),
+      nowMs,
+      "miner_",
+    );
   } catch (error) {
     console.warn(
       JSON.stringify({

src/review/parity-wire.ts

@@ -137,7 +137,19 @@ type ParityRecorderEnv = {
  */
 export async function recordNativeGateDecision(
   env: ParityRecorderEnv,
-  input: { project: string; pullNumber: number; headSha: string | null | undefined; conclusion: GateCheckConclusion; reasonCode?: string | null | undefined; action?: GateAction | undefined },
+  input: {
+    project: string;
+    pullNumber: number;
+    headSha: string | null | undefined;
+    conclusion: GateCheckConclusion;
+    reasonCode?: string | null | undefined;
+    action?: GateAction | undefined;
+    /** #2352: true when the PR's author is a confirmed official Gittensor miner (processors.ts's
+     *  `confirmedContributor`) at decision time. A coarse, non-identifying category -- NOT a login -- so this
+     *  stays within review_audit's own "no actor-identifying data" design (see migration 0144's own comment).
+     *  Omitted defaults to `false` (not miner-originated), matching every pre-#2352 caller unchanged. */
+    minerAuthored?: boolean | undefined;
+  },
 ): Promise<void> {
   // Self-hosted instances always record (their own local DB; exportOrbBatch needs this data). The cloud
   // worker keeps the exact flag-gated, byte-identical-when-off contract.
@@ -148,16 +160,17 @@ export async function recordNativeGateDecision(
   const project = input.project.slice(0, 200);
   const targetId = `${project}#${input.pullNumber}`;
   const summary = input.reasonCode ? input.reasonCode.slice(0, 200) : null;
+  const minerAuthored = input.minerAuthored === true ? 1 : 0;
   try {
     // Deterministic id per (source, project, pr, sha): a re-run at the SAME commit REPLACES its prior decision
     // (the latest finalize wins), while a new commit gets its own row. event_type/source default in the schema
     // but are written explicitly for clarity.
     await env.DB.prepare(
-      `INSERT INTO review_audit (id, project, target_id, event_type, decision, source, head_sha, summary, created_at)
-       VALUES (?, ?, ?, 'gate_decision', ?, ?, ?, ?, ?)
-       ON CONFLICT(id) DO UPDATE SET decision = excluded.decision, summary = excluded.summary, created_at = excluded.created_at`,
+      `INSERT INTO review_audit (id, project, target_id, event_type, decision, source, head_sha, summary, miner_authored, created_at)
+       VALUES (?, ?, ?, 'gate_decision', ?, ?, ?, ?, ?, ?)
+       ON CONFLICT(id) DO UPDATE SET decision = excluded.decision, summary = excluded.summary, miner_authored = excluded.miner_authored, created_at = excluded.created_at`,
     )
-      .bind(`gate:${GITTENSORY_NATIVE_SOURCE}:${targetId}@${input.headSha}`, project, targetId, action, GITTENSORY_NATIVE_SOURCE, input.headSha, summary, nowIso())
+      .bind(`gate:${GITTENSORY_NATIVE_SOURCE}:${targetId}@${input.headSha}`, project, targetId, action, GITTENSORY_NATIVE_SOURCE, input.headSha, summary, minerAuthored, nowIso())
       .run();
   } catch (error) {
     // Telemetry must never break finalization.

src/review/parity.ts

@@ -84,17 +84,22 @@ export const REVERSAL_DISCOUNT_WEIGHT = 0;
  *  pr_outcome (ground truth) is the human's realized merge/close, so it is NOT source-scoped — both
  *  systems are graded against the same answer key. Also LEFT JOINs a reversal existence check (#2348) so the
  *  fold below can additionally compute weightedMergeConfirmed/weightedCloseConfirmed alongside the existing
- *  raw counts — see REVERSAL_DISCOUNT_WEIGHT's doc comment for the formula. */
-export async function computeGateEval(env: Env, opts: { days: number; nowMs: number; source?: string }): Promise<GateEvalReport> {
+ *  raw counts — see REVERSAL_DISCOUNT_WEIGHT's doc comment for the formula.
+ *  `minerOnly` (#2352) additionally scopes the PREDICTION side to rows recorded with `miner_authored = 1`
+ *  (migration 0144) — orthogonal to `source`: both filters AND together when both are set. Ground truth stays
+ *  unscoped either way (same answer key). Omitted (the default, and every pre-#2352 caller) is byte-identical
+ *  to before this option existed. */
+export async function computeGateEval(env: Env, opts: { days: number; nowMs: number; source?: string; minerOnly?: boolean }): Promise<GateEvalReport> {
   const days = Number.isFinite(opts.days) && opts.days > 0 ? Math.min(opts.days, 730) : 90;
   const fromIso = new Date(opts.nowMs - days * 86_400_000).toISOString().slice(0, 10);
   // SQLite "bare column with MAX()" picks the column from the max-created_at row → the LATEST decision /
   // outcome per target (a reopened+reclosed PR keeps its final state).
   const sourceFilter = opts.source ? "AND source = ?" : "";
+  const minerFilter = opts.minerOnly ? "AND miner_authored = 1" : "";
   const sql = `
     WITH gd AS (
       SELECT target_id, project, decision AS pred, MAX(created_at) AS t
-      FROM review_audit WHERE event_type = 'gate_decision' AND decision IS NOT NULL AND created_at >= ? ${sourceFilter}
+      FROM review_audit WHERE event_type = 'gate_decision' AND decision IS NOT NULL AND created_at >= ? ${sourceFilter} ${minerFilter}
       GROUP BY target_id
     ),
     po AS (