Skip to content

Commit 79d6e4f

Browse files
author
JSONbored
committed
feat(orb): record a per-decision confidence signal from inter-run agreement (#8834)
Verbalized confidence alone is poorly calibrated — #8845 already had to stop reading an absent confidence as certainty. Sampling-based consistency is the better-behaved signal, and the risk-control literature this epic builds on finds two samples capture most of the benefit. Scores inter-run agreement across the reviewer stances the engine ALREADY produces (#8229's reviewerVotes) and folds it into the verbalized confidence, at zero additional AI spend. The combined score multiplies the two so it is monotonically below either input — a judgment is only as trustworthy as both how sure the judge said it was and how reproducibly the judges reached it, which is the property an abstention threshold depends on. A lone run is recorded as UNCORROBORATED at a 0.5 agreement floor rather than fabricated unanimity, so a single-reviewer or budget-degraded review records a strictly lower confidence than a genuinely corroborated one. Zero samples refuses to invent a score at all. The signal rides to DecisionRecord.aiAgreement (schema v5) so every decision joins the risk-control calibration set with its reproducibility attached. Deliberately ADDITIVE: it does not re-route the gate. Disagreement already routes to a hold today — differing stances ARE the ai_review_split finding, which blocks or holds via the existing confidence floor — so a second parallel route would double-count the same evidence instead of measuring it.
1 parent dc1136c commit 79d6e4f

8 files changed

Lines changed: 192 additions & 2 deletions

src/queue/ai-review-orchestration.ts

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,7 @@ import { buildPullRequestAdvisory } from "../rules/advisory";
2323
import { recordAuditEvent, getDecryptedRepositoryAiKey, getRepository, listCheckSummaries, listPullRequestFiles } from "../db/repositories";
2424
import { registerHeldLock, unregisterHeldLock } from "./held-lock-registry";
2525
import { recordRoutingShadow } from "../services/reviewer-routing";
26+
import { scoreJudgmentAgreement } from "../review/judgment-agreement";
2627
import { createInstallationToken } from "../github/app";
2728
import type { AgentActionMode } from "../settings/agent-execution";
2829
import { buildAiReviewDiff } from "../review/review-diff";
@@ -889,6 +890,10 @@ export async function runAiReviewForAdvisory(
889890
// the REAL reviewer identities and the REAL system prompt into DecisionRecord instead of hardcoding null.
890891
const aiJudgmentModelIds = parsedReviewModelIds(result.reviewDiagnostics ?? []);
891892
const aiJudgmentPromptDigest = result.systemPromptDigest;
893+
// #8834: inter-run agreement over the stances this review ALREADY produced (#8229's reviewerVotes) —
894+
// zero additional AI spend. Computed once and attached to whichever AI-judgment finding is built below,
895+
// so the decision record carries a per-decision confidence signal for the calibration set (#8835).
896+
const aiJudgmentAgreement = (verbalizedConfidence: number) => scoreJudgmentAgreement(result.reviewerVotes, verbalizedConfidence);
892897
if (result.consensusDefect) {
893898
findings.push({
894899
code: "ai_consensus_defect",
@@ -913,6 +918,7 @@ export async function runAiReviewForAdvisory(
913918
confidence: result.consensusDefect.confidence,
914919
modelIds: aiJudgmentModelIds,
915920
promptDigest: aiJudgmentPromptDigest,
921+
agreement: aiJudgmentAgreement(result.consensusDefect.confidence),
916922
});
917923
} else if (result.split) {
918924
// The reviewers DISAGREED — exactly one flagged a blocking defect. reviewbot's quorum treats any reviewer
@@ -940,6 +946,10 @@ export async function runAiReviewForAdvisory(
940946
: {}),
941947
modelIds: aiJudgmentModelIds,
942948
promptDigest: aiJudgmentPromptDigest,
949+
// A split IS the disagreement case: the stances differ, so agreement scores strictly below unanimity
950+
// and the recorded confidence falls with it. #8834's "disagreement routes to hold" is already this
951+
// finding's existing behavior via the confidence floor; this measures it rather than re-routing it.
952+
agreement: aiJudgmentAgreement(result.splitConfidence ?? 1),
943953
});
944954
} else if (result.inconclusive) {
945955
// Fail-CLOSED (#ai-fail-closed): block-mode AI could not return a usable verdict. Hold the PR for a human

src/queue/processors.ts

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3763,6 +3763,10 @@ async function runAgentMaintenancePlanAndExecute(
37633763
modelIds: aiJudgment?.modelIds ?? null,
37643764
promptDigest: aiJudgment?.promptDigest ?? null,
37653765
aiConfidence: aiJudgment?.confidence ?? null,
3766+
// #8834: the inter-run agreement signal computed at review time (ai-review-orchestration.ts) and
3767+
// carried on the finding, so the record captures HOW REPRODUCIBLY the judgment was reached, not just
3768+
// what the model claimed. null for a rule-only decision, exactly like the fields above.
3769+
aiAgreement: aiJudgment?.agreement ?? null,
37663770
salvageability,
37673771
// #9135: legible on the record's own face — see maybeApplyCloseAuditHoldout's doc comment.
37683772
divertedByHoldout: closeAuditHoldout?.diverted ?? false,

src/review/decision-record.ts

Lines changed: 10 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -39,7 +39,7 @@
3939
import { errorMessage, nowIso } from "../utils/json";
4040

4141
/** Bump when the record's FIELD SET changes meaning — consumers compare records only within a version. */
42-
export const DECISION_RECORD_SCHEMA_VERSION = "4"; // v4 (#9124/#9135): configDigest digests the resolved policy (+ settingsDigest split out), promptDigest digests the actual sent prompt, modelId -> modelIds (real identities), ciState populated, + divertedByHoldout; v3 (#8962): + salvageability {score, factors}; v2 (#8834): + aiConfidence, model/prompt commitments
42+
export const DECISION_RECORD_SCHEMA_VERSION = "5"; // v5 (#8834): + aiAgreement (inter-run agreement folded with the verbalized confidence); v4 (#9124/#9135): configDigest digests the resolved policy (+ settingsDigest split out), promptDigest digests the actual sent prompt, modelId -> modelIds (real identities), ciState populated, + divertedByHoldout; v3 (#8962): + salvageability {score, factors}; v2 (#8834): + aiConfidence, model/prompt commitments
4343

4444
/**
4545
* Canonical JSON: recursively key-sorted, no insignificant whitespace — the ONE serialization every digest
@@ -118,6 +118,12 @@ export type DecisionRecord = {
118118
* defect / split), null when no AI judgment contributed. Persisted so every decision joins the
119119
* risk-control calibration set (#8835) with its confidence attached. */
120120
aiConfidence: number | null;
121+
/** #8834: the per-decision confidence signal — inter-run agreement across the reviewer stances that
122+
* produced the AI judgment, folded together with that judgment's verbalized confidence (see
123+
* src/review/judgment-agreement.ts). `aiConfidence` above records what the model SAID; this records how
124+
* reproducibly the reviewers reached it, which is the input a calibrated abstention threshold (#8835)
125+
* needs. null when no AI judgment contributed, and for every record predating v5. */
126+
aiAgreement: { agreement: number; confidence: number; sampleCount: number; uncorroborated: boolean } | null;
121127
/** #8962: the deterministic salvageability score + its named factors when an AI judgment shaped the
122128
* decision — the second-axis evidence for auditing the close/hold boundary. null for rule-only decisions
123129
* (and for reconstructed/backfilled records predating v3). */
@@ -135,12 +141,13 @@ export type DecisionRecord = {
135141
/** Assemble the record and its own content digest. PURE given pre-computed digests. Normalizes the
136142
* optional-shaped caller fields (undefined -> null) HERE so call sites carry no fallback arms of their own. */
137143
export async function buildDecisionRecord(
138-
input: Omit<DecisionRecord, "schemaVersion" | "decidedAt" | "gatePack" | "ciState" | "baseSha" | "aiConfidence" | "salvageability" | "settingsDigest" | "divertedByHoldout"> & {
144+
input: Omit<DecisionRecord, "schemaVersion" | "decidedAt" | "gatePack" | "ciState" | "baseSha" | "aiConfidence" | "aiAgreement" | "salvageability" | "settingsDigest" | "divertedByHoldout"> & {
139145
decidedAt?: string;
140146
gatePack?: string | null | undefined;
141147
ciState?: string | null | undefined;
142148
baseSha?: string | null | undefined;
143149
aiConfidence?: number | null | undefined;
150+
aiAgreement?: { agreement: number; confidence: number; sampleCount: number; uncorroborated: boolean } | null | undefined;
144151
salvageability?: { score: number; factors: string[] } | null | undefined;
145152
settingsDigest?: string | null | undefined;
146153
divertedByHoldout?: boolean | undefined;
@@ -154,6 +161,7 @@ export async function buildDecisionRecord(
154161
ciState: input.ciState ?? null,
155162
baseSha: input.baseSha ?? null,
156163
aiConfidence: input.aiConfidence ?? null,
164+
aiAgreement: input.aiAgreement ?? null,
157165
salvageability: input.salvageability ?? null,
158166
settingsDigest: input.settingsDigest ?? null,
159167
divertedByHoldout: input.divertedByHoldout ?? false,

src/review/judgment-agreement.ts

Lines changed: 73 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,73 @@
1+
// Per-decision confidence via inter-run agreement (#8834, epic #8828 Phase 3) — PURE.
2+
//
3+
// Verbalized confidence alone is poorly calibrated: a model asked "how sure are you" answers on a scale it
4+
// has no grounding in, and #8845 already had to stop treating an ABSENT confidence as certainty. Sampling-based
5+
// consistency is the better-behaved signal, and the risk-control literature this epic builds on (Trust or
6+
// Escalate, ICLR 2025) finds two samples capture most of the available benefit.
7+
//
8+
// This module scores the samples the engine ALREADY runs. A dual-model review produces one independent stance
9+
// per reviewer (`LoopOverAiReviewResult.reviewerVotes`, #8229) at zero additional AI spend, so inter-run
10+
// agreement is computable today without multiplying the per-review bill. See the "not implemented here" note
11+
// at the bottom for the paid rotated-exemplar extension and why it is deliberately absent.
12+
//
13+
// The score is ADDITIVE: it is recorded with the decision (`DecisionRecord.aiAgreement`) so it can join the
14+
// risk-control calibration set (#8835) as the input an abstention threshold needs. It deliberately does NOT
15+
// re-route the gate. Disagreement already routes to a hold today — one reviewer flagging and the other not
16+
// IS the `ai_review_split` finding, which blocks or holds via the existing confidence floor — so adding a
17+
// second, parallel disagreement route would double-count the same evidence rather than measure it.
18+
19+
/** One reviewer's stance on a single evaluation run. */
20+
export type JudgmentSample = {
21+
/** Whether this run flagged a blocking defect. The agreement axis. */
22+
votedFail: boolean;
23+
};
24+
25+
export type JudgmentAgreement = {
26+
/** Share of runs holding the MODAL stance, in [0,1]. Unanimity across N≥2 runs is 1; an even split is 0.5. */
27+
agreement: number;
28+
/** Inter-run agreement combined with the verbalized confidence — the calibrated per-decision signal. */
29+
confidence: number;
30+
/** How many runs actually produced a stance. */
31+
sampleCount: number;
32+
/** True when fewer than two runs were available, so agreement was never actually observed. */
33+
uncorroborated: boolean;
34+
};
35+
36+
/** The agreement credited to a lone run. A single sample cannot corroborate itself: scoring it 1.0 would
37+
* fabricate unanimity out of one opinion, which is exactly the failure #8845 fixed for absent confidence.
38+
* 0.5 — "no evidence either way" — keeps a single-run decision strictly below any genuinely corroborated
39+
* one, so a budget-degraded run records a LOWER confidence rather than a flattering one. */
40+
export const UNCORROBORATED_AGREEMENT = 0.5;
41+
42+
/**
43+
* Score inter-run agreement and fold it into the verbalized confidence. PURE and total.
44+
*
45+
* `verbalizedConfidence` is the model's own calibrated confidence in its blocker (`AdvisoryFinding.confidence`).
46+
* The combined score multiplies the two: a claim is only as trustworthy as BOTH how sure the judge said it was
47+
* AND how reproducibly the judges reached it. Multiplication (rather than an average) keeps the result
48+
* monotonically below either input, so the combined signal can never read as more certain than its weakest
49+
* component — the property an abstention threshold depends on.
50+
*
51+
* ZERO samples (every reviewer failed to produce a usable opinion) is not a fabricated score: it reports
52+
* `uncorroborated`, the uncorroborated agreement floor, and whatever confidence was actually stated. That
53+
* decision is already held as `ai_review_inconclusive` upstream; this just refuses to invent a number for it.
54+
*/
55+
export function scoreJudgmentAgreement(samples: readonly JudgmentSample[], verbalizedConfidence: number): JudgmentAgreement {
56+
const sampleCount = samples.length;
57+
const failCount = samples.filter((sample) => sample.votedFail).length;
58+
const modal = Math.max(failCount, sampleCount - failCount);
59+
// Below two runs there is nothing to agree WITH — fall back to the uncorroborated floor rather than
60+
// dividing by a sample count that cannot express disagreement.
61+
const uncorroborated = sampleCount < 2;
62+
const agreement = uncorroborated ? UNCORROBORATED_AGREEMENT : modal / sampleCount;
63+
const stated = Number.isFinite(verbalizedConfidence) ? Math.min(1, Math.max(0, verbalizedConfidence)) : 0;
64+
return { agreement, confidence: agreement * stated, sampleCount, uncorroborated };
65+
}
66+
67+
// NOT IMPLEMENTED HERE, deliberately (#8834): the issue also describes running N=2-3 evaluations of the SAME
68+
// judge with few-shot exemplars rotated out of the golden corpus ("simulated annotators"). That is a strictly
69+
// better agreement signal than two different models voting once each — it isolates the judge's own
70+
// reproducibility instead of confounding it with the two models' differing priors — but every extra run is a
71+
// real, per-review AI charge on every reviewed PR, and this engine pays that bill in production. The scoring
72+
// above is the half that costs nothing; the extra-sampling half needs a budget decision (and a flag defaulting
73+
// OFF) before it can ship, so it is not stubbed here rather than landing as unreachable code.

src/types.ts

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -611,6 +611,11 @@ export type AdvisoryFinding = {
611611
* inline, category, improvement-signal), NOT a digest of the base constant alone. Two repos with different
612612
* `review.instructions` therefore publish different values here. Absent for every deterministic finding. */
613613
promptDigest?: string;
614+
/** #8834: inter-run agreement across the reviewer stances that produced this AI judgment, folded together
615+
* with the verbalized confidence (see src/review/judgment-agreement.ts). Carried on the finding exactly
616+
* like `confidence`/`modelIds` above so the decision-record call site can thread it into
617+
* `DecisionRecord.aiAgreement` without re-deriving the votes. Absent for every deterministic finding. */
618+
agreement?: { agreement: number; confidence: number; sampleCount: number; uncorroborated: boolean };
614619
/** Public-safe screenshot evidence for a `visual_regression_finding` / `visual_unrelated_issue_finding`
615620
* (`review.visual.bugAnalysis`) — the SAME shot URLs already rendered in the "Visual preview" collapsible,
616621
* carried alongside the finding (not just referenced by route path) so a later consumer — the PR-closed

test/unit/ai-review-advisory.test.ts

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -390,6 +390,29 @@ describe("runAiReviewForAdvisory", () => {
390390
expect(result?.notes).toContain("Likely crash.");
391391
});
392392

393+
it("#8834: the consensus finding carries the inter-run agreement signal, derived from the stances already run", async () => {
394+
const adv = advisory();
395+
await runAiReviewForAdvisory(aiEnv(async () => ({ response: defectJson() })), {
396+
mode: "live",
397+
settings: { aiReviewMode: "block" } as RepositorySettings,
398+
advisory: adv,
399+
repoFullName: "acme/widgets",
400+
pr,
401+
author: "alice",
402+
confirmedContributor: true,
403+
});
404+
const finding = adv.findings.find((f) => f.code === "ai_consensus_defect");
405+
// Present, structurally complete, and consistent with the finding's own verbalized confidence — the
406+
// record's per-decision confidence signal (#8835's calibration input) is populated at review time, not
407+
// reconstructed later from votes nothing persisted.
408+
expect(finding?.agreement).toBeDefined();
409+
expect(finding?.agreement?.sampleCount).toBeGreaterThanOrEqual(1);
410+
expect(finding?.agreement?.agreement).toBeGreaterThan(0);
411+
expect(finding?.agreement?.agreement).toBeLessThanOrEqual(1);
412+
// The combined score is never more certain than the verbalized confidence it folds in.
413+
expect(finding?.agreement?.confidence).toBeLessThanOrEqual((finding?.confidence ?? 1) + 1e-12);
414+
});
415+
393416
it("threads settings.aiReviewCombine/aiReviewOnMerge/aiReviewReviewers (#2567) into the AI review call", async () => {
394417
// settings.aiReviewCombine/OnMerge/Reviewers are resolved from `.loopover.yml gate.aiReview.*` upstream by
395418
// resolveEffectiveSettings; runAiReviewForAdvisory must forward them into runLoopOverAiReview's input so a

test/unit/decision-record.test.ts

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -52,6 +52,7 @@ function recordInput(over: Partial<DecisionRecord> = {}): Omit<DecisionRecord, "
5252
repoFullName: "o/r",
5353
pullNumber: 7,
5454
salvageability: null,
55+
aiAgreement: null,
5556
headSha: "abc1234def",
5657
baseSha: "base999",
5758
action: "close",
Lines changed: 66 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,66 @@
1+
import { describe, expect, it } from "vitest";
2+
import { scoreJudgmentAgreement, UNCORROBORATED_AGREEMENT } from "../../src/review/judgment-agreement";
3+
4+
// #8834: the per-decision confidence signal. The contract worth pinning is that the score can never read as
5+
// MORE certain than its weakest input, and that a run with nothing to corroborate it is never scored as
6+
// unanimous — the same "silence is not certainty" failure #8845 fixed for absent verbalized confidence.
7+
8+
describe("scoreJudgmentAgreement (#8834)", () => {
9+
const fail = { votedFail: true };
10+
const pass = { votedFail: false };
11+
12+
it("unanimous reviewers score full agreement and keep the verbalized confidence intact", () => {
13+
expect(scoreJudgmentAgreement([fail, fail], 0.9)).toEqual({ agreement: 1, confidence: 0.9, sampleCount: 2, uncorroborated: false });
14+
// Unanimity on the NON-fail stance is equally unanimous — agreement is about reproducibility, not verdict.
15+
expect(scoreJudgmentAgreement([pass, pass], 0.8)).toMatchObject({ agreement: 1, confidence: 0.8 });
16+
});
17+
18+
it("a split scores below unanimity and drags the combined confidence down with it", () => {
19+
const split = scoreJudgmentAgreement([fail, pass], 0.9);
20+
expect(split).toMatchObject({ agreement: 0.5, sampleCount: 2, uncorroborated: false });
21+
// 0.5 agreement x 0.9 stated = 0.45: a disagreed-upon judgment is recorded as materially less certain
22+
// than the same judgment reached unanimously (0.9 above), which is the whole point of the signal.
23+
expect(split.confidence).toBeCloseTo(0.45, 10);
24+
expect(split.confidence).toBeLessThan(scoreJudgmentAgreement([fail, fail], 0.9).confidence);
25+
});
26+
27+
it("2-of-3 is the modal stance regardless of which side holds it", () => {
28+
expect(scoreJudgmentAgreement([fail, fail, pass], 1).agreement).toBeCloseTo(2 / 3, 10);
29+
expect(scoreJudgmentAgreement([pass, pass, fail], 1).agreement).toBeCloseTo(2 / 3, 10);
30+
expect(scoreJudgmentAgreement([fail, fail, fail], 1).agreement).toBe(1);
31+
});
32+
33+
it("a lone run is UNCORROBORATED, never fabricated unanimity — the budget-degraded arm", () => {
34+
// A single reviewer (single-reviewer plan, or a dual plan whose second leg failed / was budget-cut)
35+
// cannot corroborate itself. It must record a LOWER confidence than a genuinely agreed judgment, not a
36+
// flattering 1.0.
37+
const lone = scoreJudgmentAgreement([fail], 0.9);
38+
expect(lone).toMatchObject({ agreement: UNCORROBORATED_AGREEMENT, sampleCount: 1, uncorroborated: true });
39+
expect(lone.confidence).toBeCloseTo(0.45, 10);
40+
expect(lone.confidence).toBeLessThan(scoreJudgmentAgreement([fail, fail], 0.9).confidence);
41+
});
42+
43+
it("zero samples still refuses to invent a score", () => {
44+
expect(scoreJudgmentAgreement([], 0.9)).toMatchObject({ agreement: UNCORROBORATED_AGREEMENT, sampleCount: 0, uncorroborated: true });
45+
});
46+
47+
it("clamps and totalizes a hostile verbalized confidence instead of propagating it", () => {
48+
expect(scoreJudgmentAgreement([fail, fail], 5).confidence).toBe(1);
49+
expect(scoreJudgmentAgreement([fail, fail], -3).confidence).toBe(0);
50+
expect(scoreJudgmentAgreement([fail, fail], Number.NaN).confidence).toBe(0);
51+
expect(scoreJudgmentAgreement([fail, fail], Number.POSITIVE_INFINITY).confidence).toBe(0);
52+
});
53+
54+
it("INVARIANT: the combined score never exceeds either input, at any sample shape", () => {
55+
const shapes = [[fail], [fail, fail], [fail, pass], [pass, pass], [fail, fail, pass], [pass, fail, fail], [fail, fail, fail], []];
56+
for (const stated of [0, 0.13, 0.5, 0.93, 1]) {
57+
for (const shape of shapes) {
58+
const scored = scoreJudgmentAgreement(shape, stated);
59+
expect(scored.confidence).toBeLessThanOrEqual(scored.agreement + 1e-12);
60+
expect(scored.confidence).toBeLessThanOrEqual(stated + 1e-12);
61+
expect(scored.agreement).toBeGreaterThanOrEqual(0);
62+
expect(scored.agreement).toBeLessThanOrEqual(1);
63+
}
64+
}
65+
});
66+
});

0 commit comments

Comments
 (0)