feat(council): several reviewers on one lane, merged monotonically

[senamakel]

Off by default. The machinery for running more than one reviewer over a lane's evidence and folding what they say into one review — the third slice of the agent-council plan, after local-review (#85) and the eval harness (#86).

It is not a second opinion

src/falsify is that, and its README explains why asking a second model "are these correct?" deletes the best half of a review. A council runs the other way: more reviewers so that more is found, with agreement used only to rank what comes back.

Agreement raises confidence. It never gates posting.

The obvious design is a quorum — post when two of three agree — and it is the falsification failure in a different hat. Worse here, because the architecture is deliberately built so reviewers don't overlap: ISOLATION_CLAUSE exists precisely to stop N conversations reporting the same cross-file problem. A reviewer with a different angle (or later, a tool belt the others lack) finds things the others structurally cannot see, and gating on agreement deletes exactly those.

So the merge is monotone — no finding is ever worse off for the council having run, asserted as a property test over a grid of confidences. Corroboration raises confidence by noisy-OR, capped at 0.99 because three models agreeing is not a certainty any of them claimed, and breaks ties last in lane_proposal's sort so a corroborated finding survives max_comments truncation without outranking a more serious one.

Grouping deliberately is not the fingerprint

Finding::fingerprint hashes rule, which is model-authored free text — two reviewers on one missing bounds check write unchecked-index and missing-bounds-check, and grouping on it would post both. agree::corroborates uses same lane, same file, ranges overlapping within three lines, and never compares wording (two agents describing one defect word it differently by construction — that's the point of running more than one).

That looseness is used only for grouping. Finding::identity stays exactly as anchor::stamp computed it, and a merge keeps the first sighting's identity — it is what the tinysweeper:fp= marker carries and what suppression reads back, so letting the representative bring its own would repost a finding already answered.

The representative is verbatim

Highest confidence then severity, taken whole. Nothing blended or rewritten — a merge step that can author text is a second reviewer nobody gated. Severity is the highest anyone assigned: merging must not talk a review down.

What the eval harness caught

Worth calling out, because it's the first time the corpus paid for itself. The first draft merged findings within one reviewer's output as well as across reviewers, so two findings three lines apart silently became one — a behaviour change at a single agent, which destroys the property the entire rollout rests on.

cargo test failed on ts-0068, where the critique lane legitimately raises two findings on adjacent lines of one file. Now only earlier reviewers may absorb a later finding, at most once each, with a regression test naming the case.

Security surface

• [council] is not overridable by a reviewed repository (council_keys_are_not_overridable_by_a_reviewed_repository, plus a test that a repo config trying to convene a council changes nothing and says so). Every key spends the operator's money or decides what a model is told.
• Personas are names, never text. config::remote already keeps repository prose out of the instruction position; a settable persona would be a second door. They're &'static str in council::persona, and an unknown name is a configuration error from tinysweeper check.
• Tests assert every persona tells the reviewer others are running, and that none of them tries to override the lane's own subject.

Cost

Falsification now runs once over the merged set rather than per reviewer — a reject-only filter given more inputs in one pass has identical semantics at a fraction of the calls. Each agent gets its own cache stream because the persona sits in the prefix; measured, that costs much less than expected.

Verification

cargo fmt --all -- --check
cargo clippy --locked --all-targets -- -D warnings
cargo test --locked                       # 1284 offline tests, 32 new
cargo check --locked --all-features --all-targets

The disabled path is proved unchanged empirically, not just argued. The committed evals/ corpus replays byte-identically against its existing cassettes, and those are keyed on full prompt text — a prompt that had moved by one byte would miss.

Live two-agent council (correctness + integration) over a one-file range:

$0.0007 · 6,604 in / 2,515 out · 5,812 cached (88%) · z-ai/glm-5.2

88% cache hit despite the per-persona prefix split, because the shared rules and policy dominate the prefix.

Not in this PR

Whether a second agent is worth the money. That is a question tinysweeper eval answers, and it cannot answer it yet — both corpus cases are forbidden-only, so recall and precision both render n/a. council.enabled = false ships in defaults.toml and stays there until the corpus has cases that can measure the difference. corroboration is a separate switch precisely so the merge could be proved a no-op before a second agent is what is being judged.

[coderabbitai]

Warning

Review limit reached

@senamakel, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 16 minutes

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 9b4ff2e2-1825-4e62-9bf3-7ff762bafe21

📥 Commits

Reviewing files that changed from the base of the PR and between 2162fe5 and 00cdb11.

📒 Files selected for processing (29)

• docs/modules/council/README.md
• src/app/apply.rs
• src/app/review.rs
• src/config/defaults.toml
• src/config/remote_test.rs
• src/config/types.rs
• src/config/validate.rs
• src/council/agree.rs
• src/council/agree_test.rs
• src/council/merge.rs
• src/council/merge_test.rs
• src/council/mod.rs
• src/council/mod_test.rs
• src/council/persona.rs
• src/eval/score_test.rs
• src/falsify/test.rs
• src/findings/anchor.rs
• src/findings/render.rs
• src/findings/suggest_test.rs
• src/findings/types.rs
• src/harness/cassette.rs
• src/harness/prompt.rs
• src/harness/schema.rs
• src/lanes/anchor.rs
• src/lanes/critique.rs
• src/lanes/description.rs
• src/lanes/mod.rs
• src/lib.rs
• src/overview/test.rs

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}