ci: teach CI the merge_group event and add the merge queue (ADR 0050) - #1867
Conversation
Main took 775 merges in 30 days and `already-tested` misses about half of them, because the branch was one or two commits behind at merge. Those misses are the honest report that half of what lands on main is a tree no CI run has ever seen. A merge queue builds the tree the merge will produce and merges only if it passes. CI learns a third event. `gate.py`'s PROBES table is the authority: a merge group runs both probes, classifying the group like a PR and skipping it when its tree is one a PR run already tested. The group publishes `tested-tree` and main's push finds it through the queue's `gh-readonly-queue/.../pr-<n>-<sha>` branch name, so a queued merge costs one full run rather than two. `require-checks.py --merge-queue` adds the ruleset rule. It refuses to apply while main's workflows lack a `merge_group:` trigger, because that ordering is the one failure with no visible cause: the queue waits for a check that never starts and ejects the PR with nothing red to explain it. A new gate test keeps the workflow's triggers and PROBES from drifting apart. The queue builds one group at a time and batches up to five PRs into it: the org is on the free plan's 20 concurrent jobs and one full CI run is 19 of them. The review requirement drops to zero in the same flag, because a requirement no solo-authored PR can satisfy made every merge `--admin`, and an admin merge skips the queue. The ruleset is external state, so this commit does not turn the queue on. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RJBSE57BbSEDF5s5fvWEga
a3ba2c8 to
6c5be2c
Compare
|
Rebased onto main and renumbered ADR 0049 → 0050: #1839 landed its own 0049 ( Worth noting what happened here, since it is the PR's own argument: this branch was opened, and by the time it was pushed main had moved 11 commits and the PR was |
`--merge-queue` no longer drops `required_approving_review_count` to zero. The two gates answer different questions: the queue answers whether a tree builds, review answers whether anyone read it, and on a repository where most PRs are written by agents the second is the one worth keeping. Enabling a machine that checks the first is not a reason to stop asking the second. The cost is accepted rather than engineered around. GitHub will not enqueue a PR whose merge requirements are unmet, so an unreviewed PR does not fail — it never enters the queue, and `--auto` looks like it did nothing. CLAUDE.md and the CI README both say to check `reviewDecision` first when a queued PR seems stuck. ADR 0050 records this as reconsidered: dropping review to zero moves from the decision into the rejected alternatives, with why. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RJBSE57BbSEDF5s5fvWEga
|
Reversed the review decision. The reasoning in ADR 0050 now says why: the queue answers does this tree build, review answers did anyone read it, and on a repo where most PRs are written by agents the second is the one worth keeping. Dropping review to zero moved from the decision into the rejected alternatives. The cost is accepted rather than engineered around: GitHub won't enqueue a PR whose merge requirements are unmet, so an unreviewed PR never enters the queue — it doesn't fail, 23 policy tests still green; the new one asserts the queue flag leaves the |
The index row carries each ADR's frontmatter description, so editing one without rerunning scripts/decisions-index.sh leaves the two disagreeing and the decisions workflow red. `okf validate` alone does not catch it; the index drift check does, which is why this went green locally and failed in CI. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RJBSE57BbSEDF5s5fvWEga
What
Teaches CI the
merge_groupevent and adds a GitHub merge queue, perADR 0050. The ruleset is external state, so merging this does not turn the
queue on — see Activating below.
Why
Main took 775 merges in 30 days (~26/day, peak 55).
already-testedskipsmain's re-run only when the pushed tree equals a tree some PR run actually
tested, and over 100 recent pushes it hit 53 times. The other ~47% are the
honest report that half of what lands on main is a tree no CI run has ever
seen.
That is not a correctness crisis — main failed 4 times in 300 push runs (1.3%).
The cost is a full re-run on main per miss, a delayed deploy, and
build-in-PR-and-retag being impossible while the merged tree isn't the tested
tree.
How
gate.py'sPROBEStable is the authority on what each event owes:pull_requestchangesmerge_groupchanges+already-testedpushalready-testedA merge group publishes the
tested-treeartifact and main's push finds itthrough the queue's
gh-readonly-queue/<base>/pr-<number>-<sha>branch name —the only link back from a squashed commit to the run that tested it. So a
queued merge costs one full run, not two.
Sizing
One group at a time, batching up to five PRs into it. The org is on the free
plan's 20 concurrent jobs and one full CI run is 19 of them, so
speculating two groups deep cannot run two groups; it queues the second behind
the first while starving every open PR.
One rule comes off, one deliberately stays
merges into", which the queue now does directly.
answers did anyone read it. On a repo this agent-heavy the second is worth
keeping, so
--merge-queuedoes not touch it.The accepted cost: GitHub will not enqueue a PR whose merge requirements are
unmet, so an unreviewed PR never enters the queue — it doesn't fail, and
--autolooks like it did nothing.CLAUDE.mdand the CI README both say tocheck
reviewDecisionfirst when a queued PR seems stuck.Activating (after this merges)
python3 scripts/ci/require-checks.py --merge-queue # preview python3 scripts/ci/require-checks.py --merge-queue --apply--applyrefuses while main's workflows lack amerge_group:trigger. Thatordering is the one failure with no visible cause: the queue waits out
check_response_timeout_minutesfor a check that never starts and ejects thePR, with nothing red anywhere. It also flips
allow_auto_merge, which iscurrently
false— without it--autoerrors and nothing can reach the queue.Verification
when reverted: removing the
merge_grouptrigger fails the newtrigger/PROBES parity test; dropping the queue-run lookup fails main's
reuse test; letting a queue run prove itself from a sibling queue run fails
its own test.
actionlintandshellcheckclean on all touched files.okf validate decisionsvalid, 0 errors; index regenerated.Note for whoever merges
This touches
.github/workflows/, which needs theworkflowOAuth scope —see the two-accounts note. Merge normally (
--admin, since the queue is noton yet); the activation command above is the step that switches it on.
🤖 Generated with Claude Code
https://claude.ai/code/session_01RJBSE57BbSEDF5s5fvWEga