Proposal: a merge queue for main

[Astro-Han]

Why

CI on a PR runs against main as it was when the branch was last pushed. main merges about 47 times a day, and branch protection (strict: false) does not ask for a re-run when it moves. So between "tested" and "merged" there is a gap nothing checks. #3744 shows it: a protocol-compatible-change declaration at epoch 87 against a main at 93, CI green, and the guard that would catch it never runs on main because it is limited to pull_request.

Any check whose answer depends on main has the same gap.

Options

	Tested	Catches it	Cost
Status quo	PR + main at push time	No	0
strict: true	PR + latest main, re-run every time main moves	Yes	At our merge rate a green PR expires faster than CI can re-run, and it expires for every open PR at once. Not convergent.
Merge queue	latest main + the queued batch, once, right before merge	Yes, plus same-batch conflicts	At most one extra test run per merge (batches share a run). ci.yml must handle merge_group.
Guard on push to main	the merged commit, after the fact	After merge only; main goes red until a follow-up	0, one line. Does not close the gap, only makes it visible.

What changes for you with a merge queue

• Maintainers click "Merge when ready" after approving; the PR queues, tests and merges on its own. No second visit.
• A red run in the queue kicks that PR out and notifies the author; main is untouched, PRs behind it are re-batched without it. Fix, then queue again.
• Flaky tests cost more: a flake removes you from the queue instead of a re-run. ci: cut single-job validation time by up to 40% #4523 and similar stability work should land first.
• Squash-only stays. ASF supports the queue through a raw ruleset (apache/seatunnel and apache/texera use it).

Before switching it on

1. ci.yml gets a merge_group: trigger, a BASE_SHA for that event, and the guard's if: widened.
2. Runner budget: the extra runs land on whichever label Reduce pull request runner occupancy without weakening main protection #4451 / CI cost survey: every PR lane measured, and the queue is a runner-label problem #4480 settle on.

Looking for objections or experience with queues elsewhere. If there are none in a week or so, I will open the tracking issue and the ci.yml PR.

简体中文

PR 的 CI 测的是 push 时的 main，之后 main 前进不重测，#3744 的过期 epoch 声明就是这么漏过去的。strict 在每天 47 次合并下不收敛；merge queue 在合并前用最新 main 加排队的一批测一次，红了踢回、main 不动。开关之前要先让 ci.yml 响应 merge_group，并把额外的跑量挂到 #4451/#4480 的 runner 预算上。请提反对意见或别处用队列的经验。

[Phoenix500526]

I think we can introduce the merge queue incrementally instead of rerunning the entire PR workflow for every merge group.

A possible first step would be to classify the current CI checks into two logical groups:

• A: integration-sensitive checks that may produce a different result when the PR is combined with the latest main, such as builds, type checking, unit tests, and stable integration/E2E coverage.
• B: PR-local checks whose result should not change after combining with main, such as linting, formatting, and other static policy checks.

A normal pull request would continue to run A + B, while a merge_group run would execute only A. This is a logical split and does not require two jobs or two runners: the existing job could select the relevant steps based on the event type, preserving the current single-runner design and required test status.

For an initial rollout, I would also suggest:

• setting merge queue build concurrency to 1;
• excluding known flaky tests from the required merge-group checks for now, while continuing to run and track them in PR CI or scheduled workflows;
• keeping the merge-group suite small and stable, since GitHub-hosted PR and merge-group jobs share runner capacity and do not have configurable hard priority;
• retaining the existing cancellation of superseded runs for the same PR.

If the community agrees with this direction, I would be happy to help review the current workflows, classify each check into A or B, and identify which tests are ready to gate the merge queue versus which ones should remain non-blocking until stabilized.