Flag stale PR validation builds

[ericstj]

Merging PRs with stale results can introduce build breaks or test failures because the code might have been validated on an old codebase.

We should make it obvious when we have build results that are older than a couple weeks - or maybe older by a number of commits. That way we avoid folks accidentally merging with stale results.

I know AzDo has a policy setting for this - maybe github does as well?

[ghost]

Tagging subscribers to this area: @dotnet/runtime-infrastructure
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Merging PRs with stale results can introduce build breaks or test failures because the code might have been validated on an old codebase.

We should make it obvious when we have build results that are older than a couple weeks - or maybe older by a number of commits. That way we avoid folks accidentally merging with stale results.

I know AzDo has a policy setting for this - maybe github does as well?

Author:	ericstj
Assignees:	-
Labels:	area-Infrastructure
Milestone:	-

[ViktorHofer]

GH offers the branch must be "up-to-date" setting but that doesn't work well in high traffic repositories. You would constantly need to sync to latest and wait for CI to finish. By that time, your HEAD likely isn't in sync with runtime's HEAD anymore.

In addition to that, this feature is coupled with the "require status check to pass before merge" feature which we don't have enabled in runtime.

[vcsjones]

This could be solved with a GitHub action. Have the action run on commits to main (A PR was merged) and have the action enumerate all open pull requests and their reviews, and dismiss the review if its older than whatever you configure it to be.

If this sounds interesting I can help develop the action.

[jkoritzinsky]

It might be worthwhile looking into the GitHub Merge Queue feature to support us here once we finish the Merge-on-Green initiative.

[akoeplinger]

Instead of a policy a quick reminder comment by @msftbot would probably help too and I think we have all the infrastructure in place for this since we already post the "no recent activity" comment and close PRs after a while.