Skip to content

fix(agent): deny approve/merge when a live manual-review hold exists - #3483

Merged
JSONbored merged 1 commit into
mainfrom
fix/orb-manual-review-hold-guard
Jul 5, 2026
Merged

fix(agent): deny approve/merge when a live manual-review hold exists#3483
JSONbored merged 1 commit into
mainfrom
fix/orb-manual-review-hold-guard

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

Closes #3482

Summary

  • Root cause (PR feat(enrichment): leftover console.log / debugger analyzer #3472 incident): Two independent plan-and-execute passes for the same PR (a webhook re-review racing a regate-repair/sweep pass) can each independently evaluate the same head. The loser of the AI-review lock (claimAiReviewLock) gets an instant "AI review already in progress" placeholder, the gate resolves to neutral, and the planner holds the PR for manual review (assignee + manual-review label). The winner keeps running its own, slower, real AI review; when it later resolves cleanly, that pass's own planner independently computes reviewGood && !heldForManualReview from a plan-time snapshot that predates the loser's label — and stages approve + merge. The per-PR actuation lock (claimPrActuationLock, fix(queue): concurrent webhook and sweep jobs for the same PR are not mutually excluded #2129) only serializes each pass's own critical section; it never makes one pass aware of the other's disposition, and the executor's existing freshness guard only re-checks head SHA/open state, never live labels. Net effect: the bot posted "manual review recommended" + assigned + labeled manual-review, then merged the same PR seconds later.
  • Fix: Add a live guard in executeAgentMaintenanceActions (src/services/agent-action-executor.ts), immediately before approve/merge, that re-checks the SAME live PR fetch already performed for the freshness guard (fetchPullRequestFreshness, extended to carry liveLabels) for the configured manualReviewLabel. If present, the action is denied with a clear audit event instead of executing. This is self-scoping to the same head for free — a new commit already fails the freshness guard first — and self-clears the moment a maintainer removes the label, with no new locking, persistence, or planner input required.
  • Threaded manualReviewLabel into the two call sites that can plan/execute an approve or merge: the live webhook/sweep path (src/queue/processors.ts) and the approval-queue accept-replay path (src/services/agent-approval-queue.ts), so a staged approve/merge sitting in the queue is covered too.

Scope

  • The PR title follows type(scope): short summary Conventional Commit format, for example fix(api): restore profile access checks.
  • This PR is focused and does not mix unrelated backend, UI, MCP, docs, dependency, and deploy changes.
  • This follows CONTRIBUTING.md and does not reintroduce GitHub Pages, VitePress, site/, or CNAME.
  • I linked an issue, or this is small enough that the summary explains why an issue is not needed.

Validation

  • git diff --check
  • npm run typecheck (clean)
  • npm run test:coverage (full, unsharded) — 461 test files passed, 9196 tests passed, 7 pre-existing skips, 0 failures
  • Targeted: npx vitest run test/unit/agent-action-executor.test.ts test/unit/pr-freshness.test.ts test/unit/agent-approval-queue.test.ts test/unit/mcp-automation-state.test.ts test/unit/routes-agent-approval.test.ts test/unit/queue.test.ts
  • npm run test:workers / npm run build:mcp / npm run test:mcp-pack / npm run ui:openapi:check / npm run ui:build — not run individually; no worker/MCP/OpenAPI/UI surface touched.
  • New or changed behavior has unit/integration tests for new branches, fallback paths, and sanitizer boundaries — new regression tests for: hold present → approve/merge denied; hold absent → proceeds; non-manual-review labels present → unaffected; custom manualReviewLabel name honored case-insensitively; manualReviewLabel: null disables the guard; guard is scoped to approve/merge only (label/close actions unaffected).

Safety

  • No secrets, wallet details, hotkeys, coldkeys, user PATs, private keys, raw trust scores, private rankings, or private maintainer evidence are exposed.
  • Public GitHub text stays sanitized, low-noise, and does not imply compensation guarantees or optimization tactics.
  • Auth, cookie, CORS, GitHub App, Cloudflare, or session changes include negative-path tests. — N/A; no auth/session/CORS surface touched.
  • API/OpenAPI/MCP behavior is updated and tested where needed. — N/A, no request/response contract change.
  • UI changes use live API data or real empty/error/loading states, not production mock/demo fallbacks. — N/A, no UI change.
  • Visible UI changes include a UI Evidence section below. — N/A, no visible UI change.
  • Public docs/changelogs are updated where needed. — N/A, internal reliability fix, no documented/user-facing behavior change.

Notes

Filed as a same-day follow-up to today's stack-health pass (#3431, #3438, #3440, #3441, #3443, #3456) after a live incident on the self-hosted deployment (PR #3472 was held for manual review and then auto-merged by the same bot pass).

…3472)

Two independent plan-and-execute passes can evaluate the same PR head
(a webhook re-review racing a regate-repair/sweep pass): the loser of
the AI-review lock holds the PR for manual review (label + assign),
while the winner's own slower AI review later resolves cleanly and
stages approve/merge from a plan-time snapshot that predates the
hold. Neither the per-PR actuation lock nor the executor's freshness
guard checks the PR's live labels, so the hold was silently bypassed.

Add a live guard immediately before approve/merge that re-checks the
same freshness fetch for the configured manual-review label and
denies the action if it's present -- self-scoping to the same head
(a new commit already fails freshness first) with no new locking or
persistence required.

Closes #3482
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@loopover-orb loopover-orb Bot added the gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. label Jul 5, 2026
@loopover-orb

loopover-orb Bot commented Jul 5, 2026

Copy link
Copy Markdown
Contributor

Warning

🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨

⏸️ Gittensory review result - manual review recommended

Review updated: 2026-07-05 09:32:28 UTC

10 files · 1 AI reviewer · no blockers · readiness 100/100 · CI pending · blocked

⏸️ Suggested Action - Manual Review

Review summary
This change fixes the right layer for the split-brain manual-review hold: the executor now rechecks live labels from the same freshness fetch immediately before approve/merge, and the configured label is threaded through both live execution and approval-queue replay. The tests exercise the real executor path for default, custom, disabled, and non-approve/merge cases, and `classifyPullRequestFreshness` now carries labels only on the proven-current branch. I do not see a reachable correctness defect in the visible diff.

Nits — 6 non-blocking
  • nit: src/services/agent-action-executor.ts:350 lowercases the configured manual-review label once per live label; normalize `manualReviewLabel` once before `.some()` to keep the comparison easier to read.
  • nit: src/services/agent-action-executor.ts:339 the step-7b comment is much longer than the surrounding guard comments; consider moving the incident detail into the test name/body and keeping the production comment focused on the invariant.
  • nit: test/unit/pr-freshness.test.ts:33 the nameless-label test uses `{}` as a label entry, which is useful defensively but less representative than `{ name: "" }`; add or switch to the empty-string case if that is the malformed payload shape this code is meant to tolerate.
  • src/services/agent-action-executor.ts:350: normalize once, e.g. `const normalizedManualReviewLabel = manualReviewLabel.toLowerCase();` before the `.some()` call.
  • src/services/agent-action-executor.ts:339: trim the production comment to the executor invariant: approve/merge must fail closed when the live current PR carries the configured manual-review hold label.
  • PR author also opened the linked issue — Link an issue that was opened by a different contributor, or provide a rationale for why this self-authored issue represents genuine discovery work.
Signal Result Evidence
Code review ✅ No blockers 1 reviewer
Linked issue ✅ Linked #3482
Related work ✅ No active overlap found No same-issue or scoped active PR overlap found.
Change scope ✅ 20/20 Low review scope from cached public metadata (1 linked issue).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 56 registered-repo PR(s), 46 merged, 586 issue(s).
Contributor context ✅ Confirmed Gittensor contributor JSONbored; Gittensor profile; 56 PR(s), 586 issue(s).
Gate result ✅ Passing No configured blocker found.
Review context
  • Author: JSONbored
  • Role context: owner (maintainer lane)
  • Public audience mode: oss maintainer
  • Lane context: Repository registration is not available in the local Gittensory cache.
  • Public profile languages: Python, TypeScript, JavaScript, Ruby, Go, Kotlin, MDX, Shell
  • Official Gittensor activity: 56 PR(s), 586 issue(s).
  • PR-specific overlap: none found.
Contributor next steps
  • Treat this as maintainer-lane context rather than normal contributor-lane activity.
  • No action.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.

  • Re-run Gittensory review

@codecov

codecov Bot commented Jul 5, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.03%. Comparing base (b5d2956) to head (5972df4).
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #3483   +/-   ##
=======================================
  Coverage   93.03%   93.03%           
=======================================
  Files         299      299           
  Lines       31204    31210    +6     
  Branches    11386    11388    +2     
=======================================
+ Hits        29031    29037    +6     
  Misses       1517     1517           
  Partials      656      656           
Files with missing lines Coverage Δ
src/github/pr-freshness.ts 100.00% <100.00%> (ø)
src/queue/processors.ts 93.42% <ø> (ø)
src/services/agent-action-executor.ts 96.86% <100.00%> (+0.06%) ⬆️
src/services/agent-approval-queue.ts 100.00% <ø> (ø)
🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@JSONbored
JSONbored merged commit a72b4ba into main Jul 5, 2026
10 checks passed
@JSONbored
JSONbored deleted the fix/orb-manual-review-hold-guard branch July 5, 2026 09:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ORB approves/merges a PR it just held for manual review (split-brain, PR #3472)

1 participant