fix(release): self-heal the release-please "untagged, merged" abort#7149
Merged
Conversation
Root cause traced as far as possible: release-please's own createReleases() (the step that actually tags a merged Release PR) sometimes finds nothing to tag, and createPullRequests() then aborts every subsequent run without ever completing the tag -- confirmed live across two separate linked-versions release cycles this session (#7127, #7133), both requiring a manual dispatch of each publish-*.yml workflow plus manually flipping the merged PR's autorelease:pending label. Matches multiple long-open, unresolved upstream issues (googleapis/release-please#1946/#1444/#1406), so it isn't something fixable from this repo's config alone. Adds a reconcile-stale-releases job that runs after every release-please attempt, independent of its outputs: compares each package's committed package.json version against what's actually live on npm, dispatches that package's publish workflow whenever they disagree (the same documented human-override path, automated), and flips any merged PR's autorelease label once every package is confirmed live -- unsticking release-please's own tracking for the next run. Also cleaned up one ancient, silently-stuck instance of the exact same bug (engine v1.0.0, autorelease:pending since long before this session).
Contributor
|
Superagent didn't find any vulnerabilities or security issues in this PR. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #7149 +/- ##
=======================================
Coverage 93.75% 93.75%
=======================================
Files 692 692
Lines 68832 68832
Branches 18781 18781
=======================================
Hits 64535 64535
Misses 3302 3302
Partials 995 995
Flags with carried forward coverage won't be shown. Click here to find out more. |
5 tasks
galuis116
pushed a commit
to galuis116/gittensory
that referenced
this pull request
Jul 18, 2026
…liation job The reconcile-stale-releases job (JSONbored#7149) shipped with two real bugs, found by actually running it live rather than trusting its "success" conclusion: 1. dispatch_and_wait() used `gh run list --limit 1` right after `gh workflow run` to find the just-dispatched run's ID -- unreliable, since the new run isn't always indexed by the API yet. It grabbed a STALE, already-completed prior run instead, reported that run's old "success", and silently never published the new version at all (confirmed live: packages/loopover-engine committed 3.2.3 while npm still had 3.2.2, undetected until manually checking the actual log content -- exactly the kind of unverified-success this whole session has been about not repeating). Fixed by parsing the run ID directly from `gh workflow run`'s own stdout (which prints the created run's URL), with a timestamp-filtered `gh run list` fallback for the rare case that URL isn't returned. Verified this against several real dispatches before shipping. 2. mcp/miner's own isolated pack/smoke-test resolves @loopover/engine from the real npm registry (no local workspace symlink in that bare temp dir), and can transiently ETARGET for a short window after engine's own publish reports success -- npm's registry is CDN-backed, not instantly consistent across every edge node (confirmed live: mcp and miner dispatched at the same moment against the same already-published engine version, mcp's smoke-test hit a already-propagated edge node and succeeded, miner's hit one that hadn't and failed with the exact ETARGET this job exists to fix). Added a settle delay after engine publishes plus automatic retry (with backoff) for its dependents -- this is registry propagation lag, not a genuine test regression, which would fail identically on every retry instead of resolving itself.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Root cause traced as far as possible into release-please's own source:
createReleases()(the step that actually tags a merged Release PR) sometimes finds nothing to tag, andcreatePullRequests()then aborts every subsequent run without ever completing the tag. Confirmed live across two separatelinked-versionsrelease cycles this session (#7127, #7133), both requiring a manual dispatch of eachpublish-*.ymlworkflow plus manually flipping the merged PR'sautorelease: pendinglabel. This matches several long-open, unresolved upstream issues (googleapis/release-please#1946, #1444, #1406 -- same exact message), so it isn't something fixable from this repo's config alone.Adds a
reconcile-stale-releasesjob that runs after everyrelease-pleaseattempt, independent of its outputs (which are unreliable exactly when this bug is in play):package.jsonversion against what's actually live on npm, and dispatches that package's publish workflow whenever they disagree -- the same "bare manual dispatch" human-override path eachpublish-*.ymlalready documents, just triggered automatically instead of by a human noticing red CI on an unrelated PR days later.@loopover/enginefrom the real npm registry, so it fails withETARGETif engine hasn't actually published yet -- confirmed live when I first tried dispatching all three in parallel).autorelease: pendingtoautorelease: tagged, unsticking release-please's own tracking for its next run.Real npm publishes still go through the same
environment: releaserequired-reviewer gate as always -- this doesn't remove that security control, it just eliminates the diagnostic/dispatch/label-cleanup toil around it.Side effect I verified is safe and beneficial: running the version-comparison logic locally against this repo's real current state found one ancient, silently-stuck instance of the exact same bug (
engine v1.0.0's Release PR, stillautorelease: pendingsince long before this session) -- the new job's cleanup step will resolve that too.Test plan
npm run actionlint-- cleanrun:scripts and ranbash -non each -- syntax valid