Cut CI setup and tarball smoke time: skip npm registry round trips - #3046
Merged
Conversation
Every CI job bootstrapped pnpm through pnpm/action-setup, which runs `npm ci` and `pnpm self-update` against the npm registry: 30 s at best, and 7 minutes on runs where the registry stalled, on every job of the run at once. Setup workspace ran at a median of 140-160 s per job across the last 30 runs, against 25 s for a job that got a fast registry. The tarball smoke, the critical path of nearly every run at a median of 382 s on Linux, spent most of that in `npx --package <tarball>`: unlike the smoke's `npm install`, the npx calls never passed `--no-audit`, so each one blocked on the registry's bulk advisories endpoint. Locally that single call took 211 s of a 246 s smoke. - The setup action downloads the pinned pnpm release binary from GitHub Releases, verified against a committed sha256 list, instead of using pnpm/action-setup. - The smoke passes `--no-audit --no-fund` to both npx invocations. - The smoke prints per-stage timings so the next regression is attributable from the CI log instead of needing a local reproduction. Local smoke: 246 s before, 40 s after. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Collaborator
Author
|
CI timings for this PR's run (33830272710) against the last 30
Per-stage smoke timings now appear in the log; on Linux the
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Human comments
What was wrong
CI wall time ranged from 9 to 30 minutes for the same workload, and the variance was almost entirely time spent waiting on the npm registry. Every job bootstraps pnpm through
pnpm/action-setup, which runsnpm ciand thenpnpm self-updateagainst the registry: 30 s on a good day, and 7 minutes on the runs where the registry stalled (the stall hits every job of a run at once, so a whole run slips). Across the last 30 runs the Setup workspace step had a median of 140–160 s per job and a p90 of 300–450 s, against 25 s when the registry answered promptly.The tarball smoke is the critical path of nearly every run (median 382 s on Linux, 345 s on macOS, max 940 s). Profiling it locally with
--timingshowed thenpx --yes --package <tarball>call blocking for 172 s onPOST /-/npm/v1/security/advisories/bulk: the smoke'snpm installpasses--no-audit, but neither of itsnpxinvocations did, so each ran an audit against the registry's slow advisories endpoint.What changed
.github/actions/setup-workspace: pnpm is now downloaded as the pinned release binary from GitHub Releases (install-pnpm.sh, checked against the committedpnpm-9.15.0.sha256), replacingpnpm/action-setup. BumpingPNPM_VERSIONneeds a new checksum file; the script fails with instructions when one is missing.packages/bb-app/scripts/smoke-tarball.mjs: bothnpxcalls pass--no-audit --no-fund, matching the existingnpm install. The smoke also prints per-stage timings (bb-app tarball smoke: <stage> <seconds>) so the next regression is attributable from the CI log.Other workflows that use
pnpm/action-setupdirectly (deploys, publish, desktop builds) are unchanged; they can adopt the same script later.How you verified
install-pnpm.shrun locally with fakeGITHUB_PATH/GITHUB_ENV: installs pnpm 9.15.0 andpnpx; exits 1 with a clear message for a version without a checksum file and for a checksum mismatch.pnpm exec turbo run smoke:tarball --filter=bb-applocally: 246 s before (211 s in the npx step), 40 s after (npx step 4.6 s).