Skip to content

Commit dcaa3cb

Browse files
zoomote[bot]roomoteedelauna
authored
[Chore] Harden E2E workflow against VS Code binary download failures (#1045)
* ci: harden E2E workflow against VS Code binary download failures Add restore-keys fallback to the VS Code test binary cache, probe the VS Code update API before running tests, fall back to a stale cached binary (via VSCODE_VERSION) when the CDN is unreachable, and retry the mocked E2E step so transient network blips don't fail the merge queue. Skip writing the pass marker when the stale-binary fallback was used. Closes #1044 * ci: scope E2E retries to binary download, probe CDN endpoint, prune stale binaries Address review feedback on #1045: - Retry only the VS Code binary download (its own step), so genuine test failures fail fast instead of re-running the whole suite. - Probe the archive CDN endpoint (via the 302-redirect HEAD) in addition to the update API, so the stale-binary fallback also engages when only the CDN host is down. - Restore/save the binary cache with restore+save actions and prune older vscode-linux-x64-* dirs so each saved entry keeps one binary instead of growing with every version bump; skip the save when the stale fallback was used. - Emit a ::warning:: when endpoints are unreachable with no cached binary, and echo why the pass marker is skipped on fallback runs. --------- Co-authored-by: Roomote <roomote@roomote.dev> Co-authored-by: edelauna <54631123+edelauna@users.noreply.github.com>
1 parent 1ae8b5b commit dcaa3cb

1 file changed

Lines changed: 101 additions & 4 deletions

File tree

‎.github/workflows/e2e.yml‎

Lines changed: 101 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -45,9 +45,100 @@ jobs:
4545
VERSION=$(node -p 'require("./apps/vscode-e2e/package.json").devDependencies["@types/vscode"]')
4646
echo "version=$VERSION" >> $GITHUB_OUTPUT
4747
48-
- name: Cache VS Code test binary
48+
- name: Restore VS Code test binary cache
4949
if: github.event_name != 'pull_request' || steps.e2e-marker.outputs.cache-hit != 'true'
50-
uses: actions/cache@0057852bfaa89a56745cba8c7296529d2fc39830 # v4
50+
id: vscode-cache
51+
uses: actions/cache/restore@0057852bfaa89a56745cba8c7296529d2fc39830 # v4
52+
with:
53+
path: |
54+
apps/vscode-e2e/.vscode-test/
55+
key: vscode-test-${{ runner.os }}-${{ steps.vscode-ver.outputs.version }}-v1
56+
# Fall back to the most recent stale binary so a version bump or cache
57+
# eviction doesn't force a download when the VS Code CDN is unreachable.
58+
restore-keys: |
59+
vscode-test-${{ runner.os }}-
60+
61+
- name: Probe VS Code download endpoints
62+
if: (github.event_name != 'pull_request' || steps.e2e-marker.outputs.cache-hit != 'true') && steps.vscode-cache.outputs.cache-hit != 'true'
63+
id: vscode-probe
64+
# The version-resolution API and the archive CDN are different hosts. The
65+
# archive URL 302-redirects to the CDN, so a HEAD probe that follows the
66+
# redirect covers both. The fallback engages when either is unreachable.
67+
run: |
68+
API_OK=false
69+
CDN_OK=false
70+
if curl -sf --max-time 10 https://update.code.visualstudio.com/api/releases/stable > /dev/null; then
71+
API_OK=true
72+
fi
73+
if curl -sfIL --max-time 20 -o /dev/null "https://update.code.visualstudio.com/${{ steps.vscode-ver.outputs.version }}/linux-x64/stable"; then
74+
CDN_OK=true
75+
fi
76+
if [ "$API_OK" = "true" ] && [ "$CDN_OK" = "true" ]; then
77+
echo "reachable=true" >> "$GITHUB_OUTPUT"
78+
else
79+
echo "reachable=false" >> "$GITHUB_OUTPUT"
80+
echo "::warning::VS Code download endpoints unreachable (api=$API_OK, cdn=$CDN_OK); will try the stale cached binary"
81+
fi
82+
83+
# @vscode/test-electron only skips its live version-resolution request when the
84+
# requested version already exists in .vscode-test/. On an exact-key miss it calls
85+
# the update API before its download retry loop, so an unreachable CDN fails the
86+
# job before any test runs. When the endpoints are down, point the runner at the
87+
# stale binary restored above (runTest.ts honors VSCODE_VERSION) so the suite
88+
# still runs.
89+
- name: Fall back to stale VS Code binary when CDN is unreachable
90+
if: (github.event_name != 'pull_request' || steps.e2e-marker.outputs.cache-hit != 'true') && steps.vscode-cache.outputs.cache-hit != 'true' && steps.vscode-probe.outputs.reachable == 'false'
91+
id: vscode-fallback
92+
run: |
93+
STALE=$(ls -d apps/vscode-e2e/.vscode-test/vscode-linux-x64-* 2>/dev/null | sort -V | tail -n 1 || true)
94+
if [ -n "$STALE" ]; then
95+
echo "VSCODE_VERSION=${STALE##*-}" >> "$GITHUB_ENV"
96+
echo "used=true" >> "$GITHUB_OUTPUT"
97+
echo "VS Code download endpoints are unreachable; falling back to cached VS Code ${STALE##*-}"
98+
else
99+
echo "::warning::VS Code download endpoints are unreachable and no cached binary is available; the download step will retry but may fail"
100+
fi
101+
102+
# Retry only the binary download: version resolution and the archive fetch are
103+
# the network-fragile parts. Genuine test failures should fail fast, so the
104+
# test step itself is deliberately not retried.
105+
- name: Download VS Code test binary
106+
if: github.event_name != 'pull_request' || steps.e2e-marker.outputs.cache-hit != 'true'
107+
id: vscode-download
108+
run: |
109+
cd apps/vscode-e2e
110+
for attempt in 1 2 3; do
111+
if node -e "const { downloadAndUnzipVSCode } = require('@vscode/test-electron'); const version = process.env.VSCODE_VERSION || require('./package.json').devDependencies['@types/vscode']; downloadAndUnzipVSCode(version).catch((err) => { console.error(err); process.exit(1); });"; then
112+
exit 0
113+
fi
114+
echo "VS Code download attempt ${attempt} failed"
115+
if [ "$attempt" -lt 3 ]; then
116+
sleep 15
117+
fi
118+
done
119+
exit 1
120+
121+
# restore-keys can bring back older binaries alongside the one just
122+
# downloaded; keep only the effective version so each saved cache entry
123+
# stays at one binary instead of growing with every version bump.
124+
- name: Prune stale VS Code binaries
125+
if: github.event_name != 'pull_request' || steps.e2e-marker.outputs.cache-hit != 'true'
126+
run: |
127+
EFFECTIVE="${VSCODE_VERSION:-${{ steps.vscode-ver.outputs.version }}}"
128+
for dir in apps/vscode-e2e/.vscode-test/vscode-linux-x64-*; do
129+
[ -e "$dir" ] || continue
130+
if [ "$(basename "$dir")" != "vscode-linux-x64-$EFFECTIVE" ]; then
131+
echo "Pruning stale VS Code binary $(basename "$dir")"
132+
rm -rf "$dir"
133+
fi
134+
done
135+
136+
# Skip the save when the stale-binary fallback ran: the pinned-version key
137+
# must not be populated with an older binary.
138+
- name: Save VS Code test binary cache
139+
if: (github.event_name != 'pull_request' || steps.e2e-marker.outputs.cache-hit != 'true') && steps.vscode-cache.outputs.cache-hit != 'true' && steps.vscode-fallback.outputs.used != 'true'
140+
continue-on-error: true
141+
uses: actions/cache/save@0057852bfaa89a56745cba8c7296529d2fc39830 # v4
51142
with:
52143
path: |
53144
apps/vscode-e2e/.vscode-test/
@@ -59,12 +150,18 @@ jobs:
59150
if: github.event_name != 'pull_request' || steps.e2e-marker.outputs.cache-hit != 'true'
60151
run: xvfb-run -a pnpm --filter @roo-code/vscode-e2e test:ci:mock
61152

153+
- name: Explain skipped mocked E2E pass marker
154+
if: steps.e2e-marker.outputs.cache-hit != 'true' && steps.run-e2e.outcome == 'success' && steps.vscode-fallback.outputs.used == 'true'
155+
run: echo "Skipping mocked E2E pass marker because tests ran against a stale cached VS Code binary (VS Code download endpoints unreachable)."
156+
62157
- name: Write mocked E2E pass marker
63-
if: steps.e2e-marker.outputs.cache-hit != 'true' && steps.run-e2e.outcome == 'success'
158+
# Skip when the stale-binary fallback ran: a pass against an older VS Code
159+
# must not mint a marker for the intended-version source hash.
160+
if: steps.e2e-marker.outputs.cache-hit != 'true' && steps.run-e2e.outcome == 'success' && steps.vscode-fallback.outputs.used != 'true'
64161
run: mkdir -p .cache/e2e-pass && date -u > .cache/e2e-pass/passed
65162

66163
- name: Save mocked E2E pass marker
67-
if: steps.e2e-marker.outputs.cache-hit != 'true' && steps.run-e2e.outcome == 'success'
164+
if: steps.e2e-marker.outputs.cache-hit != 'true' && steps.run-e2e.outcome == 'success' && steps.vscode-fallback.outputs.used != 'true'
68165
continue-on-error: true
69166
uses: actions/cache/save@0057852bfaa89a56745cba8c7296529d2fc39830 # v4
70167
with:

0 commit comments

Comments
 (0)