Skip to content

test(e2e): run demo suites one at a time - #296

Merged
Imod7 merged 1 commit into
mainfrom
test-e2e-serial
Aug 19, 2026
Merged

test(e2e): run demo suites one at a time#296
Imod7 merged 1 commit into
mainfrom
test-e2e-serial

Conversation

@Imod7

@Imod7 Imod7 commented Aug 12, 2026

Copy link
Copy Markdown
Collaborator

Description

Runs the demo end to end suites one package at a time instead of four at once, so the E2E job reports a usable result again.

Changes

product-sdk/package.json, one line, the test:e2e script.

- "test:e2e": "pnpm --filter \"@parity/product-sdk-*-demo\" test:e2e",
+ "test:e2e": "pnpm --workspace-concurrency=1 --no-bail --filter \"@parity/product-sdk-*-demo\" test:e2e",

--workspace-concurrency=1 runs one demo package at a time, which is what avoids the failure.

--no-bail keeps every package reporting. pnpm stops at the first failing package by default, so run one at a time a hang in contracts, third of nine alphabetically, would leave the six after it unrun.

Why these changes

The E2E job is failing often enough on main and the failure only reproduces when demo packages run at the same time. A demo on its own passes 8 runs of 8, the full suite four at once fails 4 of 5, the same suite one at a time passes 6 of 6.

Running them one at a time is therefore the smallest change that makes the job report a usable result.

Testing

Run from product-sdk/:

CI=1 pnpm test:e2e

All nine demo packages should report one after another, in about 2.5 minutes on a clean pass. On a failing run, check that all nine still report, which is what --no-bail is for.

Test results

Measured locally on main plus the host-api-test-sdk 0.12.0 bump.

Scenario Runs Result
Before, four packages at once 5 4 failed, 1 passed
After, one package at a time 6 6 passed, 0 failed
One demo on its own 8 8 passed, 0 failed

This branch has not been measured on CI yet.

Known limitations

This lowers the chance of hitting the hang, it does not remove it. In 2 of the 6 passing runs a package reported one fewer pass and ran about 90 seconds long, which is what Playwright absorbing a failure as a flaky retry looks like. That was never directly observed, so treat it as indicated, not proven.

Two reasons to expect residual flakiness. The bad window varies in length, and some specs burn both Playwright attempts at about 3 minutes each, which no retry policy survives. And a separate known flake, the public Paseo RPC, produces the same "connecting" and 90 second signature.

Six runs is thin, consistent with a residual failure rate between 1 in 4 and 1 in 6. A single red run after this lands does not mean the change failed.

@github-actions

Copy link
Copy Markdown

📦 Bundle size impact

Comparing 2026-08-12T07:44:12.017Z2026-08-12T07:43:21.879Z

No size changes detected. 🟢

@Imod7
Imod7 merged commit ac570dc into main Aug 19, 2026
12 of 13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants