Skip to content

e2e: changed-file inference — run only relevant e2e tests per PR #587

Description

@Anton-Fil

Overview

Currently every PR runs the same fixed set of smoke tests regardless of what was changed. A PR that only touches src/components/apikey/ still runs all 58 smoke tests. Changed-file inference would run only the test relevant to the modified files, reducing PR gate time from ~11 min to ~2-5 min in most cases.

Part of epic #584


Problem

The current smoke suite always runs the same 6 test files on every PR. This is safe but slow — a small change to one component triggers the full smoke run even though most tests are completely unrelated to the change.


Solution

Add a suite-router script that:

  1. Gets the list of files changed in the PR via git diff
  2. Maps changed files to relevant test files using a known mapping
  3. Runs only the mapped tests
  4. Falls back to the full smoke suite if no mapping is found or if shared files are changed (e.g. src/utils/, src/hooks/)

Example mapping:
src/components/apikey/ → apikey-lifecycle.spec.ts
apikey-approvals.spec.ts
apiproduct-apikeys-tab.spec.ts

src/components/apiproduct/ → apiproduct-crud.spec.ts
apiproduct-overview-tab.spec.ts
apiproduct-details-tabs.spec.ts
api-product-list.spec.ts

src/components/ → rbac.spec.ts (always run — RBAC is global)

e2e/ → full smoke suite (test infra changed)

Fallback: if changed files don't match any mapping or include shared
utilities → run full smoke suite to stay safe.


Reference

This approach is already implemented in the mcp-gateway team:
Kuadrant/mcp-gateway#1197

Their build/suite-router.sh can be used as a reference implementation.


Main risk

If the file→test mapping is wrong or incomplete a bug could slip through undetected. Requires careful mapping and a reliable fallback to the full smoke suite. The label system (see related issue) should be in place first as it makes the mapping more maintainable.


Location

New file: build/suite-router.sh
Modified: .github/workflows/e2e.yaml


Dependencies

Requires the label system to be implemented first (#585 ).
The suite-router uses @smoke/@nightly Playwright tags to determine
which tests are relevant for each component mapping.


Acceptance Criteria

  • suite-router.sh reads changed files from git diff
  • Script maps changed files to relevant test files
  • Unknown or shared file changes fall back to full smoke suite
  • PR gate time reduced to ~2-5 min for targeted changes
  • Full smoke suite still runs as fallback when mapping is unclear
  • Label system (e2e: add @smoke/@nightly Playwright tags to replace file-based CI split #585 ) is implemented first

Metadata

Metadata

Assignees

Labels

triage/acceptedReviewed, prioritised, and ready for work

Type

No type

Projects

Status
Done

Relationships

None yet

Development

No branches or pull requests

Issue actions