Skip to content

ui tests: redistribute from 4 into 6 shards - #234

Merged
lukaskollmer merged 5 commits into
mainfrom
lukas/ui-test-six-shards
Nov 5, 2026
Merged

lukaskollmer merged 5 commits into
mainfrom
lukas/ui-test-six-shards

Conversation

@lukaskollmer

Copy link
Copy Markdown
Collaborator

♻️ Current situation & Problem

CI now has 6 self-hosted runners, but the UI tests are split into only 4 shards. A failing test also reruns its whole shard, and testECG currently fails its first attempt (#233), so one shard takes about 31 min instead of 16.

⚙️ Release Notes

  • Split the UI tests into 6 CI shards instead of 4. No app changes.

Before merging: remove these required checks from the Main ruleset. The shard names changed, so they will never report:

  • App UI Tests (Run; Consent Major, Other, Timed Walk, Regression)
  • App UI Tests (Run; Consent Minor, Onboarding, Basic Usage)
  • App UI Tests (Run; Consent Review and Patch, Prompted Actions, Tasks)
  • App UI Tests (Run; Consent Skip, Health Dashboard, Scheduled)

App UI Tests / Test using xcodebuild or run fastlane stays required and still fails if any shard fails.

📚 Documentation

Shards are balanced on median test times and listed longest first:

Shard Contents Test time
01 Consent Review and Minor, Basic Usage 11.9 min
02 Consent Skip, Prompted Actions 11.8 min
03 Other, Timed Walk 11.6 min
04 Consent Patch, Onboarding 11.6 min
05 Consent Major, Health Dashboard, Regression, Study Participation 10.8 min
06 Scheduled, Tasks 4.7 min

Shard 06 is short on purpose: it contains testECG, so a whole-shard retry takes 9.4 min and finishes before the other shards.

Per-run data puts the slowest shard at 12–13 min, down from about 31 min. UI tests from start to finish should drop from about 40 min to about 22–23 min. The whole CI run should drop from about 42 min to about 34 min, limited by Utils Unit Tests (Linux Release).

Decisions:

  • Timings use only runs since 2026-11-02 on GitHub Runner 1–6, because earlier runs had a different runner count.
  • The Main ruleset change is made manually before merging.

✅ Testing

  • actionlint passes on build-and-test.yml.
  • A scan of the test sources places all 37 tests in exactly one shard.
  • Not run locally: the UI tests and validate-ui-test-shards.sh, which needs built test products. CI runs both on this PR.
  • The timings above are predictions from earlier runs. This PR's run is the first real measurement.

Code of Conduct & Contributing Guidelines

By creating and submitting this pull request, you agree to follow our Code of Conduct and Contributing Guidelines:

🤖 Generated with Claude Code

Use all six self-hosted runners. Shards are balanced on timings from
runs since 2026-11-02. A failing test reruns its whole shard, so
TaskHandlingTests sits in a short shard to keep a testECG retry off the
critical path.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Nov 4, 2026 •

Copy link
Copy Markdown

Important

  • Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: dd8eeba5-1f11-41c6-b37d-e0784e05d011
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@lukaskollmer
lukaskollmer changed the base branch from main to lukas/update-deps November 5, 2026 21:38
@lukaskollmer
lukaskollmer changed the base branch from lukas/update-deps to main November 5, 2026 21:38
@lukaskollmer lukaskollmer self-assigned this Nov 5, 2026
@lukaskollmer lukaskollmer added the enhancement New feature or request label Nov 5, 2026
@github-project-automation github-project-automation Bot moved this to In Progress in MyHeartCounts Nov 5, 2026
@lukaskollmer lukaskollmer changed the title Split UI tests into six CI shards ui tests: redistribute from 4 into 6 shards Nov 5, 2026
lukaskollmer and others added 3 commits November 5, 2026 23:17
#235 replaced the account sheet's logout button with a "Log Out" button
and alert, so testLogout could no longer find "Logout".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#235 moved these actions into a single `alert(_:item:)`. AsyncButton
runs its action in a `.task`, and the `item:` alert removes the button
as soon as tapping it dismisses the alert, so neither action ever ran.
testLogout caught this: the alert closed, but the account sheet stayed
open and the user stayed signed in.

Give each action its own `isPresented:` alert, which is what Grove's
AccountOverview does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The AsyncButton never fit here: the alert dismisses itself on tap, so
the button can't show progress, and its task is tied to the alert's
content. Use a plain Button that starts a Task owned by the sheet, so
the action outlives the alert and a single `item:` alert covers both
actions again.

Also disable the account section while an action is running, since
closing the alert would otherwise re-enable its buttons.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@lukaskollmer
lukaskollmer merged commit 51440c9 into main Nov 5, 2026
25 of 27 checks passed
@lukaskollmer
lukaskollmer deleted the lukas/ui-test-six-shards branch November 5, 2026 23:42
@github-project-automation github-project-automation Bot moved this from In Progress to Done in MyHeart Counts Project Planning Nov 5, 2026
@github-project-automation github-project-automation Bot moved this from In Progress to Done in MyHeartCounts Nov 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

1 participant