Skip to content

test(e2e): use neutral iOS slot simulators #9334

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
mikehardy wants to merge 1 commit into
base: main
Choose a base branch
from
Open

test(e2e): use neutral iOS slot simulators #9334

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/scripts/resolve-ios-simulator-name.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
#!/usr/bin/env bash
# Resolve the iOS simulator *device name* for boot-simulator.sh (CI + Jet reboot).
# Prefer RNFB_IOS_SIMULATOR (slot env already exports `RNFB E2E iOS slot-N`;
# Prefer RNFB_IOS_SIMULATOR (slot env already exports `RN E2E iOS slot-N`;
# serial default follows scripts/e2e/create-ios-simulators.sh / E2E_DEFAULT_IOS_SIMULATOR /
# RNFB_IOS_BASE_SIMULATOR). Detoxrc fallback skips comment lines so a comment
# mentioning iPhone cannot become the device name.
Expand Down
4 changes: 3 additions & 1 deletion okf-bundle/ci-workflows/detox-patches.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,12 @@ timestamp: 2026-08-26T00:00:00Z

# Detox yarn patches

E2E runs on **Detox 20.51.0** (`tests/package.json`), applied via Yarn Berry patch:
E2E runs on **Detox 20.51.0** (`tests/package.json`). This file inventories the **Yarn Berry** patch:

`.yarn/patches/detox-npm-20.51.0-3e13b6e309.patch`

**Not this file:** iOS neutral-slot exact-base-or-fail allocation is [`tests/patches/detox+20.51.0.patch`](../../tests/patches/detox+20.51.0.patch) via `tests` `prepare` (`patch-package`). Contract: [running e2e § iOS slot simulators](../testing/running-e2e.md#ios-slot-simulators). Do not add that hunk here.

Patches are in-repo. Prefer direct patch-file edits or headless workflow; `yarn patch-commit` may prompt and fail in non-interactive shells.

## Inventory
Expand Down
2 changes: 1 addition & 1 deletion okf-bundle/monorepo-tooling/prepare-and-cache.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -105,7 +105,7 @@ Add `nx.json`; keep the `yarn lerna:prepare` entrypoint name, with `NX_NO_CLOUD`

Most packages' `prepare` is bob transpile (`lib/**` → `dist/**`) and is correctly cached under [MonoTool-AD-11](architecture-decisions.md#monotool-ad-11--scope-prepare-cache-inputs-with-a-jssource-namedinput--accepted). The **tests** and **tests-macos** workspaces are different: their `prepare` script is **`patch-package`**, which mutates that app's `node_modules/**`.

- **`tests/`** (mobile RN **0.86.2**): still runs `patch-package` for non-fmt patches (e.g. `@firebase+rules-unit-testing`). Mobile RN ships fmt **12.1.0** upstream; there is **no** `tests/patches/react-native+*.patch` fmt bump.
- **`tests/`** (mobile RN **0.86.2**): still runs `patch-package` for non-fmt patches (e.g. `@firebase+rules-unit-testing`, [`detox+20.51.0.patch`](../../tests/patches/detox+20.51.0.patch) — [iOS slot simulators](../testing/running-e2e.md#ios-slot-simulators)). Mobile RN ships fmt **12.1.0** upstream; there is **no** `tests/patches/react-native+*.patch` fmt bump.
- **`tests-macos/`** (RN **0.78.3** + `react-native-macos@0.78.6`): applies [`tests-macos/patches/react-native+0.78.3.patch`](../../tests-macos/patches/react-native+0.78.3.patch) (fmt **12.1.0**) and the macos patch.

Those targets must set **`"cache": false`** on the project-level Nx `prepare` override. Reasons:
Expand Down
13 changes: 11 additions & 2 deletions okf-bundle/testing/agent-command-policy.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@ Single source for **which shell commands agents may run** in this repo. E2e `yar
| E2e + coverage | [running e2e](running-e2e.md) — **only** `yarn tests:*` | `jet`, `npx jet`, `yarn jet`, `detox test`, bare `detox`, `cd tests && …`, `cd tests-macos && …`, direct Metro/emulator starts |
| iOS Detox framework cache rebuild | `yarn tests:ios:detox-framework-cache:rebuild` | `cd tests && yarn detox clean-framework-cache`, `cd tests && yarn detox build-framework-cache`, bare `detox …` |
| Host pre-flight (before each `:test-cover`) | [running e2e § pre-flight](running-e2e.md#pre-flight-is-the-host-clear-to-start) — host-clear + services ready + **[checkout ownership](running-e2e.md#services-checkout-ownership-blocking)** + harness tier. `yarn tests:e2e:check` / `yarn tests:e2e:release` ([host-clear probes](running-e2e.md#host-clear-probes)) | Port/HTTP checks alone when Metro/emulators belong to another worktree; `pgrep`/spawn probes of Jet/Detox as completion signals; ad-hoc `pgrep` / hardcoded `:8090` only; improvised kill lists; `--all-slots` while another owner is live |
| E2e slot devices (first use of a slot) | `yarn tests:e2e:setup-android-avds [count]` / `yarn tests:e2e:setup-ios-sims [count]` — **create/reuse-only** of the exact iOS neutral base; count is a strict decimal **1..8** ([slot index](running-e2e.md#e2e-slot-index)). Patched Detox reuses that base or fails when registry-taken; check/release target its exact name — [iOS slot simulators](running-e2e.md#ios-slot-simulators). Teardown of a running device is `yarn tests:e2e:release --devices` (shutdown, never delete) | Direct `xcrun simctl create/delete/erase/rename`, `avdmanager` / `sdkmanager` device commands, `simctl delete all`/`unavailable`, `simctl shutdown all`. Removing a legacy simulator is a **human-operator** step — [migration](running-e2e.md#ios-slot-simulator-migration) |
| Slotted parallel | [running e2e § slot lifecycle](running-e2e.md#slot-lifecycle) — `eval "$(yarn tests:e2e:export-slot-env <platform> N)"` then the **same** `yarn tests:packager:*` / `yarn tests:emulator:start` / `pod:install` / `:build` / `:test-cover` as serial. macOS packager is `yarn tests:macos:packager:*` (`tests-macos/`); mobile is `yarn tests:packager:*` (`tests/`) | ad-hoc port math; second lifecycle helpers as the runbook; rsync between local worktrees; host flock; Metro-after-build; disabling Swift explicit modules; Debug `RCT_NO_LAUNCH_PACKAGER` as a Metro fix; using the mobile packager for macOS Jet (or vice versa) |
| TurboModule codegen (all migrated / CI) | `yarn codegen:verify` (wipe + regen + diff); `yarn codegen:all` for local regen via package scripts | ad-hoc CLI without wipe; inventing alternate codegen yarn scripts — see [TurboModule codegen](#turbomodule-codegen) |

Expand Down Expand Up @@ -86,7 +87,7 @@ Single source for **which shell commands agents may run** in this repo. E2e `yar
**Before any** `yarn tests:ios:build`, `yarn tests:android:build`, or other Detox native build path:

1. **Root `yarn` MUST have run and exited 0** in this checkout. Required on a fresh checkout, after deleting `node_modules`, after pulling patch changes, and whenever patches may be stale. Do **not** start native `:build` until that install finished successfully.
2. Root `yarn` applies **`.yarn/patches`** (jet, detox, mocha-remote) and workspace **`patch-package`** patches. **macOS** still applies **`tests-macos/patches/react-native+0.78.3.patch`** (fmt **12.1.0**). **Mobile RN 0.86.2** ships fmt **12.1.0** upstream (no `tests/patches/react-native+*.patch` fmt bump). `tests` / `tests-macos` `prepare` is `patch-package` and must **not** be Nx-cache-skipped ([MonoTool-AD-12](../monorepo-tooling/architecture-decisions.md#monotool-ad-12--never-nx-cache-prepare-when-the-script-is-patch-package--accepted)).
2. Root `yarn` applies **`.yarn/patches`** (jet, detox, mocha-remote) and workspace **`patch-package`**. **`tests/patches/detox+20.51.0.patch`** is required for slotted iOS exact-base-or-fail allocation ([running e2e § iOS slot simulators](running-e2e.md#ios-slot-simulators)) — it is **not** the Yarn Berry Detox patch. **macOS** still applies **`tests-macos/patches/react-native+0.78.3.patch`** (fmt **12.1.0**). **Mobile RN 0.86.2** ships fmt **12.1.0** upstream (no `tests/patches/react-native+*.patch` fmt bump). `tests` / `tests-macos` `prepare` is `patch-package` and must **not** be Nx-cache-skipped ([MonoTool-AD-12](../monorepo-tooling/architecture-decisions.md#monotool-ad-12--never-nx-cache-prepare-when-the-script-is-patch-package--accepted)). Without the tests Detox patch applied, an occupied neutral slot may cause stock Detox to create a suffixed device outside the slot contract.
3. **Verify** the React Native fmt podspec reports version **≥ 12.1.0** (Xcode 26 / Apple Clang 21-safe floor for this pin). **Yarn exit 0 alone is not sufficient** — always run the check below before native `:build` (a prepare cache-skip historically left fmt at **11.0.2** despite a green install):

```bash
Expand All @@ -99,10 +100,16 @@ rg 'spec\.version|:tag' tests-macos/node_modules/react-native/third-party-podspe

Expect `12.1.0` (or higher) on both `spec.version` and `:tag`.

Before **slotted** iOS `:test-cover`, also confirm `tests/patches/detox+20.51.0.patch` landed (expect a match; missing means an occupied neutral base can trigger stock Detox clone creation / suffix escape instead of exact-base-or-fail — [iOS slot simulators](running-e2e.md#ios-slot-simulators)):

```bash
rg 'RN_E2E_SLOT_BASE_NAME' tests/node_modules/detox/src/devices/allocation/drivers/ios/SimulatorAllocDriver.js
```

4. If fmt is still **11.0.2** (or anything **< 12.1.0**): **STOP**. Re-run root `yarn` / fix patch application for **macOS** (including Nx `tests-macos:prepare` cache policy — [MonoTool-AD-12](../monorepo-tooling/architecture-decisions.md#monotool-ad-12--never-nx-cache-prepare-when-the-script-is-patch-package--accepted)). On mobile 0.86+, fmt should already be ≥12.1.0 without a patch; if not, investigate the resolved `react-native` version. Do **not** invent Podfile `post_install` fmt hacks, `FMT_USE_CONSTEVAL` / `base.h` patches, c++17-for-fmt-only, or web-search workarounds.
5. **Before `yarn tests:ios:build` on a clean checkout:** run root `yarn` (which includes `yarn ruby:install`), then run **`yarn tests:ios:pod:install`** and wait for exit 0. Re-run the Pod install after `tests/ios/Podfile` / `Podfile.lock`, package podspec, or native dependency-resolution changes, and whenever `tests/ios/Pods` is absent. Root `yarn` does **not** create the CocoaPods xcconfig or file-list support files; the shared build invokes Detox/Xcode and does **not** provision them. CI installs the root bundle before yarn, then uses this same Pods-before-build order in `.github/workflows/tests_e2e_ios.yml`.

**Symptoms when violated:** Apple Clang 21 consteval errors compiling unpatched fmt **11.0.2**; missing `tests/ios/Pods` xcconfig or file-list support files; agents inventing Podfile/fmt workarounds instead of re-running root `yarn` / fixing prepare cache policy.
**Symptoms when violated:** Apple Clang 21 consteval errors compiling unpatched fmt **11.0.2**; missing `tests/ios/Pods` xcconfig or file-list support files; an occupied slotted iOS base causing stock Detox to create a suffixed simulator instead of failing (tests Detox patch-package skipped); agents inventing Podfile/fmt workarounds instead of re-running root `yarn` / fixing prepare cache policy.

## When install or prepare fails

Expand Down Expand Up @@ -130,6 +137,7 @@ Expect `12.1.0` (or higher) on both `spec.version` and `:tag`.
| `detox test`, bare `detox`, `cd tests && detox …` | E2e agent rule |
| bare `bundle install` at repo root | Use **`yarn ruby:install`** or root **`yarn`** (`postinstallDev` includes ruby:install) |
| Ad-hoc Metro / emulator start | Use `yarn tests:packager:jet` (iOS/Android) or `yarn tests:macos:packager:jet` (macOS), `yarn tests:emulator:start` ([running e2e](running-e2e.md#rules)) |
| Direct `xcrun simctl delete` / `erase` / `rename` / `create` (and `simctl delete all` / `unavailable`, `simctl shutdown all`) as e2e device management | Destroys devices other worktrees, repos, or operators may be mid-run on. Provision with `yarn tests:e2e:setup-ios-sims` (create/reuse-only), free with `yarn tests:e2e:release --devices` (shutdown only). Legacy-simulator removal is human-operator — [migration](running-e2e.md#ios-slot-simulator-migration) |
| Spawn / PATH probes to “test” Jet or genversion | Log triage only; fix product code and re-run canonical command |

## Known traps
Expand Down Expand Up @@ -240,6 +248,7 @@ Never react-native init / npx @react-native-community/cli init — @react-native
Never: yarn workspace prepare, yarn jet, npx jet, cd packages/* && yarn prepare/build for diagnostics.
Never invent format/install: yarn google-java-format, bare/npx google-java-format, npm install, yarn install in tests/ alone — use root yarn first; Java format = yarn lint:android ONLY.
Never invent Android Gradle: ad-hoc ./gradlew outside yarn tests:android:unit / :build / :post-e2e-coverage / :test:jacoco-report; bare detox/jet/metro.
E2e slot devices: yarn tests:e2e:setup-ios-sims / setup-android-avds (create/reuse-only of the exact iOS neutral base; count decimal 1..8). Patched Detox reuses that base or fails when registry-taken (tests/patches/detox+20.51.0.patch via tests prepare; exact slot-0..7). yarn tests:e2e:release --devices shuts down the exact base. Direct check/release --slot=N uses the same 0..effective max guard as export-slot-env. Never direct xcrun simctl create/delete/erase/rename, simctl delete all|unavailable, simctl shutdown all. Legacy RNFB E2E iOS slot-N removal is human-operator only — okf-bundle/testing/running-e2e.md#ios-slot-simulator-migration.
Prepare/install: yarn or yarn lerna:prepare must exit 0 before ANY other command — never parallelize with e2e/Metro/build.
Before native :build: root yarn exit 0 + verify tests/node_modules/react-native/third-party-podspecs/fmt.podspec (and tests-macos copy when building macOS) ≥ 12.1.0 — okf-bundle/testing/agent-command-policy.md#install-patch-fmt-gate-blocking. Before iOS build on a clean checkout: root yarn, then yarn tests:ios:pod:install exit 0. If fmt < 12.1.0: STOP and re-run yarn; never invent Podfile/FMT_USE_CONSTEVAL/c++17 fmt hacks.
Area harness: okf-bundle/testing/running-e2e.md#local-harness-overrides-harnessoverridesjs — copy harness.overrides.example.js to gitignored harness.overrides.js; set modules + RNFBDebug; delete overrides after run.
Expand Down
Loading
Loading