ci(ios): the iOS simulator smoke lane is failing on main across several rotating signatures

[thymikee]

The fixture-backed iOS simulator E2E smoke job is failing frequently on main, with different assertions on different runs, which makes it unusable as a merge signal: a red smoke on a PR currently says nothing about that PR.

Rate. Of the last 12 ios.yml runs on main (2026-09-10 → 2026-09-11): 6 failures, 2 success, 4 cancelled.

Signatures observed, all on main or reproducing independently of a change:

assertion	scenario / step	seen in
wait for the WebView page to expose its link	smoke:webview-remote-content	main 34592935964
id="automation-longpress" did not become visible after scrolling	scroll + is visible	PR run 34595714111
wait timed out for text: Agent Device Tester, wait_capture_stalled, readableCaptures: 0 on the FIRST capture after a cold open --relaunch	smoke:automation-input	main 34400074702, 34226333904 — already tracked as #2343
xcrun --sdk iphonesimulator --show-sdk-version (spawnSync xcrun ETIMEDOUT) after ** TEST BUILD SUCCEEDED **	runner build step	PR run 34490275779 — matches #2422 (closed, recurring)
Test timed out in 5000ms	"Verify clean-installed Simulator snapshot bridge preparation"	PR run 34519750958

The clearest evidence that this is the lane and not the change under test: two runs of the same commit (6ba5b3da81, job 103251301238 then the re-run 103255078739) failed with two different assertions — automation-longpress first, then the webview-link one that main is also failing. A deterministic regression cannot rotate its symptom.

Why it matters now. Reviewers are being asked to judge merge-readiness against a signal that is red for unrelated reasons, and the honest response to each red is a manual bisect against main — which is exactly the cost this lane exists to avoid. #2343 covers one signature; the others are unfiled.

Suggested direction (not prescriptive): step 14 runs node --test directly with no retry layer, while the replay steps in the same job use --retries 2, so a single cold-start stall fails the whole job. Worth deciding per-scenario which failures are genuinely non-retriable, and separating "the lane found a real defect" from "the simulator was cold or slow". The capture-stall class in #2343 already has a measured mechanism to build on.

Related: #2343, #2422.

[thymikee]

Keep the failures separated by signature. #2493 addresses the WebView landmark probe and the wait that gives up on RUNNER_BUSY; it does not resolve the cold-capture, scrolling or build-timeout cases. Different failures on the same commit justify investigation, but do not establish that every red run is unrelated to a PR. Confirm each current signature against main and keep this issue open for the remaining cases after #2493 lands.

[thymikee]

Root cause for row 1 (wait for the WebView page to expose its link)

Traced it end to end on the newest occurrence, job 103282380298 (PR #2448 head 28c6e86bee). It is a lane defect, not a change-under-test defect: the failure is the same signature as main's 34592935964, the log has zero system-surface mentions, and the mechanism below is platform-wide and predates the branch.

The failing step never polled. The error JSON carries none of the wait evidence fields (reason, timeoutMs, polls, captures, waitedMs, readableCaptures). It is the raw capture error:

command: agent-device wait text Jump to form 20000 ...
"code": "RUNNER_BUSY",
"message": "The iOS runner is still finishing a previous command that exceeded its
            execution watchdog (usually an accessibility capture on a heavy or
            animating screen)."
"hint":    "Wait a few seconds and retry. ..."
details: { command: "snapshot", lifecycleState: "failed",
           recovery: "runner_reported_failure" }

So: a WebView accessibility capture exceeded the runner's execution watchdog, the next capture got RUNNER_BUSY, and wait gave up on its first poll with 20 s of budget unspent.

Why it gives up, in three steps.

1. The runner error is built as retriable. packages/platform-apple/src/runner/runner-session.ts:920 sets retriable: true for RUNNER_BUSY, and the code is in DIAGNOSTIC_ONLY_RUNNER_ERROR_CODES so it stays COMMAND_FAILED with details.runnerErrorCode preserved.

2. Nothing retries on that. retriable is a wire classification only: request-router.ts and request-finalization.ts copy it outward to the client, and no host-side caller consults it. The only code that reads details.runnerErrorCode for a policy decision is packages/platform-apple/src/alert.ts:149, and it reads ALERT_NOT_FOUND. RUNNER_BUSY appears in that file and in alert-contract.ts only as prose precedent for the pattern.

3. The wait poll loop has exactly one error-tolerance channel, and it is Android-only. wait text builds its polling at src/commands/interaction/runtime/wait-text.ts:19 with no classification argument, so createWaitPolling defaults to isUnreadableCaptureContentError, which inspects details.androidSnapshotHelperFailureReason and nothing else. Any other throw propagates out of unreadable.attempt and out of the whole wait.

The wait family therefore has a budget it cannot spend on the one error whose own hint says to wait and retry. The lane is a heavy animating WebView, which is precisely the screen that trips the watchdog, so this row will keep rotating back until the tolerance channel admits a retriable runner stall.

Filed as #2496 for the fix. Recording it here so row 1 has a cause rather than a count.

[thymikee]

Correction to my comment above, and the fix is already in flight: #2493.

Two things in my diagnosis were wrong or incomplete.

The trigger is not "a heavy animating WebView trips the watchdog." It is a landmark mismatch. acceptDeepLinkConfirmationIfPresent had a hard-coded Automation-lab readiness landmark, so on any other route it could never match and always fell through to its alert get probe. An XCTest alert query against a live WKWebView exceeds the runner's 30 s main-thread execution watchdog, and that is what leaves the runner refusing later commands as RUNNER_BUSY. smoke:regular-visible-depth-frontier carried the same mismatch and the same doomed probe. So row 1 is a scenario-authoring defect first, and the wait's surrender is what turned it into an opaque failure.

There is a second classification path I did not find. A runner error recovered from the lifecycle journal after a lost transport response was built with a bare toAppErrorCode, so on that path RUNNER_BUSY reached callers as a wire code with no retriable flag at all.

#2493 fixes all three: the route landmark, the wait ride-out, and one classifyRunnerReportedError behind both paths. It carries live before/after validation on a booted simulator. My wait-poll trace above is accurate as a mechanism but is superseded as a root cause. I also filed #2496 before finding #2493 and have closed it as a duplicate.

Row 1 should be considered owned by #2493 rather than open here.

[thymikee]

Keep row 1 assigned to #2493, but not resolved yet: its latest exact-head iOS smoke still fails at the WebView page wait. The error now has the corrected COMMAND_FAILED / retriable classification, which confirms that part of the fix reached the lane; successful observation of the page is still missing. The other signatures remain separate work.