Skip to content

fix(ios): recover simulator recorder startup failures - #2447

Merged
thymikee merged 1 commit into
callstack:mainfrom
PLASMA-FR:fix/ios-recorder-recovery-a-o9z3yuye
Sep 11, 2026
Merged

thymikee merged 1 commit into
callstack:mainfrom
PLASMA-FR:fix/ios-recorder-recovery-a-o9z3yuye

Conversation

@PLASMA-FR

@PLASMA-FR PLASMA-FR commented Sep 10, 2026

Copy link
Copy Markdown
Contributor

Summary

Closes #2170.

Recover iOS Simulator recording startup without leaving CoreSimulator's host-wide recording slot stuck. Unpublished recorder rollback now sends SIGINT, waits a bounded five seconds for graceful detach, and escalates only if the process remains alive. A native simctl recordVideo exit 16 is returned as non-retriable DEVICE_IN_USE with details.reason: apple_simulator_recording_busy and actionable recovery guidance; other exits retain their existing classification.

Startup now observes exits that settle before PID discovery or on the final process-identity poll. The provider scenario proves a failed start adopts no recorder and a later start/stop succeeds. Five files, +195/−26.

Validation

Tested 06e98c7e6a7bf0517c31339fb2a4814ae055d9cc with Node 24.13.1 and pnpm 11.17.0.

  • The focused regressions failed before the fix; 19 simulator-host tests and the provider contention/recovery scenario pass.
  • AGENT_DEVICE_VITEST_MAX_WORKERS=1 pnpm check:affected --run passes: 301 files / 2,144 tests, 12 command-doc tests, format, lint, types, layering, Fallow, and build.
  • GitHub Coverage, Integration, and every platform smoke job pass on this head. This Linux host cannot run CoreSimulator, and ordinary iOS smoke does not poison/recover the host recording slot; that native contention scenario remains a disclosed review risk.

@thymikee

Copy link
Copy Markdown
Member

No code findings at 06e98c7. The tests cover error classification, graceful rollback and failed-start cleanup through the daemon. Before this is ready, please add a macOS/CoreSimulator run that observes real recording contention and confirms a subsequent recording starts and stops after the SIGINT rollback; the injected exit-16 process and ordinary iOS smoke do not prove the host recording slot is released.

thymikee added a commit that referenced this pull request Sep 10, 2026
…grace

The exit-16 classifier and graceful start rollback are owned by #2447 with the DEVICE_IN_USE contract; drop the overlapping COMMAND_FAILED classifier here to keep one start-path error contract. Keep only the non-overlapping daemon-startup fix: an orphaned simctl recorder is reaped with the same finalize grace the live stop path allows, plus a startup-wiring test. Refs #2170.
@thymikee

Copy link
Copy Markdown
Member

Sharing the CoreSimulator contention + recovery evidence you asked for (gathered on iPhone 16 with an isolated daemon state dir, host restored afterward). Relevant to the DEVICE_IN_USE / apple_simulator_recording_busy contract in this PR.

1. Real recording contention — raw xcrun simctl io <UDID> recordVideo, second recorder while the first holds the host lock:

rec2 exit code=16
Error starting video recorder: Error Domain=NSPOSIXErrorDomain Code=16 "Resource busy"
  UserInfo={NSLocalizedFailureReason=Host recording is already in progress}.
An error was encountered processing the command (domain=NSPOSIXErrorDomain, code=16):
The operation couldn’t be completed. Host recording is already in progress

Confirms exit 16 / POSIX EBUSY and the host-wide ("Host recording is already in progress") nature of the lock — the exact signal this PR classifies.

2. Orphan recovery + a subsequent recording starts and stops (agent-device record start --scope device):

  • record start → recorder live; raw contender → exit=16 (host lock held).
  • kill -9 the daemon mid-recording → daemon=dead recorder=ALIVE(orphaned); raw contender → exit=16 (orphan keeps the host lock stuck — the iOS sim: ungraceful recorder kill leaves CoreSimulator host recording lock dangling (code 16 for all later recordings) #2170 failure state).
  • Restart daemon → the orphaned recorder is reaped with a graceful SIGINT finalize; raw contender then recorded OK (lock freed).
  • Fresh session: record start succeeded (no EBUSY) and record stop produced a valid mp4 → a subsequent recording starts and stops after recovery.

3. Host restored (graceful stop, sim shut down, no leftover recordVideo).

Context: #2457 was narrowed to the non-overlapping daemon-startup reap grace; the start-path classifier and graceful rollback stay here. This evidence covers the contention + subsequent-start/stop ask for both.

@thymikee

Copy link
Copy Markdown
Member

The new CoreSimulator run proves real exit-16 contention and daemon-startup orphan recovery, which covers #2457. It does not exercise this PR’s unpublished-recorder startup rollback. The remaining validation is a failed start that invokes this build’s SIGINT rollback, followed by a successful recording start and stop on the same host. Code review remains clean and checks are green at 06e98c7; that path-specific evidence is still needed.

thymikee added a commit that referenced this pull request Sep 11, 2026
…window (#2457)

* fix(ios): hint the busy simctl recorder and detach recorders gracefully

Classify `simctl recordVideo` exit 16 (POSIX EBUSY) as a typed COMMAND_FAILED with
an actionable hint and the recorder's stderr instead of an unclassified UNKNOWN, so
a caller learns another recording holds the one host-wide CoreSimulator slot or a
prior recorder died without detaching.

Give a recorder that is rolled back or reaped the same graceful SIGINT -> SIGTERM ->
SIGKILL escalation the live stop path already uses, and widen the daemon-startup
reaper's recorder grace to match it, so finalize-and-detach releases the host-wide
recording lock instead of a mid-detach SIGKILL dangling it for every later recording.

Closes #2170

* refactor(ios): inline the busy-recorder classifier and drop the rollback helper

Simplify #2170. The exit-code classification lives beside `startError` that is its only
caller, so drop the separate module and its test and assert through the real start path
instead. Test through `normalizeError` to keep the wire-shape (hint lifted, reason and
exit code retained) guarantee.

Revert the start-rollback SIGKILL sites: the identity arm fires only when the recorder
already exited, and the acquisition rollback discards an explicitly canceled recording, so
fast-kill there is fine. The dangling-host path is the startup reaper, which keeps the
wider finalize grace added earlier.

* refactor(ios): scope this change to the daemon-startup recorder reap grace

The exit-16 classifier and graceful start rollback are owned by #2447 with the DEVICE_IN_USE contract; drop the overlapping COMMAND_FAILED classifier here to keep one start-path error contract. Keep only the non-overlapping daemon-startup fix: an orphaned simctl recorder is reaped with the same finalize grace the live stop path allows, plus a startup-wiring test. Refs #2170.
@thymikee

Copy link
Copy Markdown
Member

@thymikee Built this PR's head (06e98c7) and ran the path-specific validation on iPhone 16 (isolated daemon state dir, host restored after). It exercises the unpublished-recorder SIGINT rollback on a failed start, followed by a successful start+stop on the same daemon (no restart).

A. Failed start with the host already busy → DEVICE_IN_USE + rollback cleanup, then success on the same session:

$ agent-device record start failed.mp4 --scope device      # raw `simctl recordVideo` holder active
Error (DEVICE_IN_USE): CoreSimulator host recording is already in progress
Hint: Stop the active recording ... restart the CoreSimulator stream service before retrying.
failed.mp4 -> removed-by-rollback
recordVideo procs after failed start -> ONLY the holder (no orphan from the failed attempt)

After releasing the holder: record start on the same session/daemon succeeded and record stop produced a valid mp4 (52,178 B).

B. Failed start via client-cancel during acquisition → SIGINT rollback of a LIVE recorder (acquireSimulatorProcess abort -> rollbackAcquiredSimulatorProcess):

recorder during acquisition: pid=87117            # host free, recorder is live
(cancel client mid-startup)
live recorder 87117 rolled back after 0.21s, alive=False
recordVideo procs now -> none                     # graceful SIGINT detach, no orphan
cancel.mp4 -> removed

The same daemon stayed healthy: a following record start/record stop succeeded (52,178 B).

Both runs show the failed-start rollback removing the partial output and leaving no orphaned simctl recordVideo, so a subsequent recording starts and stops on the same host without a daemon restart. Checks are green at 06e98c7; this is the path-specific evidence for the rollback.

@thymikee

Copy link
Copy Markdown
Member

The new CoreSimulator evidence at 06e98c7 closes the remaining validation gap: cancelling acquisition rolled back a live unpublished recorder, removed the partial output, and allowed a subsequent recording start/stop on the same daemon without restart. The real busy-host case also confirms DEVICE_IN_USE classification and cleanup. Code review remains clean, current-head checks pass, and there are no conflicts. Ready for human review and merge.

@thymikee thymikee added the ready-for-human Valid work that needs human implementation, judgment, or maintainer merge label Sep 11, 2026
@thymikee
thymikee merged commit fd80c1e into callstack:main Sep 11, 2026
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-human Valid work that needs human implementation, judgment, or maintainer merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

iOS sim: ungraceful recorder kill leaves CoreSimulator host recording lock dangling (code 16 for all later recordings)

2 participants