Skip to content

Touch input has no visible effect on a headless iOS 27 simulator (Xcode 27) #153

Description

@appletimemac

Summary

On a headlessly booted iOS 27 iPad simulator under Xcode 27, every tap and drag I tried produced no visible response, including on SpringBoard. The touch debug messages print and serve-sim tap exits 0. Rotation works, and a hardware lock button press works, so the legacy HID channel itself is delivering.

I could not work out why touch specifically has no effect, and a comment in the installed source says this path is verified on iOS 27, so I am probably missing something about my setup. Details below in case it is useful.

Environment

  • macOS 27.0 (26A428), Apple Silicon
  • Xcode 27.0 (27A266a), the only Xcode installed. No Simulator.app on the machine; Xcode 27 ships DeviceHub.app.
  • iOS 27.0 simulator runtime, iPad Pro 13-inch (M5), booted with xcrun simctl boot (no UI host attached)
  • serve-sim 0.1.45, 0.1.46 and 0.1.47-beta.122.1 — same behaviour on all three
  • Node 26.7.0

Reproduction

Nothing app-specific; SpringBoard is enough.

UDID=<an iPad sim udid>
xcrun simctl boot "$UDID"
serve-sim --detach --no-preview "$UDID"

serve-sim rotate landscape_left -d "$UDID"      # works
serve-sim button lock -d "$UDID"                # works, screen locks
serve-sim button lock -d "$UDID"                # unlock again

xcrun simctl launch "$UDID" com.apple.springboard
xcrun simctl io "$UDID" screenshot before.png   # note an app icon's position

serve-sim tap <x> <y> -d "$UDID"                # normalized to that icon; exits 0
xcrun simctl io "$UDID" screenshot after.png    # identical; the app did not open

before.png and after.png came out pixel-identical for me (mean channel delta 0.0).

What I observed

Driving the native module directly, bypassing the CLI and the server:

SERVE_SIM_DEBUG_HID=1 node -e "
const m=require('/opt/homebrew/lib/node_modules/serve-sim/dist/native/serve-sim-native.node');
const h=new m.SimHID(process.env.UDID);
(async()=>{ await new Promise(r=>setTimeout(r,1500));
  await h.touch('begin',0.715,0.343,1376,1032,0);
  await new Promise(r=>setTimeout(r,70));
  await h.touch('end',0.715,0.343,1376,1032,0);
  await new Promise(r=>setTimeout(r,800)); process.exit(0); })();"
[hid] IndigoHIDMessageForButton loaded
[hid] IndigoHIDMessageForHIDArbitrary loaded
[hid] IndigoHIDMessageForKeyboardArbitrary loaded
[hid] IndigoHIDMessageForDigitalCrownEvent loaded
[hid] SimDeviceLegacyHIDClient created
[hid] IndigoHIDMessageForMouseNSEvent loaded (with edge gesture support)
[hid] Sending begin at (0.715,0.343)
[hid] Sending end at (0.715,0.343)

The initialization and send messages all appear, and I saw no error in the captured output.

Things I checked, in case they save you time:

  • IndigoHIDMessageForMouseNSEvent is present in Xcode 27's SimulatorKit (/Applications/Xcode.app/Contents/SharedFrameworks/SimulatorKit.framework/SimulatorKit, confirmed with nm -gU), and the dlsym succeeds.
  • begin/end, and begin/move/end with a 180 ms dwell, behave the same.
  • The legacy HID channel is delivering. button lock goes through sendHIDButtonIndigoHIDMessageForButtonSimDeviceLegacyHIDClient and reliably locks and unlocks the screen. (I first tried button home as the control and it was misleading: that case calls launchSpringBoard, which shells out to xcrun simctl launch com.apple.springboard, so it proves nothing about HID.)
  • Same on 0.1.45, 0.1.46 and 0.1.47-beta.122.1.
  • Running with the preview server rather than --no-preview made no difference.
  • Launching DeviceHub.app and retrying made no difference, though I could not confirm the simulator window was actually visible and frontmost at the time.

What I could not determine

I have not established why touch has no effect here. Two things in the installed source cut against the obvious guess:

  • HIDInjector.swift describes scroll as a touch drag on the digitizer (target 0x32), "the same path taps/swipes use, which is verified to scroll on iOS 27". So target 0x32 touch is expected to work on this runtime.
  • DeviceHubKeyboardBridge covers keyboard only, so touch is presumably not meant to need Device Hub.

Given both, I suspect something specific to my configuration — most likely that the simulator is booted with no UI host at all, since this machine has no Simulator.app and Device Hub is not normally open. Is a UI host expected for touch delivery? Anything else I should check?

Minor, separate: tap exiting 0 does not mean the touch was delivered

After the WebSocket opens, tap sends begin, schedules end after 40 ms, then schedules closing and resolving after a further 50 ms. It installs no reply handler and does not confirm injection or guest handling. Argument validation and connection failures do still fail the command.

That is reasonable for a fire-and-forget CLI, but it is what made this take a while to narrow down: a silently dropped event and a delivered one look identical from the shell. A warning when the injector is unavailable, or an opt-in --verify, would make this class of problem visible.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions