fix: reject distinct ambiguous mutation targets

Author: thymikee

CHANGELOG.md

@@ -5,6 +5,7 @@
 - `agent-device help workflow` is now a compact ~8KB card instead of a ~41KB dump; the same depth still exists, split into `help scripting` (save-script, secret-safe fills, batch JSON, replay divergence/repair, recording) and `help gestures` (multi-touch shapes and platform quirks), plus a few paragraphs folded into the topics that already owned the subject (`help debugging`, `help physical-device`, `help validate`). Every `help <topic>` first line is now `agent-device <version> — <topic>` so an agent can read the installed version from its mandatory first help read instead of a separate `agent-device --version` call.
 - Cloud iOS (BrowserStack, AWS Device Farm): `snapshot` and `diff` no longer fail with `SESSION_NOT_FOUND` on a live provider session (#1658). The app-session guard they ran belongs to the local XCUITest runner, which must attach to a target app; a cloud capture reads the provider's own driver session and needs no app identity, so the guard now applies to local Apple targets only. Relatedly, a cloud iOS `open com.example.app` now records that bundle id on the session — the provider path skips local app resolution (no simctl/devicectl reaches a hosted device), and used to drop an explicitly spelled bundle id along with it, leaving the session with no app identity at all. Opening a second bundle id replaces the first, matching the local path, where an explicitly spelled target always wins over the session's current app; deep links, display names, and bare `open` still keep the app already tracked.
 - Cloud `fill` (BrowserStack, AWS Device Farm) now witnesses that the field it tapped actually holds text-entry focus before sending its keys, instead of dispatching tap and keys in back-to-back requests (#1658). A WebView input — an OAuth/SSO page in a Safari view controller, for example — takes first responder asynchronously, so the keys used to land with nothing focused while `fill` still answered "Filled N chars"; tapping and filling as two separate commands worked only because the round trip between them gave the field time to focus. The witness is the focused element's own geometry: `fill` polls the active element and proceeds only once it contains the point it tapped, which is the one signal that identifies *which* field took focus. Keyboard visibility cannot — it reads the same before and after a second fill into an already-open form, so it could not tell a focused password field from the email field the previous fill left focused. The response discloses `textEntryReadiness`: `focused-element`, or `keyboard-shown` when the driver has no active-element route but the keyboard rose from hidden after the tap. Both describe a fill that witnessed focus before typing; there is deliberately no value for typing without evidence, because nothing renders this field and such a value would reach a caller as an ordinary success. Breaking: when focus cannot be witnessed, cloud `fill` now FAILS with `COMMAND_FAILED` / `text_entry_focus_not_observed` and sends no keys, instead of typing into whatever holds first responder and answering "Filled N chars" — a fill with no witness must not read as a filled field. That covers a tap that focused nothing, a keyboard already up on a driver that cannot name the focused field, and a driver that reports neither (`text_entry_focus_unobservable`, which points at `press` + `type` as the deliberate way to enter text unwitnessed). Only a positively classified unimplemented route counts as unsupported, so a dead session, an auth rejection, or a grid outage surfaces instead of degrading into a blind text entry.
+- Changed mutating selector ambiguity semantics (press/click/fill/longpress): duplicate accessibility wrappers collapse only when every match forms one ancestor-descendant chain resolving to the same actionable node. Matches in distinct subtrees now fail fast with `AMBIGUOUS_MATCH` and a bounded, immediately reusable candidate-ref frame; visible/depth/area geometry no longer silently picks a mutation target. The direct iOS XCTest path now counts raw exact matches before hittability and delegates ambiguity to the same runtime rule. AppControlBench provenance: element-14 ran on 0.20.5; this change is intended for 0.20.7+, and comparative benchmark reports should note that it can replace a wrong-success recovery loop with one candidate-pick turn while occasionally adding that turn for genuinely distinct duplicates.
 - `scroll` and `back` now accept `--settle` (with `--settle-quiet` and `--timeout`), collapsing scroll-then-observe and back-then-observe into one call (#1638). The response carries the same settled payload the touch commands return — verdict, changed-lines diff with fresh refs on added lines, the unchanged-interactive tail, and `refsGeneration` when the settled tree was stored — and is best-effort: it never fails the action. One difference is deliberate: `scroll`/`back` resolve no element, so the diff baseline is the session's stored pre-action tree ("the last tree you observed") rather than a freshly resolved pre-action capture. Both commands now also preserve the daemon on timeout, like the other settle-capable commands.
 - Security: repository `./agent-device.json` now accepts only project-safe automation defaults. It rejects daemon endpoint/auth/transport/server settings, tenant/run/lease selectors, provider/cloud and Metro connection fields, headers, executable reporter modules, local write destinations, and other operator-controlled values before local module loading or any daemon health/RPC request. Put remote endpoint and token together in protected CI environment variables, user config, an explicit `--config` file, or the existing `connect`/`--remote-config` workflow. Daemon auth tokens no longer travel in serialized command flags.
 - `viewport` is now rejected during capability admission on Apple targets instead of reaching the device and failing inside dispatch. No Apple backend can resize a screen — simulator and device geometry is fixed by the selected device type — so `viewport` on iOS/iPadOS/tvOS/macOS now fails with `UNSUPPORTED_OPERATION`, `viewport is not supported on this device`, and a hint pointing at `--platform web` and at picking a different simulator. `capabilities` no longer advertises `viewport` on Apple targets. Web viewport resizing (`agent-device viewport 1280 900 --platform web`) is unchanged, and Android was already denied.

apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerSelectorMatchPolicy.swift

@@ -0,0 +1,60 @@
+struct SelectorCandidateFacts {
+  let isHittable: Bool
+  let hasTappableFrame: Bool
+  let containsExpectedPoint: Bool
+
+  init(
+    isHittable: Bool,
+    hasTappableFrame: Bool,
+    containsExpectedPoint: Bool = true
+  ) {
+    self.isHittable = isHittable
+    self.hasTappableFrame = hasTappableFrame
+    self.containsExpectedPoint = containsExpectedPoint
+  }
+}
+
+enum DirectSelectorCandidateDecision: Equatable {
+  case noMatch
+  case selected(index: Int, usedNonHittableFallback: Bool)
+  case ambiguous
+}
+
+/// Normal direct selector mutations count every raw exact match before
+/// hittability can choose a winner. Maestro's explicitly requested coordinate
+/// fallback keeps its point-filtered compatibility behavior.
+func classifyDirectSelectorCandidates(
+  _ candidates: [SelectorCandidateFacts],
+  allowNonHittableFallback: Bool,
+  filtersByExpectedPoint: Bool = false
+) -> DirectSelectorCandidateDecision {
+  let eligible = candidates.indices.filter { index in
+    !filtersByExpectedPoint || candidates[index].containsExpectedPoint
+  }
+
+  if !allowNonHittableFallback {
+    guard eligible.count <= 1 else { return .ambiguous }
+    guard let index = eligible.first, candidates[index].isHittable else { return .noMatch }
+    return .selected(index: index, usedNonHittableFallback: false)
+  }
+
+  var hittableIndex: Int?
+  var fallbackIndex: Int?
+  for index in eligible {
+    let candidate = candidates[index]
+    if candidate.isHittable {
+      guard hittableIndex == nil else { return .ambiguous }
+      hittableIndex = index
+    } else if candidate.hasTappableFrame {
+      guard fallbackIndex == nil else { return .ambiguous }
+      fallbackIndex = index
+    }
+  }
+  if let hittableIndex {
+    return .selected(index: hittableIndex, usedNonHittableFallback: false)
+  }
+  if let fallbackIndex {
+    return .selected(index: fallbackIndex, usedNonHittableFallback: true)
+  }
+  return .noMatch
+}

apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+Interaction.swift

@@ -149,35 +149,31 @@ extension RunnerTests {
       return SelectorElementMatch(element: nil, isAmbiguous: false, usedNonHittableFallback: false)
     }
 
-    var matchedElement: XCUIElement?
-    var nonHittableElement: XCUIElement?
     let matches = app.descendants(matching: .any).matching(predicate).allElementsBoundByIndex
-    for element in matches where element.exists {
-      if let expectedPoint, !element.frame.contains(expectedPoint) {
-        continue
-      }
-      if !element.isHittable {
-        if allowNonHittableFallback && hasTappableFrame(app: app, element: element) {
-          guard nonHittableElement == nil else {
-            return SelectorElementMatch(element: nil, isAmbiguous: true, usedNonHittableFallback: false)
-          }
-          nonHittableElement = element
-        }
-        continue
-      }
-      guard matchedElement == nil else {
-        return SelectorElementMatch(element: nil, isAmbiguous: true, usedNonHittableFallback: false)
-      }
-      matchedElement = element
+      .filter(\.exists)
+    let facts = matches.map { element in
+      SelectorCandidateFacts(
+        isHittable: element.isHittable,
+        hasTappableFrame: hasTappableFrame(app: app, element: element),
+        containsExpectedPoint: expectedPoint.map(element.frame.contains) ?? true
+      )
     }
-    if let matchedElement {
-      return SelectorElementMatch(element: matchedElement, isAmbiguous: false, usedNonHittableFallback: false)
+    switch classifyDirectSelectorCandidates(
+      facts,
+      allowNonHittableFallback: allowNonHittableFallback,
+      filtersByExpectedPoint: expectedPoint != nil
+    ) {
+    case .noMatch:
+      return SelectorElementMatch(element: nil, isAmbiguous: false, usedNonHittableFallback: false)
+    case .ambiguous:
+      return SelectorElementMatch(element: nil, isAmbiguous: true, usedNonHittableFallback: false)
+    case let .selected(index, usedNonHittableFallback):
+      return SelectorElementMatch(
+        element: matches[index],
+        isAmbiguous: false,
+        usedNonHittableFallback: usedNonHittableFallback
+      )
     }
-    return SelectorElementMatch(
-      element: nonHittableElement,
-      isAmbiguous: false,
-      usedNonHittableFallback: nonHittableElement != nil
-    )
   }
 
   // Maestro-compat gate for the non-hittable coordinate fallback: an element

apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+SelectorMatchPolicyTests.swift

@@ -0,0 +1,41 @@
+import XCTest
+
+extension RunnerTests {
+#if AGENT_DEVICE_RUNNER_UNIT_TESTS
+  func testDirectSelectorRejectsTwoRawMatchesBeforeHittabilityPreference() {
+    let decision = classifyDirectSelectorCandidates(
+      [
+        SelectorCandidateFacts(isHittable: true, hasTappableFrame: true),
+        SelectorCandidateFacts(isHittable: false, hasTappableFrame: true),
+      ],
+      allowNonHittableFallback: false
+    )
+
+    XCTAssertEqual(decision, .ambiguous)
+  }
+
+  func testDirectSelectorAcceptsOneRawHittableMatch() {
+    XCTAssertEqual(
+      classifyDirectSelectorCandidates(
+        [SelectorCandidateFacts(isHittable: true, hasTappableFrame: true)],
+        allowNonHittableFallback: false
+      ),
+      .selected(index: 0, usedNonHittableFallback: false)
+    )
+  }
+
+  func testMaestroSelectorKeepsExpectedPointAndNonHittableFallbackSemantics() {
+    XCTAssertEqual(
+      classifyDirectSelectorCandidates(
+        [
+          SelectorCandidateFacts(isHittable: false, hasTappableFrame: true, containsExpectedPoint: false),
+          SelectorCandidateFacts(isHittable: false, hasTappableFrame: true, containsExpectedPoint: true),
+        ],
+        allowNonHittableFallback: true,
+        filtersByExpectedPoint: true
+      ),
+      .selected(index: 1, usedNonHittableFallback: true)
+    )
+  }
+#endif
+}

docs/adr/0011-interaction-guarantee-contract.md

@@ -2,7 +2,7 @@
 
 ## Status
 
-Accepted (implemented through Layer 3, 2026-07-04: #1080, #1082–#1086, #1091, #1092)
+Accepted (implemented through Layer 3, 2026-07-04: #1080, #1082–#1086, #1091, #1092; ambiguity contract amended 2026-08-07)
 
 ## Context
 
@@ -64,7 +64,7 @@ every cell to be classified:
 
 ```ts
 export const INTERACTION_GUARANTEES = [
-  'disambiguation',        // visible > deepest > smallest; ties fail
+  'disambiguation',        // collapse one equivalent wrapper chain; distinct subtrees fail with candidates
   'occlusion',             // covered targets are refused
   'offscreen',             // tap point (rect center) must lie in the root viewport
   'nonHittable',           // promotion + targetHittable/hint annotation
@@ -239,6 +239,23 @@ the daemon there destroyed every healthy app session the daemon owned.
   press guarantees"), which matters for small-model agents that only read the
   contract, never the code.
 
+### 2026-08-07 amendment: mutating ambiguity fails fast
+
+Element-14 realized the matrix's previously owned success-path gap: four exact
+`Team Standup` matches in distinct accessibility subtrees reached ordinary
+runtime selector resolution, whose visible/depth/area ranking silently chose a
+different semantic target. Mutating selectors no longer use geometry to choose
+among distinct subtrees.
+
+The replacement contract is structural. Multiple matches collapse only when
+all matches form one ancestor–descendant chain and every member resolves to the
+same actionable node. Otherwise the mutation fails with `AMBIGUOUS_MATCH`, a
+bounded list of snapshot candidate lines, and a partial ref frame generation so
+the caller can retry one listed candidate immediately. The direct XCTest path
+counts all raw exact matches before hittability can select a winner and delegates
+multiple matches to the runtime classifier; Maestro's explicit expected-point /
+non-hittable compatibility path remains intentionally separate.
+
 ### Synthesized iOS gesture policy
 
 Synthesized iOS gestures (`scroll`, synthesized coordinate `tap`, synthesized