Skip to content

Commit 1a10037

Browse files
committed
feat(wait): carry the poll evidence on the replay landmark-mismatch refusal
Review follow-up. A replayed selector wait refused for a recorded landmark mismatch threw without the captures/polls evidence, and when its final poll ended in a runner restart the refusal hid that outcome. The refusal now carries the same failure evidence a timeout does, next to its mismatch details; two regressions cover a mismatch followed by a deadline-cancelled capture and by a runner restart. Docs and changelog name the refusal alongside the polling timeout paths.
1 parent 6adfa7a commit 1a10037

4 files changed

Lines changed: 98 additions & 3 deletions

File tree

CHANGELOG.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,8 @@
66
after a readable capture) carry a per-poll timeline in `error.details` (`captures`, `polls[]`
77
with `startedMs`, `durationMs`, and a typed `outcome`: readable, unreadable, deadline,
88
runner-restart) next to the unchanged `reason`, so a failure says where its budget went without
9-
opening the request log. Long waits keep the first five and last twenty-five polls. `wait
9+
opening the request log. Long waits keep the first five and last twenty-five polls. The replay
10+
landmark-mismatch refusal carries the same poll evidence next to its mismatch details; `wait
1011
--stable` timeouts and a never-readable strict absence keep their existing diagnostics.
1112
- Fixed: the iOS Simulator AX snapshot route bounds how long a capture waits for app discovery
1213
and stops starting a discovery per capture. Discovery (`simctl launchctl list` through xcrun)

src/commands/interaction/runtime/wait-selector.test.ts

Lines changed: 92 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -207,6 +207,98 @@ test('runtime wait fails closed at the deadline when only impostors matched the
207207
assert.equal(observed.label, 'Screen X');
208208
const ancestry = error.details?.observedAncestry as Array<{ role: string; label?: string }>;
209209
assert.equal(ancestry[0]?.label, 'List Screen');
210+
assertReadablePollEvidence(error);
211+
});
212+
213+
/** The refusal carries the same poll evidence a plain timeout would. */
214+
function assertReadablePollEvidence(error: AppError): void {
215+
const details = error.details ?? {};
216+
const polls = details.polls as Array<{ outcome: string }>;
217+
assert.ok(polls.length >= 1);
218+
assert.ok(polls.every((poll) => poll.outcome === 'readable'));
219+
assert.equal(details.captures, polls.length);
220+
assert.equal(details.readableCaptures, polls.length);
221+
assert.equal(typeof details.waitedMs, 'number');
222+
}
223+
224+
/**
225+
* An impostor capture, then a capture that outlives the deadline: the refusal still names the
226+
* landmark mismatch, and its poll timeline shows the deadline cutting the last capture short,
227+
* with the runner-restart evidence that capture carried.
228+
*/
229+
async function landmarkRefusalAfter(
230+
finalCapture: () => Promise<ReturnType<typeof landmarkScreen>>,
231+
): Promise<AppError> {
232+
const recorded = recordedLandmarkFor(landmarkScreen('Detail Screen'));
233+
let call = 0;
234+
const impostor = landmarkScreen('List Screen');
235+
const device = createAgentDevice({
236+
backend: {
237+
platform: 'ios',
238+
captureSnapshot: async () => {
239+
call += 1;
240+
if (call === 1) return { snapshot: impostor };
241+
return { snapshot: await finalCapture() };
242+
},
243+
} satisfies AgentDeviceBackend,
244+
artifacts: createLocalArtifactAdapter(),
245+
sessions: createMemorySessionStore([{ name: 'default', snapshot: impostor }]),
246+
policy: localCommandPolicy(),
247+
clock: createFakeClock(100),
248+
});
249+
const error = await device.selectors
250+
.wait({
251+
session: 'default',
252+
target: {
253+
kind: 'selector',
254+
selector: 'label="Screen X"',
255+
timeoutMs: 400,
256+
recordedLandmark: recorded,
257+
},
258+
})
259+
.then(
260+
() => undefined,
261+
(error: unknown) => error,
262+
);
263+
assert.ok(error instanceof AppError);
264+
assert.equal(error.details?.reason, WAIT_LANDMARK_MISMATCH_REASON);
265+
return error;
266+
}
267+
268+
test('landmark refusal after a deadline-cancelled capture keeps the poll timeline', async () => {
269+
const error = await landmarkRefusalAfter(async () => {
270+
await new Promise((resolve) => setTimeout(resolve, 600));
271+
return landmarkScreen('List Screen');
272+
});
273+
274+
const polls = error.details?.polls as Array<{ outcome: string }>;
275+
assert.deepEqual(
276+
polls.map((poll) => poll.outcome),
277+
['readable', 'deadline'],
278+
);
279+
assert.equal(error.details?.readableCaptures, 1);
280+
assert.equal(error.details?.captures, 2);
281+
});
282+
283+
test('landmark refusal after a runner restart keeps the restart outcome', async () => {
284+
const error = await landmarkRefusalAfter(async () => {
285+
await new Promise((resolve) => setTimeout(resolve, 600));
286+
throw new AppError('COMMAND_FAILED', 'runner restarted', {
287+
runnerRestarted: true,
288+
runnerRestartReason: 'runner_readiness_preflight_failed_before_command_send',
289+
});
290+
});
291+
292+
const polls = error.details?.polls as Array<{ outcome: string }>;
293+
assert.deepEqual(
294+
polls.map((poll) => poll.outcome),
295+
['readable', 'runner-restart'],
296+
);
297+
assert.equal(error.details?.runnerRestarted, true);
298+
assert.equal(
299+
error.details?.runnerRestartReason,
300+
'runner_readiness_preflight_failed_before_command_send',
301+
);
210302
});
211303

212304
test('runtime wait with a recorded landmark keeps the plain timeout when the selector never matched', async () => {

src/commands/interaction/runtime/wait-selector.ts

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -119,10 +119,12 @@ export async function waitForSelector<Runtime extends SelectorWaitRuntime>(
119119
await polling.sleepUntilNextPoll();
120120
}
121121
if (deadline !== 'capture-stalled' && landmarkMismatch) {
122+
// The refusal keeps the poll evidence a plain timeout would carry, including a runner
123+
// restart on the final poll: the mismatch is the verdict, not the whole story of the wait.
122124
throw new AppError(
123125
'COMMAND_FAILED',
124126
`wait matched selector ${selectorExpression} but no candidate carried the recorded landmark identity`,
125-
{ reason: WAIT_LANDMARK_MISMATCH_REASON, ...landmarkMismatch },
127+
{ reason: WAIT_LANDMARK_MISMATCH_REASON, ...polling.failureEvidence(), ...landmarkMismatch },
126128
);
127129
}
128130
throw waitTimeoutError(`wait timed out for selector: ${selectorExpression}`, polling, deadline);

website/docs/docs/commands.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -419,7 +419,7 @@ agent-device alert dismiss
419419
- Because `wait @ref` is text-based after resolution, duplicate labels can match a different element than the original ref target.
420420
- `wait` shares the selector/snapshot resolution flow used by `click`, `fill`, `get`, and `is`.
421421
- Wait failures carry a structured `error.details.reason` in `--json` output: `wait_target_absent` proves a positive wait never found a match; `wait_target_present` means strict `wait absent` reached its deadline with valid captures that still contained matches; `predicate_failed` means strict `wait absent` could not prove absence because no valid capture arrived, with the final observation/diagnostic preserved; `wait_capture_stalled` means no readable capture arrived and is retriable; `wait_deadline_exceeded` means a later capture consumed the remaining budget after an earlier readable capture; `wait_landmark_identity_mismatch` is a replay destination-guard refusal; and `wait_stable_timeout` means the UI did not settle. Use `readableCaptures`, `waitedMs`, `matches`, and `firstMatch` instead of parsing error text. `firstMatch` carries identity/text evidence only; absence failures do not claim visibility or rect evidence.
422-
- Polling wait timeouts (`wait <selector>`, `wait text`, `wait @ref`, and `wait absent` once a readable capture has been seen) also carry `captures` (every poll attempted), `readableCaptures`, and `polls`, one entry per poll with `startedMs` on the wait's own clock, `durationMs`, and `outcome` (`readable`, `unreadable`, `deadline`, or `runner-restart`), so a timeout says where its budget went; long waits keep the first five and last twenty-five polls. `wait --stable` timeouts and a never-readable strict absence keep their own diagnostics. `logPath` links the full request log.
422+
- Polling wait timeouts (`wait <selector>`, `wait text`, `wait @ref`, and `wait absent` once a readable capture has been seen) also carry `captures` (every poll attempted), `readableCaptures`, and `polls`, one entry per poll with `startedMs` on the wait's own clock, `durationMs`, and `outcome` (`readable`, `unreadable`, `deadline`, or `runner-restart`), so a timeout says where its budget went; long waits keep the first five and last twenty-five polls. A replayed selector wait refused for a recorded landmark mismatch (`wait_landmark_identity_mismatch`) carries the same poll evidence next to its mismatch details. `wait --stable` timeouts and a never-readable strict absence keep their own diagnostics. `logPath` links the full request log.
423423
- `alert` inspects or handles system alerts on iOS simulator, macOS desktop, and Android native/runtime permission dialogs.
424424
- `alert` without an action is equivalent to `alert get`.
425425
- `accept` and `dismiss` are sent once on every platform. A lost or unconfirmed response is reported as an error and never replayed; run `alert get` before acting again.

0 commit comments

Comments
 (0)