Skip to content

fix(cloud): cancel a screen read that its caller gave up on - #2516

Merged
thymikee merged 5 commits into
mainfrom
fix/cloud-source-cancellation
Sep 13, 2026
Merged

thymikee merged 5 commits into
mainfrom
fix/cloud-source-cancellation

Conversation

@thymikee

@thymikee thymikee commented Sep 12, 2026

Copy link
Copy Markdown
Member

Summary

A hosted page-source read that outran its request was dropped by the client while the driver kept
walking the UI tree. Nothing at the wire said the answer was no longer wanted, so the read stayed in
flight and the session stayed busy serving a capture nobody was waiting for. On a screen that never
goes still — looping video, live ticker, continuous animation — that is what turns one stuck
snapshot -i into a frozen session.

The read is now bound to the request that asked for it, on Android and iOS, and a source read that
runs out of budget keeps its webdriver_request_timeout reason and names the cause.

agent-device snapshot -i --session df
# reason webdriver_request_timeout; the hint names a never-idle screen and says --timeout cannot widen this read

Closes #2509. Touched files: 10. The lease half of the investigation is stacked separately in #2517.

Validation

  • pnpm check:affected --run passed at 513df9839 (all runnable checks; provider integration and
    coverage are CI-owned jobs on this head).
  • npx vitest run packages/provider-webdriver/src test/integration/provider-scenarios/cloud-webdriver-runtime.test.ts
    — green. Red without the fix: the abandoned capture was still running after its client hung up.
  • Live on AWS Device Farm, Google Pixel 8a (Android 14): connect aws-device-farm, snapshot -i
    (66.4s cold, 25 nodes), screenshot (6.3s, 1080x2400), warm snapshot -i (0.37s). Session closed
    and disconnected.
  • Review round: --timeout is gone from the hint and the docs. It widens the command around the read,
    not the read itself — which is why the report's 65-second attempt changed nothing. A real source-read
    budget is unimplemented and deserves its own issue.
  • Docs placement: the never-idle rule now also sits on the Snapshots page, where shared hosted
    WebDriver behaviour belongs; the provider page keeps the metered-device part.
  • Remaining risk: aborting at the wire cannot cancel the driver's own tree walk. The reporter's
    lease not active is a distinct mechanism and is not fixed here.

@thymikee
thymikee added this pull request to stack #2518 September 12, 2026 17:10
@thymikee thymikee changed the title fix/cloud source cancellation fix(cloud): cancel a screen read that its caller gave up on Sep 12, 2026
@thymikee
thymikee marked this pull request as ready for review September 12, 2026 17:11
@github-actions

github-actions Bot commented Sep 12, 2026

Copy link
Copy Markdown
PR Preview Action v1.8.1
Preview removed because the pull request was closed.
2026-09-13 07:24 UTC

@github-actions

github-actions Bot commented Sep 12, 2026

Copy link
Copy Markdown

Size Report

Metric Base Current Diff
Installed (including dependencies) 4.50 MB 4.50 MB +555 B
Package (unpacked) 4.50 MB 4.50 MB +555 B
Package (download) 1.33 MB 1.33 MB +201 B

Startup median (7 runs, lower is better):

Scenario Base Current Diff
CLI --version 19.0 ms 19.7 ms +0.8 ms
CLI --help 52.1 ms 55.1 ms +3.0 ms

@thymikee

Copy link
Copy Markdown
Member Author

The cancellation wiring is sound at 1adbc57, but the new recovery advice cannot work: webdriver-interactor.ts:611-614 and the AWS docs suggest a larger --timeout for webdriver_request_timeout, while that flag only widens the daemon envelope. The source request still uses the transport policy timeout (30 seconds by default), with no timeout override from snapshot. Remove that advice or connect it to an actual source-read budget. Current checks pass and there are no conflicts; the reported AWS run covers ordinary reads, not cancellation and recovery on the provider.

@thymikee
thymikee force-pushed the fix/cloud-source-cancellation branch from 63ef151 to c740e5c Compare September 12, 2026 18:44
@thymikee

Copy link
Copy Markdown
Member Author

Fixed in c740e5ca6. Confirmed your reading first: WebDriverRequestOverrides.timeoutMs is never
passed for /source, so every read runs on the transport's 30s policy with one retry no matter what
--timeout says — which is also why the report's 65-second attempt changed nothing.

The hint now names whose budget it is and drops the advice, the AWS docs say the same, and two added
assertions pin the hint text so the advice cannot come back. A source-read budget the caller can
actually set is still missing; that needs a surface decision of its own rather than riding in here.

@thymikee
thymikee force-pushed the fix/cloud-source-cancellation branch from c652fd1 to 2d7d741 Compare September 12, 2026 18:59
@thymikee

Copy link
Copy Markdown
Member Author

The timeout advice is corrected on 2d7d741. One documentation issue remains in website/docs/docs/snapshots.md:63: cancelling the client read does not establish that later provider commands stop waiting behind the abandoned capture. The Appium tree walk can continue, as the PR description acknowledges. Please bound that paragraph to stopping client-side waiting and acknowledge that provider work may continue. The aborting fixture proves local cleanup, not provider queue recovery. Current checks pass and there are no conflicts.

A hosted page-source read that outran its request was dropped by the client while
the driver kept walking the UI tree. Nothing at the wire said the answer was no
longer wanted, so the read stayed in flight and held the session it was blocking.
On a screen that never goes still -- a looping video, a live ticker, continuous
animation -- every later command then queued behind a capture nobody was waiting
for, which is what makes one stuck `snapshot -i` look like a frozen session.

Bind the read to the request that asked for it, on both Android and iOS, and say
what a source read that runs out of budget was waiting for. The timeout keeps its
`webdriver_request_timeout` reason and gains a hint naming a screen that never goes
still, so the rented minutes end with a cause rather than a silent hang.

Closes #2509
The hint and the AWS docs both told the caller to retry with a larger `--timeout`.
That flag widens the command envelope around the read; the read's own budget is the
transport's, so the advice could not work and cost rented minutes to discover. The
report on #2509 shows exactly that experiment failing at 65 seconds.

Say whose budget it is, and offer the two things that do work: a `screenshot`, which
never reads the tree, and the `@refs` an earlier snapshot captured.
The scenario comment credited this layer with the lease-gone symptom, which a
ten-minute cloud WebDriver lease rules out for the reported run. And one assertion
message said the driver's own tree walk had been cancelled, when what the test
observes is our request being hung up at the wire.

Behaviour and coverage are unchanged; the test now says what it measures.
…lives

The read that fails is shared hosted WebDriver behaviour, so it belongs on the
Snapshots page rather than only under one provider. The provider page keeps the part
that is about being metered.

Both pages name the two dead ends, since both were tried on the reported run: a
larger `--timeout`, which widens the command around the read, and `settings
animations`, which hosted WebDriver sessions do not implement.
Hanging up the client read proves agent-device stops waiting and stops holding the
session. It says nothing about the provider, whose tree walk can keep running and can
still occupy that session's queue server-side. The paragraph claimed the recovery the
fixture does not prove.
@thymikee
thymikee force-pushed the fix/cloud-source-cancellation branch from 2d7d741 to 513df98 Compare September 13, 2026 06:09
@thymikee

Copy link
Copy Markdown
Member Author

Bound in 513df9839. The paragraph now claims only what the fixture proves: agent-device stops
waiting and stops holding the session, and the driver's own walk can keep running on the provider and
still occupy that session's queue.

Stack was also rebased onto main 973b74cc14 in the same pass, so both heads carry a fresh
pnpm check:affected --run on the new merge-base: green at 513df9839 here and 25c563d02 on #2517.

@thymikee

Copy link
Copy Markdown
Member Author

The cancellation wording is now accurate on 513df98: client waiting stops, while the provider walk may continue and occupy its queue. No actionable findings remain. The runtime changes are preserved by the rebase and the reported live AWS evidence still applies; current CI is running, so merge readiness awaits those checks.

@thymikee thymikee added the ready-for-human Valid work that needs human implementation, judgment, or maintainer merge label Sep 13, 2026
@thymikee
thymikee merged commit b92b6ca into main Sep 13, 2026
20 checks passed
@thymikee
thymikee deleted the fix/cloud-source-cancellation branch September 13, 2026 07:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-human Valid work that needs human implementation, judgment, or maintainer merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

snapshot -i hangs on AWS Device Farm Android when the screen plays looping video, then the whole session freezes

1 participant