Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[wptrunner] Lazily post testdriver result while polling the next message #49513

Merged

Conversation

jonathan-j-lee
Copy link
Contributor

@jonathan-j-lee jonathan-j-lee commented Dec 4, 2024

Currently, a typical testdriver call takes three WebDriver commands:

  1. Poll the next browser → executor message via the resume script
  2. Execute the requested testdriver command (e.g., "element click" for test_driver.click())
  3. Post the result of (2) from executor → browser, which resolves the call's promise

The testharness executor repeats the above steps in an event loop until a message from (1) indicates the test is complete (or a helper thread times out the loop).

This PR optimizes the speed of testdriver-heavy tests by combining (3) and (1) for pairs of consecutive calls by running them in the same "execute async script" invocation. This PR should be entirely transparent to tests because the browser should eventually see all send_message() calls in the same order.

When tested downstream, no tests regress, and the following local test sees a modest speedup from 16 to 15.5 min:

run_headless_shell_wpt -vv -j 1 \
  external/wpt/editing/other/join-different-white-space-style-paragraphs.html

See Also: https://crbug.com/381927516

@jonathan-j-lee jonathan-j-lee marked this pull request as ready for review December 4, 2024 00:25
@wpt-pr-bot wpt-pr-bot added infra wptrunner The automated test runner, commonly called through ./wpt run labels Dec 4, 2024
... for symmetry with `send_message()`, which the executor uses to
settle async testdriver calls browser-side. This is a no-op refactor
that gives vendors more flexibility how to exchange messages with the
test page.
Currently, a typical testdriver call takes three WebDriver commands:
1. Poll the next browser -> executor message via the resume script
2. Execute the requested testdriver command (e.g., "element click"
   for `test_driver.click()`)
3. Post the result of (2) from executor -> browser, which resolves the
   call's promise

The testharness executor repeats the above steps in an event loop until
a message from (1) indicates the test is complete (or a helper thread
times out the loop).

This PR optimizes the speed of testdriver-heavy tests by combining (3)
and (1) for pairs of consecutive calls by running them in the same
"execute async script" invocation [a]. This PR should be entirely
transparent to tests because the browser should eventually see all
`send_message()` calls in the same order, with no changes in testdriver
semantics.

When tested downstream, no tests regress [b], and the following local
test sees a modest speedup from 16 to 15.5 min:

```sh
run_headless_shell_wpt external/wpt/editing/other/join-different-white-space-style-paragraphs.html -vv -j 1
```

[a]: https://w3c.github.io/webdriver/#execute-async-script
[b]: https://chromium-review.googlesource.com/c/chromium/src/+/6068186/1?tab=checks

See Also: https://crbug.com/381927516
@jonathan-j-lee jonathan-j-lee force-pushed the wptrunner/lazy-testdriver branch from 06ef5e1 to 6fea24d Compare December 4, 2024 22:02
@jonathan-j-lee jonathan-j-lee enabled auto-merge (rebase) December 4, 2024 22:03
auto-merge was automatically disabled December 4, 2024 22:13

Pull request was closed

@jonathan-j-lee jonathan-j-lee reopened this Dec 4, 2024
@jonathan-j-lee jonathan-j-lee reopened this Dec 5, 2024
@jonathan-j-lee
Copy link
Contributor Author

@KyleJu @DanielRyanSmith The CI failure looks like #49545 (occurring on other PRs), and this PR in chromium/src CQ didn't introduce any noticeable functional or performance regressions. Please force merge? Thanks!

@KyleJu KyleJu merged commit 3684d7f into web-platform-tests:master Dec 5, 2024
35 of 37 checks passed
@jonathan-j-lee jonathan-j-lee deleted the wptrunner/lazy-testdriver branch December 5, 2024 18:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
infra wptrunner The automated test runner, commonly called through ./wpt run
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants