From cc4bec9677adab5b60ffa7666af059a2b02929e3 Mon Sep 17 00:00:00 2001 From: Maksim Sadym Date: Tue, 12 Mar 2024 12:21:24 +0100 Subject: [PATCH 01/16] Reapply "RFC 182: Add WebDriver BiDi support to testdriver.js (#182)" (#184) This reverts commit 9c018edbc09bcf88101b1caae00e0e623eefdd5c. --- rfcs/testdriver_bidi.md | 165 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 165 insertions(+) create mode 100644 rfcs/testdriver_bidi.md diff --git a/rfcs/testdriver_bidi.md b/rfcs/testdriver_bidi.md new file mode 100644 index 00000000..0b1a2d18 --- /dev/null +++ b/rfcs/testdriver_bidi.md @@ -0,0 +1,165 @@ +# RFC 182: Add WebDriver BiDi support to testdriver.js +## Summary +Add “testdriver.js” support for [WebDriver BiDi](https://w3c.github.io/webdriver-bidi/) events and actions. To support WebDriver BiDi, the following parts are required: +* **Wptrunner support for asynchronous actions.** WebDriver BiDi commands are asynchronous. To implement them, the test runner must support asynchronous actions. It should support concurrent actions as well. This means all async actions can be concurrent. +* **Wptrunner - testdriver transport support for events.** WebDriver BiDi events can be sent from the browser to the client. In order to support WebDriver BiDi events, wptrunner should be able to send events to testdriver. +* **Extend testdriver API.** Testdriver should be extended to support WebDriver BiDi commands and events. +* **Update RFC policy.** In [RFC policy](https://github.com/web-platform-tests/rfcs/blob/master/README.md) do not require RFC for changes extending “testdriver.js” with a method that closely matches a [WebDriver BiDi](https://w3c.github.io/webdriver-bidi) protocol. +## Background +The “testdriver.js” framework (or testdriver for simplicity) is based on the one-directional WebDriver Classic protocol and lacks the ability to receive events from the browser via wptrunner, limiting our ability to test event-based functionalities such as console events in WPT (e.g. many [console log tests](https://github.com/web-platform-tests/wpt/tree/master/console) are [manual](https://github.com/web-platform-tests/wpt/blob/master/console/console-timing-logging-manual.html)!). To overcome this limitation, this RFC proposes using the [WebDriver BiDi](https://w3c.github.io/webdriver-bidi) protocol, an extension for WebDriver Classic offering bidirectional communication between the test and the browser. +This document presents changes in testdriver and wptrunner so that it incorporates both WebDriver Classic and WebDriver BiDi functionalities, demonstrating the addition of tests for console events as an [example](https://github.com/web-platform-tests/wpt/pull/44649). +## Details +### Glossary +* **testdriver**: “testdriver.js” framework used by tests and providing a way to communicate with the browser via wptrunner. +* **wptrunner**: backend python scripts implementing test harness, providing communication between testdriver and browser. +* **test harness**: communication channel between testdriver and wptrunner. +### Example +Here is an [example](https://github.com/sadym-chromium/wpt/blob/sadym/testdriver-bidi/console/console-log-logged.html) of a test using the proposed `test_driver.bidi` API for testing console log events. +```javascript +promise_test(async (test) => { + const some_message = "SOME MESSAGE"; + // Subscribe to `log.entryAdded` BiDi events. This will not add a listener to the page. + await test_driver.bidi.log.entry_added.subscribe(); + test.add_cleanup(async function() { + // Unsubscribe from `log.entryAdded` BiDi events. + await test_driver.bidi.log.entry_added.unsubscribe(); + }); + // Add a listener for the log.entryAdded event. This will not subscribe to the event, so the subscription is + // required before. After the event is received, the subscription will not be removed, so the cleanup is + // required. + const log_entry_promise = test_driver.bidi.log.entry_added.once(); + // Emit a console.log message. + console.log(some_message) + // Wait for the log.entryAdded event to be received. + const event = await log_entry_promise; + // Assert the log.entryAdded event has the expected message. + assert_equals(event.args.length, 1); + const event_message = event.args[0]; + assert_equals(event_message.value, some_message); +}, "Assert log event is logged"); +``` +### Required changes +The proposal changes the following parts: +#### Testdriver +The testdriver requires support of event processing, and exposing it as an API. +##### Testdriver API +Introduce a new namespace `bidi` to the `test_driver`, and to `test_driver_internal`. The namespaces will be divided into separate WebDriver BiDi modules like in the [example](https://github.com/web-platform-tests/wpt/pull/44649/files#diff-1fe2b624679a3150e5c86f84682c5901b715dad750096a524e8cb23939e5590fR54): `test_driver.bidi.{module}.{event/action}`. +###### Alternative +[Expose raw WebDriver BiDi protocol to `test_driver`](#expose-raw-webdriver-bidi-protocol-to-test_driver). +###### Actions +The BiDi actions are functionally equivalent to the classic actions. They are placed in the `test_driver.bidi.{module}.{action}`. In order to prevent action name clashes with the classic ones, action names (those used to communicate to the wptrunner) should have a form of `bidi.{module}.{action}`. +###### Alternative +[Flatten bidi and classic actions in `test_driver`](#flatten-bidi-and-classic-actions-in-test_driver). +###### Events +The exposed events API in `test_driver` provides the following methods for each event: +* `test_driver.bidi.{module}.{event}.subscribe(): Promise`. The async method subscribes to the given WebDriver BiDi event, but does not add event listeners yet. +* `test_driver.bidi.{module}.{event}.unsubscribe(): Promise`. The async method unsubscribes from the given WebDriver BiDi event. +* `test_driver.bidi.{module}.{event}.on(handler: (event)=>void): ()=>void`. The method makes the handler be called each time the event emitted. The callback argument is the event params. It returns a handle for removing this listener. +* (optional) `test_driver.bidi.{module}.{event}.once(): Promise`. Creates a listener and returns a promise, which will be resolved after the required event is emitted for the first time. Removes the listener after the first event. This method is not required, but allows easier writing of some tests (e.g. it is used in the example above). It can be easily implemented via the `on` method. Note: this wrapper does not unsubscribe from the event. +###### Alternatives +* [Make generic event emitting in testdriver](#make-generic-event-emitting-in-testdriver). +* [Expose generic `session.subscribe` method in `test_driver` instead of `test_driver.bidi.{module}.{event}.subscribe()`](#expose-generic--sessionsubscribe-method-in-test_driver-instead-of-test_driverbidimoduleeventsubscribe) +* [Do not expose `test_driver.bidi.{module}.{event}.subscribe()`, do it silently in the `on` method](#do-not-expose-test_driverbidimoduleeventsubscribe-do-it-silently-in-the-onmethod) +* [Use `EventTarget`-like events API](#do-not-expose-test_driverbidimoduleeventsubscribe-do-it-silently-in-the-onmethod). +##### Event processing +The proposal extends the event processing implemented in [`testdriver-extra.js`](https://github.com/web-platform-tests/wpt/blob/master/tools/wptrunner/wptrunner/testdriver-extra.js) with a new message type [“testdriver-event”](https://github.com/web-platform-tests/wpt/pull/44649/files#diff-46aeeb40b0a0c13031b151392ca70a17614295533d3e890c0cb4360cb3b91542R43). This message is sent by the wptrunner to notify the testdriver about new events. It uses the `message` field to store JSON-serialized `params` and `method`. The `testdriver-extra.js` script overrides `window.test_driver_internal.bidi.{modules}.{event}` object methods `subscribe`, `unsubscribe`, and `on`. +#### Wptrunner +Currently, the wptrunner processes communication with the testdriver in a blocking way, which makes it impossible to process events from the browser. To overcome this limitation, support of coroutines should be added. +##### Introduce `async_actions` +Analogous to `actions`, introduce an async_actions which will support asynchronous implementations (e.g. [`BidiSessionSubscribeAction`](https://github.com/web-platform-tests/wpt/pull/44649/files#diff-9d6d3e870d033b8bc4d75b388fcc11dcf332e0293c286b2d57734c74e22020f7R4)). These async actions will be processed by `AsyncCallbackHandler`. +##### Alternatives +* [Extend `CallbackHandler` to support async actions](#extend-callbackhandler-to-support-async-actions). +* [Extend testdriver - wptrunner transport with `async_action`](#extend-testdriver---wptrunner-transport-with-async_action). +##### Introduce `AsyncCallbackHandler` +Add a `AsyncCallbackHandler`, which is an extension of `CallbackHandler` with support for async actions. It requires an event loop, in which it will schedule the tasks. Async action processing is done in the same way as CallbackHandler is done, but in the dedicated task. This would not allow raising unexpected exceptions in wptrunner. Testdriver will still be notified about those unexpected exceptions. [Example](https://github.com/web-platform-tests/wpt/pull/44649/files#diff-3de439d55203876d452599a1a14968409df69e5869b16918ceec378a6b3345a4R813) implementation. +##### Extend `WebDriverProtocol` +As long as WebDriver Classic and WebDriver BiDi share the session, the `WebDriverProtocol` should be able to set a `enable_bidi` flag when creating the session. Extend the class with the `enable_bidi=false` flag, which will be overridden by `WebDriverBidiProtocol`. +##### Introduce `WebDriverBidiProtocol` +The specific protocol implementation would be required to opt-in for this protocol. Now it’s `EdgeChromiumDriverProtocol` and `ChromeDriverProtocol`. +###### Dependency on `WebDriverProtocol` +New [`WebDriverBidiProtocol`](https://github.com/web-platform-tests/wpt/pull/44649/files#diff-e5a8911dd97e0352b1b26d8ce6ef0a92b25378f7c9e79371a1eb1b1834bc9a8dR568) should be introduced. It is an extension of WebDriverProtocol, but with additional BiDi-specific protocol parts, e.g. [`BidiEventsProtocolPart`](https://github.com/web-platform-tests/wpt/pull/44649/files#diff-47192f72de48b834a50992ada793229686299b5165612c2d97af6811764514c7R325) and [`BidiScriptProtocolPart`](https://github.com/web-platform-tests/wpt/pull/44649/files#diff-47192f72de48b834a50992ada793229686299b5165612c2d97af6811764514c7R358). In order to enable the bidi session, the inherited `enable_bidi` flag should be overridden to `true`. +###### Dedicated loop +To be able to process events from the browser and from the tests in the same time, the WebDriverBidiProtocol protocol should have a dedicated [event loop](https://github.com/web-platform-tests/wpt/pull/44649/files#diff-e5a8911dd97e0352b1b26d8ce6ef0a92b25378f7c9e79371a1eb1b1834bc9a8dR577), which will be used by wptrunner to run async actions and process events in WebDriver BiDi transport. +###### Alternative +[Add `AsyncProtocolPart` with `get_loop` method](#add-asyncprotocolpart-with-get_loop-method). +##### Extend `TestExecutor` +In this example we consider changes in [`WebDriverTestharnessExecutor.do_testharness`](https://github.com/web-platform-tests/wpt/pull/44649/files#diff-e5a8911dd97e0352b1b26d8ce6ef0a92b25378f7c9e79371a1eb1b1834bc9a8dL591). Other `TestExecutor` (`MarionetteTestharnessExecutor`, `SeleniumTestharnessExecutor`, `ServoWebDriverTestharnessExecutor`, +`WKTRTestharnessExecutor` etc) implementations should be done in the same way in order to support WebDriver BiDi. +###### Messages from testdriver to wptrunner +The communication channel between testdriver and wptrunner is set in the `do_testharness` method. Currently the messages from testdriver to wptrunner it is done via long poll by calling BaseProtocolPart.execute_async_script which is synchronous (async in the name indicates that the script is wrapped in a special way before sent to the browser), and blocks processing the event loop in wptrunner, which in turn blocks the event processing while waiting for the next message form testdriver to wptrunner. To overcome this limitation, this RFC changes the wptrunner - testdriver communication channel to asynchronous non-blocking [`BidiScriptProtocolPart.async_call_function`](https://github.com/web-platform-tests/wpt/pull/44649/files#diff-e5a8911dd97e0352b1b26d8ce6ef0a92b25378f7c9e79371a1eb1b1834bc9a8dR761) if available. `WebDriverBidiProtocol`’s event loop should be used for message processing. This behavioral change is done only for protocols supporting BiDi. +###### Processing asynchronous actions +If `WebDriverBidiProtocol` is available, wptrunner uses [`AsyncCallbackHandler`](https://github.com/web-platform-tests/wpt/pull/44649/files#diff-3de439d55203876d452599a1a14968409df69e5869b16918ceec378a6b3345a4R791) with the `WebDriverBidiProtocol`’s event loop instead of the sync `CallbackHandler`. +###### Forwarding events from wptrunner to testdriver +If `BidiEventsProtocolPart` is available, wptrunner sets a callback via [`add_event_listener`](https://github.com/web-platform-tests/wpt/pull/44649/files#diff-e5a8911dd97e0352b1b26d8ce6ef0a92b25378f7c9e79371a1eb1b1834bc9a8dR696) and forwards all the events to the testdriver. +###### Alternative +[Add `async_execute_script` to `BaseProtocolPart`](#add-async_execute_script-to-baseprotocolpart). +##### Introduce `BidiEventsProtocolPart` and `BidiScriptProtocolPart` +###### `BidiEventsProtocolPart` +To provide events from wptrunner to testdriver, the [`BidiEventsProtocolPart`](https://github.com/web-platform-tests/wpt/pull/44649/files#diff-47192f72de48b834a50992ada793229686299b5165612c2d97af6811764514c7R325) implements [`add_event_listener`](https://github.com/web-platform-tests/wpt/pull/44649/files#diff-47192f72de48b834a50992ada793229686299b5165612c2d97af6811764514c7R345). `TestExecutor` will subscribe and forward events to testdriver. +###### `BidiScriptProtocolPart` +[`BidiScriptProtocolPart`](https://github.com/web-platform-tests/wpt/pull/44649/files#diff-47192f72de48b834a50992ada793229686299b5165612c2d97af6811764514c7R358) is required for implementing asynchronous poll messages from testdriver. It implements async [`async_call_function`](https://github.com/web-platform-tests/wpt/pull/44649/files#diff-e5a8911dd97e0352b1b26d8ce6ef0a92b25378f7c9e79371a1eb1b1834bc9a8dR142) action. +###### Alternative +[Implement a generic `BidiProtocolPart`](#implement-a-generic-bidiprotocolpart). +## Risks +### Future changes in WebDriver BiDi protocol +The WebDriver BiDi protocol may still undergo specification changes that could impact WPT tests. +#### Mitigation +* **Add commands on-demand.** To reduce the effects of future changes to the protocol, actions can be added as and when required instead of preemptively including all possible actions in the testdriver API. +* **Keep the testdriver API minimally required.** By keeping the testdriver API as minimal as possible, the impact of future protocol changes can be further reduced, as fewer API changes will be required. +### WebDriver BiDi implementation differences +Different implementations of WebDriver BiDi may have variations that could potentially complicate its use within the context of the WPT tests. +#### Mitigation +Ensure the testdriver API is compliant with the specification by avoiding reliance on implementation-specific data and behavior. +### WebDriver BiDi events might not agree with some observable side effects +In certain tests, the observable side effects may not correspond entirely with the WebDriver BiDi events. One such instance arises when a `log.entryAdded` BiDi event occurs, which does not guarantee the display of the console message in the browser's developer tools. This can cause false-positive test passes if the BiDi event does not have observable behavior. +#### Mitigation +This risk has a low priority impact and should not impede the project's progress. +### Different WebDriver BiDi implementation status +The implementation of WebDriver BiDi may vary across different endpoints, which can lead to tests utilizing BiDi functionality failing for endpoints lacking the required test components. +#### Mitigation +Expanding testdriver with WebDriver BiDi shouldn't cause previously passing tests to fail. This is because any regressions would only occur due to new tests that utilize WebDriver BiDi encountering incompatible endpoints - a scenario explicitly excluded from regression consideration. +## Alternatives considered +The list of alternatives of different granularity levels: +### Expose raw WebDriver BiDi protocol to `test_driver` +In this alternative approach, the `test_driver` is directly exposed to the raw WebDriver BiDi protocol. However, this approach was declined because it would not support a gradual implementation of WebDriver BiDi. Additionally, it would rely on the specific endpoint implementation status of the WebDriver BiDi, which could lead to flaky tests in case of protocol changes. While this alternative provides ultimate flexibility, its API would become challenging to use and prone to incorrect usage. +### Flatten bidi and classic actions in `test_driver` +In this alternative, in testdriver actions, the new WebDriver BiDi methods are added directly to the root testdriver object instead of introducing a dedicated `bidi` namespace. This approach has both advantages and disadvantages: +#### Advantages +No need in introducing a new namespace, and continue using the established pattern of adding new methods. +#### Disadvantages +* Potential for name clashes: If a new WebDriver BiDi action or event has the same name as an existing WebDriver Classic action or event, it could cause conflicts. +* Harder to determine if a specific testdriver method requires WebDriver BiDi support: Developers may need to consult the documentation or code to determine if a particular method requires WebDriver BiDi support. +* Can be done later after WebDriver BiDi is widely adopted, the `test_drive`’s root-level methods can be silently shifted to use BiDi under the hood. +### Make generic event emitting in testdriver +In this alternative, `test_driver` exposes an `EventTarget`, or methods `addEventListener(type, listener)`, and `removeEventListener(type, listener)`. Test will be able to subscribe to any WebDriver BiDi event. +#### Advantages +This approach is more flexible, as there will be no need in adding support for each WebDriver BiDi event. +#### Disadvantages +The generic events subscription would allow using any text as an event name while subscribing, which would require being familiar with WebDriver BiDi protocol. Also it would make it possible for tests to rely on not implemented or browser-specific events. +### Expose generic `session.subscribe` method in `test_driver` instead of `test_driver.bidi.{module}.{event}.subscribe()` +In this alternative, the `test_driver.session.subscribe` method is exposed in the testdriver API. This allows for more flexibility, as all the BiDi events are supported by default. This suggestion was rejected because the API is challenging to use. Test writers must be familiar with WebDriver BiDi, and there is a risk of [future changes in WebDriver BiDi protocol](#future-changes-in-webdriver-bidi-protocol), which is a concern. +### Do not expose `test_driver.bidi.{module}.{event}.subscribe()`, do it silently in the `on`method +In this alternative, the `test_driver.bidi.{module}.{event}.subscribe()` method is not exposed in the testdriver API. Instead, each `on` handler subscribes to the required event. The API becomes simpler, as test writers would not require care about the concept of WebDriver BiDi subscriptions at all. Nonetheless, this option was rejected because: +* Event enabling API in BiDi is done pre-session. Meaning if `event.on()` is called twice, and after that the first listener is removed, should the subscription be removed as well? +* The `once` handler cannot be implemented since subscription is an asynchronous process. +### Use `EventTarget`-like events API +In this alternative, an [`EventTarget`](https://developer.mozilla.org/en-US/docs/Web/API/EventTarget)-like API is exposed for BiDi events: +`addEventListener(type, listener, options)`. +Options: +* `once: bool` Optional. A boolean value indicating that the listener should be invoked at most once after being added. If true, the listener would be automatically removed when invoked. If not specified, defaults to false. +* `subscribe: bool` indicates if the subscription should be set, if not yet. Defaults to `true`. +* `removeEventListener(type, listener, options)`. + Options: + * `unsubscribe: bool` indicates if the subscription should be stopped. Defaults to `true`. + +This API is more common. However, more workarounds are required for it to be used in tests. Also, the subscription handling is harder and less explicit. The alternative was declined in order to prioritize test driver ergonomics. +### Extend `CallbackHandler` to support async actions +In this alternative, the async actions are supported by `CallbackHandler`. This would require making a `CallbackHandler` of the event loop in order to schedule actions processing. The alternative is rejected, as it would require all other test executors to support event loops. No gradual implementation is possible, which increases risks while introducing this change. After the RFC is implemented, `CallbackHandler` can be deprecated in favor of `AsyncCallbackHandler`. +### Extend testdriver - wptrunner transport with async_action +In this alternative, the testdriver - wptrunner transport is extended with a special `async_action` type. Rejected, as testdriver should not be aware of the specific actions implementation. From the testdriver perspective all the actions are asynchronous. +### Add `AsyncProtocolPart` with `get_loop` method +Instead of exposing the event loop in the WebDriverBidiProtocol, add a dedicated `AsyncProtocolPart` protocol part, which exposes the loop via `get_loop` method. The alternative is rejected, as the event loop is not a part of the protocol, but rather an implementation detail. +### Add `async_execute_script` to `BaseProtocolPart` +In this alternative, the `BaseProtocolPart` should be extended with the async `async_execute_script` method. The alternative is rejected, as it requires significant and risky changes in the WebDriver Classic client. Brings the same risks as alternative [Extend `CallbackHandler` to support async actions](#extend-callbackhandler-to-support-async-actions). +### Implement a generic `BidiProtocolPart` +In this alternative, instead of adding protocol parts one-by-one, a generic `BidiProtocolPart` is introduced. Declined, as it would require all the protocol implementations to either support the full WebDrive BiDi protocol or not. From deb9e6c47dceb7b94ccf7901b51976cce4a5a72b Mon Sep 17 00:00:00 2001 From: Maksim Sadym Date: Tue, 12 Mar 2024 12:25:17 +0100 Subject: [PATCH 02/16] Update RFC number --- README.md | 2 +- rfcs/testdriver_bidi.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index e0737e10..81178036 100644 --- a/README.md +++ b/README.md @@ -21,7 +21,7 @@ or users of web-platform-tests. Examples of where it is likely to be useful: Cases where the RFC process need *not* be used: - Introducing a new top-level directory for a new specification. - - Extending testdriver.js with a method that closely matches a [WebDriver](https://w3c.github.io/webdriver/) endpoint. To notify maintainers of testdriver.js vendor integration, label the pull request `testdriver.js`. (This exemption was introduced by [RFC 127](https://github.com/web-platform-tests/rfcs/blob/master/rfcs/rfc_exemption_testdriver_method.md).) + - Extending testdriver.js with a method that closely matches a [WebDriver Classic](https://w3c.github.io/webdriver/) or [WebDriver BiDi](https://w3c.github.io/webdriver-bidi) endpoints. To notify maintainers of testdriver.js vendor integration, label the pull request `testdriver.js`. (This exemption was introduced by [RFC 127](https://github.com/web-platform-tests/rfcs/blob/master/rfcs/rfc_exemption_testdriver_method.md) and [RFC 185](https://github.com/web-platform-tests/rfcs/blob/master/rfcs/testdriver_bidi.md).) - Minor changes in behavior in where all call sites are known and accounted for. - Behavior-preserving refactoring with a low risk of regressions. diff --git a/rfcs/testdriver_bidi.md b/rfcs/testdriver_bidi.md index 0b1a2d18..07c5d1b6 100644 --- a/rfcs/testdriver_bidi.md +++ b/rfcs/testdriver_bidi.md @@ -1,4 +1,4 @@ -# RFC 182: Add WebDriver BiDi support to testdriver.js +# RFC 185: Add WebDriver BiDi support to testdriver.js ## Summary Add “testdriver.js” support for [WebDriver BiDi](https://w3c.github.io/webdriver-bidi/) events and actions. To support WebDriver BiDi, the following parts are required: * **Wptrunner support for asynchronous actions.** WebDriver BiDi commands are asynchronous. To implement them, the test runner must support asynchronous actions. It should support concurrent actions as well. This means all async actions can be concurrent. From cd4c50bf73984b68ee431c2ed253ac5ee3f72860 Mon Sep 17 00:00:00 2001 From: Maksim Sadym Date: Wed, 13 Mar 2024 11:40:23 +0100 Subject: [PATCH 03/16] Wrap headers with empty lines --- rfcs/testdriver_bidi.md | 112 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 112 insertions(+) diff --git a/rfcs/testdriver_bidi.md b/rfcs/testdriver_bidi.md index 07c5d1b6..d97c433f 100644 --- a/rfcs/testdriver_bidi.md +++ b/rfcs/testdriver_bidi.md @@ -1,19 +1,29 @@ # RFC 185: Add WebDriver BiDi support to testdriver.js + ## Summary + Add “testdriver.js” support for [WebDriver BiDi](https://w3c.github.io/webdriver-bidi/) events and actions. To support WebDriver BiDi, the following parts are required: * **Wptrunner support for asynchronous actions.** WebDriver BiDi commands are asynchronous. To implement them, the test runner must support asynchronous actions. It should support concurrent actions as well. This means all async actions can be concurrent. * **Wptrunner - testdriver transport support for events.** WebDriver BiDi events can be sent from the browser to the client. In order to support WebDriver BiDi events, wptrunner should be able to send events to testdriver. * **Extend testdriver API.** Testdriver should be extended to support WebDriver BiDi commands and events. * **Update RFC policy.** In [RFC policy](https://github.com/web-platform-tests/rfcs/blob/master/README.md) do not require RFC for changes extending “testdriver.js” with a method that closely matches a [WebDriver BiDi](https://w3c.github.io/webdriver-bidi) protocol. + ## Background + The “testdriver.js” framework (or testdriver for simplicity) is based on the one-directional WebDriver Classic protocol and lacks the ability to receive events from the browser via wptrunner, limiting our ability to test event-based functionalities such as console events in WPT (e.g. many [console log tests](https://github.com/web-platform-tests/wpt/tree/master/console) are [manual](https://github.com/web-platform-tests/wpt/blob/master/console/console-timing-logging-manual.html)!). To overcome this limitation, this RFC proposes using the [WebDriver BiDi](https://w3c.github.io/webdriver-bidi) protocol, an extension for WebDriver Classic offering bidirectional communication between the test and the browser. + This document presents changes in testdriver and wptrunner so that it incorporates both WebDriver Classic and WebDriver BiDi functionalities, demonstrating the addition of tests for console events as an [example](https://github.com/web-platform-tests/wpt/pull/44649). + ## Details + ### Glossary + * **testdriver**: “testdriver.js” framework used by tests and providing a way to communicate with the browser via wptrunner. * **wptrunner**: backend python scripts implementing test harness, providing communication between testdriver and browser. * **test harness**: communication channel between testdriver and wptrunner. + ### Example + Here is an [example](https://github.com/sadym-chromium/wpt/blob/sadym/testdriver-bidi/console/console-log-logged.html) of a test using the proposed `test_driver.bidi` API for testing console log events. ```javascript promise_test(async (test) => { @@ -38,111 +48,203 @@ promise_test(async (test) => { assert_equals(event_message.value, some_message); }, "Assert log event is logged"); ``` + ### Required changes + The proposal changes the following parts: + #### Testdriver + The testdriver requires support of event processing, and exposing it as an API. + ##### Testdriver API + Introduce a new namespace `bidi` to the `test_driver`, and to `test_driver_internal`. The namespaces will be divided into separate WebDriver BiDi modules like in the [example](https://github.com/web-platform-tests/wpt/pull/44649/files#diff-1fe2b624679a3150e5c86f84682c5901b715dad750096a524e8cb23939e5590fR54): `test_driver.bidi.{module}.{event/action}`. + ###### Alternative + [Expose raw WebDriver BiDi protocol to `test_driver`](#expose-raw-webdriver-bidi-protocol-to-test_driver). + ###### Actions + The BiDi actions are functionally equivalent to the classic actions. They are placed in the `test_driver.bidi.{module}.{action}`. In order to prevent action name clashes with the classic ones, action names (those used to communicate to the wptrunner) should have a form of `bidi.{module}.{action}`. + ###### Alternative + [Flatten bidi and classic actions in `test_driver`](#flatten-bidi-and-classic-actions-in-test_driver). + ###### Events + The exposed events API in `test_driver` provides the following methods for each event: * `test_driver.bidi.{module}.{event}.subscribe(): Promise`. The async method subscribes to the given WebDriver BiDi event, but does not add event listeners yet. * `test_driver.bidi.{module}.{event}.unsubscribe(): Promise`. The async method unsubscribes from the given WebDriver BiDi event. * `test_driver.bidi.{module}.{event}.on(handler: (event)=>void): ()=>void`. The method makes the handler be called each time the event emitted. The callback argument is the event params. It returns a handle for removing this listener. * (optional) `test_driver.bidi.{module}.{event}.once(): Promise`. Creates a listener and returns a promise, which will be resolved after the required event is emitted for the first time. Removes the listener after the first event. This method is not required, but allows easier writing of some tests (e.g. it is used in the example above). It can be easily implemented via the `on` method. Note: this wrapper does not unsubscribe from the event. + ###### Alternatives + * [Make generic event emitting in testdriver](#make-generic-event-emitting-in-testdriver). * [Expose generic `session.subscribe` method in `test_driver` instead of `test_driver.bidi.{module}.{event}.subscribe()`](#expose-generic--sessionsubscribe-method-in-test_driver-instead-of-test_driverbidimoduleeventsubscribe) * [Do not expose `test_driver.bidi.{module}.{event}.subscribe()`, do it silently in the `on` method](#do-not-expose-test_driverbidimoduleeventsubscribe-do-it-silently-in-the-onmethod) * [Use `EventTarget`-like events API](#do-not-expose-test_driverbidimoduleeventsubscribe-do-it-silently-in-the-onmethod). + ##### Event processing + The proposal extends the event processing implemented in [`testdriver-extra.js`](https://github.com/web-platform-tests/wpt/blob/master/tools/wptrunner/wptrunner/testdriver-extra.js) with a new message type [“testdriver-event”](https://github.com/web-platform-tests/wpt/pull/44649/files#diff-46aeeb40b0a0c13031b151392ca70a17614295533d3e890c0cb4360cb3b91542R43). This message is sent by the wptrunner to notify the testdriver about new events. It uses the `message` field to store JSON-serialized `params` and `method`. The `testdriver-extra.js` script overrides `window.test_driver_internal.bidi.{modules}.{event}` object methods `subscribe`, `unsubscribe`, and `on`. + #### Wptrunner + Currently, the wptrunner processes communication with the testdriver in a blocking way, which makes it impossible to process events from the browser. To overcome this limitation, support of coroutines should be added. + ##### Introduce `async_actions` + Analogous to `actions`, introduce an async_actions which will support asynchronous implementations (e.g. [`BidiSessionSubscribeAction`](https://github.com/web-platform-tests/wpt/pull/44649/files#diff-9d6d3e870d033b8bc4d75b388fcc11dcf332e0293c286b2d57734c74e22020f7R4)). These async actions will be processed by `AsyncCallbackHandler`. + ##### Alternatives + * [Extend `CallbackHandler` to support async actions](#extend-callbackhandler-to-support-async-actions). * [Extend testdriver - wptrunner transport with `async_action`](#extend-testdriver---wptrunner-transport-with-async_action). + ##### Introduce `AsyncCallbackHandler` + Add a `AsyncCallbackHandler`, which is an extension of `CallbackHandler` with support for async actions. It requires an event loop, in which it will schedule the tasks. Async action processing is done in the same way as CallbackHandler is done, but in the dedicated task. This would not allow raising unexpected exceptions in wptrunner. Testdriver will still be notified about those unexpected exceptions. [Example](https://github.com/web-platform-tests/wpt/pull/44649/files#diff-3de439d55203876d452599a1a14968409df69e5869b16918ceec378a6b3345a4R813) implementation. + ##### Extend `WebDriverProtocol` + As long as WebDriver Classic and WebDriver BiDi share the session, the `WebDriverProtocol` should be able to set a `enable_bidi` flag when creating the session. Extend the class with the `enable_bidi=false` flag, which will be overridden by `WebDriverBidiProtocol`. + ##### Introduce `WebDriverBidiProtocol` + The specific protocol implementation would be required to opt-in for this protocol. Now it’s `EdgeChromiumDriverProtocol` and `ChromeDriverProtocol`. + ###### Dependency on `WebDriverProtocol` + New [`WebDriverBidiProtocol`](https://github.com/web-platform-tests/wpt/pull/44649/files#diff-e5a8911dd97e0352b1b26d8ce6ef0a92b25378f7c9e79371a1eb1b1834bc9a8dR568) should be introduced. It is an extension of WebDriverProtocol, but with additional BiDi-specific protocol parts, e.g. [`BidiEventsProtocolPart`](https://github.com/web-platform-tests/wpt/pull/44649/files#diff-47192f72de48b834a50992ada793229686299b5165612c2d97af6811764514c7R325) and [`BidiScriptProtocolPart`](https://github.com/web-platform-tests/wpt/pull/44649/files#diff-47192f72de48b834a50992ada793229686299b5165612c2d97af6811764514c7R358). In order to enable the bidi session, the inherited `enable_bidi` flag should be overridden to `true`. + ###### Dedicated loop + To be able to process events from the browser and from the tests in the same time, the WebDriverBidiProtocol protocol should have a dedicated [event loop](https://github.com/web-platform-tests/wpt/pull/44649/files#diff-e5a8911dd97e0352b1b26d8ce6ef0a92b25378f7c9e79371a1eb1b1834bc9a8dR577), which will be used by wptrunner to run async actions and process events in WebDriver BiDi transport. + ###### Alternative + [Add `AsyncProtocolPart` with `get_loop` method](#add-asyncprotocolpart-with-get_loop-method). + ##### Extend `TestExecutor` + In this example we consider changes in [`WebDriverTestharnessExecutor.do_testharness`](https://github.com/web-platform-tests/wpt/pull/44649/files#diff-e5a8911dd97e0352b1b26d8ce6ef0a92b25378f7c9e79371a1eb1b1834bc9a8dL591). Other `TestExecutor` (`MarionetteTestharnessExecutor`, `SeleniumTestharnessExecutor`, `ServoWebDriverTestharnessExecutor`, `WKTRTestharnessExecutor` etc) implementations should be done in the same way in order to support WebDriver BiDi. + ###### Messages from testdriver to wptrunner + The communication channel between testdriver and wptrunner is set in the `do_testharness` method. Currently the messages from testdriver to wptrunner it is done via long poll by calling BaseProtocolPart.execute_async_script which is synchronous (async in the name indicates that the script is wrapped in a special way before sent to the browser), and blocks processing the event loop in wptrunner, which in turn blocks the event processing while waiting for the next message form testdriver to wptrunner. To overcome this limitation, this RFC changes the wptrunner - testdriver communication channel to asynchronous non-blocking [`BidiScriptProtocolPart.async_call_function`](https://github.com/web-platform-tests/wpt/pull/44649/files#diff-e5a8911dd97e0352b1b26d8ce6ef0a92b25378f7c9e79371a1eb1b1834bc9a8dR761) if available. `WebDriverBidiProtocol`’s event loop should be used for message processing. This behavioral change is done only for protocols supporting BiDi. + ###### Processing asynchronous actions + If `WebDriverBidiProtocol` is available, wptrunner uses [`AsyncCallbackHandler`](https://github.com/web-platform-tests/wpt/pull/44649/files#diff-3de439d55203876d452599a1a14968409df69e5869b16918ceec378a6b3345a4R791) with the `WebDriverBidiProtocol`’s event loop instead of the sync `CallbackHandler`. + ###### Forwarding events from wptrunner to testdriver + If `BidiEventsProtocolPart` is available, wptrunner sets a callback via [`add_event_listener`](https://github.com/web-platform-tests/wpt/pull/44649/files#diff-e5a8911dd97e0352b1b26d8ce6ef0a92b25378f7c9e79371a1eb1b1834bc9a8dR696) and forwards all the events to the testdriver. + ###### Alternative + [Add `async_execute_script` to `BaseProtocolPart`](#add-async_execute_script-to-baseprotocolpart). + ##### Introduce `BidiEventsProtocolPart` and `BidiScriptProtocolPart` + ###### `BidiEventsProtocolPart` + To provide events from wptrunner to testdriver, the [`BidiEventsProtocolPart`](https://github.com/web-platform-tests/wpt/pull/44649/files#diff-47192f72de48b834a50992ada793229686299b5165612c2d97af6811764514c7R325) implements [`add_event_listener`](https://github.com/web-platform-tests/wpt/pull/44649/files#diff-47192f72de48b834a50992ada793229686299b5165612c2d97af6811764514c7R345). `TestExecutor` will subscribe and forward events to testdriver. + ###### `BidiScriptProtocolPart` + [`BidiScriptProtocolPart`](https://github.com/web-platform-tests/wpt/pull/44649/files#diff-47192f72de48b834a50992ada793229686299b5165612c2d97af6811764514c7R358) is required for implementing asynchronous poll messages from testdriver. It implements async [`async_call_function`](https://github.com/web-platform-tests/wpt/pull/44649/files#diff-e5a8911dd97e0352b1b26d8ce6ef0a92b25378f7c9e79371a1eb1b1834bc9a8dR142) action. + ###### Alternative + [Implement a generic `BidiProtocolPart`](#implement-a-generic-bidiprotocolpart). + ## Risks + ### Future changes in WebDriver BiDi protocol + The WebDriver BiDi protocol may still undergo specification changes that could impact WPT tests. + #### Mitigation + * **Add commands on-demand.** To reduce the effects of future changes to the protocol, actions can be added as and when required instead of preemptively including all possible actions in the testdriver API. * **Keep the testdriver API minimally required.** By keeping the testdriver API as minimal as possible, the impact of future protocol changes can be further reduced, as fewer API changes will be required. + ### WebDriver BiDi implementation differences + Different implementations of WebDriver BiDi may have variations that could potentially complicate its use within the context of the WPT tests. + #### Mitigation + Ensure the testdriver API is compliant with the specification by avoiding reliance on implementation-specific data and behavior. + ### WebDriver BiDi events might not agree with some observable side effects + In certain tests, the observable side effects may not correspond entirely with the WebDriver BiDi events. One such instance arises when a `log.entryAdded` BiDi event occurs, which does not guarantee the display of the console message in the browser's developer tools. This can cause false-positive test passes if the BiDi event does not have observable behavior. + #### Mitigation + This risk has a low priority impact and should not impede the project's progress. + ### Different WebDriver BiDi implementation status + The implementation of WebDriver BiDi may vary across different endpoints, which can lead to tests utilizing BiDi functionality failing for endpoints lacking the required test components. + #### Mitigation + Expanding testdriver with WebDriver BiDi shouldn't cause previously passing tests to fail. This is because any regressions would only occur due to new tests that utilize WebDriver BiDi encountering incompatible endpoints - a scenario explicitly excluded from regression consideration. + ## Alternatives considered + The list of alternatives of different granularity levels: + ### Expose raw WebDriver BiDi protocol to `test_driver` + In this alternative approach, the `test_driver` is directly exposed to the raw WebDriver BiDi protocol. However, this approach was declined because it would not support a gradual implementation of WebDriver BiDi. Additionally, it would rely on the specific endpoint implementation status of the WebDriver BiDi, which could lead to flaky tests in case of protocol changes. While this alternative provides ultimate flexibility, its API would become challenging to use and prone to incorrect usage. + ### Flatten bidi and classic actions in `test_driver` + In this alternative, in testdriver actions, the new WebDriver BiDi methods are added directly to the root testdriver object instead of introducing a dedicated `bidi` namespace. This approach has both advantages and disadvantages: + #### Advantages + No need in introducing a new namespace, and continue using the established pattern of adding new methods. + #### Disadvantages + * Potential for name clashes: If a new WebDriver BiDi action or event has the same name as an existing WebDriver Classic action or event, it could cause conflicts. * Harder to determine if a specific testdriver method requires WebDriver BiDi support: Developers may need to consult the documentation or code to determine if a particular method requires WebDriver BiDi support. * Can be done later after WebDriver BiDi is widely adopted, the `test_drive`’s root-level methods can be silently shifted to use BiDi under the hood. + ### Make generic event emitting in testdriver + In this alternative, `test_driver` exposes an `EventTarget`, or methods `addEventListener(type, listener)`, and `removeEventListener(type, listener)`. Test will be able to subscribe to any WebDriver BiDi event. + #### Advantages + This approach is more flexible, as there will be no need in adding support for each WebDriver BiDi event. + #### Disadvantages + The generic events subscription would allow using any text as an event name while subscribing, which would require being familiar with WebDriver BiDi protocol. Also it would make it possible for tests to rely on not implemented or browser-specific events. + ### Expose generic `session.subscribe` method in `test_driver` instead of `test_driver.bidi.{module}.{event}.subscribe()` + In this alternative, the `test_driver.session.subscribe` method is exposed in the testdriver API. This allows for more flexibility, as all the BiDi events are supported by default. This suggestion was rejected because the API is challenging to use. Test writers must be familiar with WebDriver BiDi, and there is a risk of [future changes in WebDriver BiDi protocol](#future-changes-in-webdriver-bidi-protocol), which is a concern. + ### Do not expose `test_driver.bidi.{module}.{event}.subscribe()`, do it silently in the `on`method + In this alternative, the `test_driver.bidi.{module}.{event}.subscribe()` method is not exposed in the testdriver API. Instead, each `on` handler subscribes to the required event. The API becomes simpler, as test writers would not require care about the concept of WebDriver BiDi subscriptions at all. Nonetheless, this option was rejected because: * Event enabling API in BiDi is done pre-session. Meaning if `event.on()` is called twice, and after that the first listener is removed, should the subscription be removed as well? * The `once` handler cannot be implemented since subscription is an asynchronous process. + ### Use `EventTarget`-like events API + In this alternative, an [`EventTarget`](https://developer.mozilla.org/en-US/docs/Web/API/EventTarget)-like API is exposed for BiDi events: `addEventListener(type, listener, options)`. Options: @@ -153,13 +255,23 @@ Options: * `unsubscribe: bool` indicates if the subscription should be stopped. Defaults to `true`. This API is more common. However, more workarounds are required for it to be used in tests. Also, the subscription handling is harder and less explicit. The alternative was declined in order to prioritize test driver ergonomics. + ### Extend `CallbackHandler` to support async actions + In this alternative, the async actions are supported by `CallbackHandler`. This would require making a `CallbackHandler` of the event loop in order to schedule actions processing. The alternative is rejected, as it would require all other test executors to support event loops. No gradual implementation is possible, which increases risks while introducing this change. After the RFC is implemented, `CallbackHandler` can be deprecated in favor of `AsyncCallbackHandler`. + ### Extend testdriver - wptrunner transport with async_action + In this alternative, the testdriver - wptrunner transport is extended with a special `async_action` type. Rejected, as testdriver should not be aware of the specific actions implementation. From the testdriver perspective all the actions are asynchronous. + ### Add `AsyncProtocolPart` with `get_loop` method + Instead of exposing the event loop in the WebDriverBidiProtocol, add a dedicated `AsyncProtocolPart` protocol part, which exposes the loop via `get_loop` method. The alternative is rejected, as the event loop is not a part of the protocol, but rather an implementation detail. + ### Add `async_execute_script` to `BaseProtocolPart` + In this alternative, the `BaseProtocolPart` should be extended with the async `async_execute_script` method. The alternative is rejected, as it requires significant and risky changes in the WebDriver Classic client. Brings the same risks as alternative [Extend `CallbackHandler` to support async actions](#extend-callbackhandler-to-support-async-actions). + ### Implement a generic `BidiProtocolPart` + In this alternative, instead of adding protocol parts one-by-one, a generic `BidiProtocolPart` is introduced. Declined, as it would require all the protocol implementations to either support the full WebDrive BiDi protocol or not. From 2be9f03e95f8798cebf9fca21e27a5ceac47a188 Mon Sep 17 00:00:00 2001 From: Maksim Sadym <69349599+sadym-chromium@users.noreply.github.com> Date: Wed, 13 Mar 2024 13:58:00 +0100 Subject: [PATCH 04/16] Update rfcs/testdriver_bidi.md MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Philip Jägenstedt --- rfcs/testdriver_bidi.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rfcs/testdriver_bidi.md b/rfcs/testdriver_bidi.md index d97c433f..217457f2 100644 --- a/rfcs/testdriver_bidi.md +++ b/rfcs/testdriver_bidi.md @@ -12,7 +12,7 @@ Add “testdriver.js” support for [WebDriver BiDi](https://w3c.github.io/webdr The “testdriver.js” framework (or testdriver for simplicity) is based on the one-directional WebDriver Classic protocol and lacks the ability to receive events from the browser via wptrunner, limiting our ability to test event-based functionalities such as console events in WPT (e.g. many [console log tests](https://github.com/web-platform-tests/wpt/tree/master/console) are [manual](https://github.com/web-platform-tests/wpt/blob/master/console/console-timing-logging-manual.html)!). To overcome this limitation, this RFC proposes using the [WebDriver BiDi](https://w3c.github.io/webdriver-bidi) protocol, an extension for WebDriver Classic offering bidirectional communication between the test and the browser. -This document presents changes in testdriver and wptrunner so that it incorporates both WebDriver Classic and WebDriver BiDi functionalities, demonstrating the addition of tests for console events as an [example](https://github.com/web-platform-tests/wpt/pull/44649). +This RFC presents changes in testdriver and wptrunner so that it incorporates both WebDriver Classic and WebDriver BiDi functionalities, demonstrating the addition of tests for console events as an [example](https://github.com/web-platform-tests/wpt/pull/44649). ## Details From 95781fd6ad1a9a339c275552694adb2e9d286b50 Mon Sep 17 00:00:00 2001 From: Maksim Sadym Date: Wed, 10 Apr 2024 14:01:43 +0200 Subject: [PATCH 05/16] Remove `ActionContext` from `AsyncCallbackHandler` --- rfcs/testdriver_bidi.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/rfcs/testdriver_bidi.md b/rfcs/testdriver_bidi.md index 217457f2..a074fdd4 100644 --- a/rfcs/testdriver_bidi.md +++ b/rfcs/testdriver_bidi.md @@ -107,7 +107,9 @@ Analogous to `actions`, introduce an async_actions which will support asynchrono ##### Introduce `AsyncCallbackHandler` -Add a `AsyncCallbackHandler`, which is an extension of `CallbackHandler` with support for async actions. It requires an event loop, in which it will schedule the tasks. Async action processing is done in the same way as CallbackHandler is done, but in the dedicated task. This would not allow raising unexpected exceptions in wptrunner. Testdriver will still be notified about those unexpected exceptions. [Example](https://github.com/web-platform-tests/wpt/pull/44649/files#diff-3de439d55203876d452599a1a14968409df69e5869b16918ceec378a6b3345a4R813) implementation. +Add a `AsyncCallbackHandler`, which is an extension of `CallbackHandler` with support for async actions. It requires an event loop, in which it will schedule the tasks. Async action processing has the following differences from `CallbackHandler`: +* Async action is processed in a dedicated task. This would not allow raising unexpected exceptions in wptrunner. Testdriver will still be notified about those unexpected exceptions. [Example](https://github.com/web-platform-tests/wpt/pull/44649/files#diff-3de439d55203876d452599a1a14968409df69e5869b16918ceec378a6b3345a4R813) implementation. +* Async action does not need an `ActionContext`, as BiDi commands can be sent to any browsing context, not only the currently active one. ##### Extend `WebDriverProtocol` From 2d7804ab3b3eae81b7456ef601052a3a16e762f5 Mon Sep 17 00:00:00 2001 From: Maksim Sadym Date: Wed, 10 Apr 2024 14:02:01 +0200 Subject: [PATCH 06/16] fix whitespaces --- rfcs/testdriver_bidi.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rfcs/testdriver_bidi.md b/rfcs/testdriver_bidi.md index a074fdd4..160e3f72 100644 --- a/rfcs/testdriver_bidi.md +++ b/rfcs/testdriver_bidi.md @@ -138,7 +138,7 @@ In this example we consider changes in [`WebDriverTestharnessExecutor.do_testhar ###### Messages from testdriver to wptrunner -The communication channel between testdriver and wptrunner is set in the `do_testharness` method. Currently the messages from testdriver to wptrunner it is done via long poll by calling BaseProtocolPart.execute_async_script which is synchronous (async in the name indicates that the script is wrapped in a special way before sent to the browser), and blocks processing the event loop in wptrunner, which in turn blocks the event processing while waiting for the next message form testdriver to wptrunner. To overcome this limitation, this RFC changes the wptrunner - testdriver communication channel to asynchronous non-blocking [`BidiScriptProtocolPart.async_call_function`](https://github.com/web-platform-tests/wpt/pull/44649/files#diff-e5a8911dd97e0352b1b26d8ce6ef0a92b25378f7c9e79371a1eb1b1834bc9a8dR761) if available. `WebDriverBidiProtocol`’s event loop should be used for message processing. This behavioral change is done only for protocols supporting BiDi. +The communication channel between testdriver and wptrunner is set in the `do_testharness` method. Currently the messages from testdriver to wptrunner is done via long poll by calling `BaseProtocolPart.execute_async_script` which is synchronous (`async` in the name indicates that the script is wrapped in a special way before sent to the browser), and blocks processing the event loop in wptrunner, which in turn blocks the event processing while waiting for the next message form testdriver to wptrunner. To overcome this limitation, this RFC changes the wptrunner - testdriver communication channel to asynchronous non-blocking [`BidiScriptProtocolPart.async_call_function`](https://github.com/web-platform-tests/wpt/pull/44649/files#diff-e5a8911dd97e0352b1b26d8ce6ef0a92b25378f7c9e79371a1eb1b1834bc9a8dR761) if available. `WebDriverBidiProtocol`’s event loop should be used for message processing. This behavioral change is done only for protocols supporting BiDi. ###### Processing asynchronous actions From d3dc045c8b99c90b51301c5d19411d59781dfb90 Mon Sep 17 00:00:00 2001 From: Maksim Sadym Date: Thu, 11 Apr 2024 11:34:11 +0200 Subject: [PATCH 07/16] remove unsubscribe --- rfcs/testdriver_bidi.md | 31 +++++++++++++------------------ 1 file changed, 13 insertions(+), 18 deletions(-) diff --git a/rfcs/testdriver_bidi.md b/rfcs/testdriver_bidi.md index 160e3f72..7b7bd1aa 100644 --- a/rfcs/testdriver_bidi.md +++ b/rfcs/testdriver_bidi.md @@ -26,17 +26,12 @@ This RFC presents changes in testdriver and wptrunner so that it incorporates bo Here is an [example](https://github.com/sadym-chromium/wpt/blob/sadym/testdriver-bidi/console/console-log-logged.html) of a test using the proposed `test_driver.bidi` API for testing console log events. ```javascript -promise_test(async (test) => { +promise_test(async () => { const some_message = "SOME MESSAGE"; // Subscribe to `log.entryAdded` BiDi events. This will not add a listener to the page. await test_driver.bidi.log.entry_added.subscribe(); - test.add_cleanup(async function() { - // Unsubscribe from `log.entryAdded` BiDi events. - await test_driver.bidi.log.entry_added.unsubscribe(); - }); // Add a listener for the log.entryAdded event. This will not subscribe to the event, so the subscription is - // required before. After the event is received, the subscription will not be removed, so the cleanup is - // required. + // required before. The cleanup is done automatically after the test is finished. const log_entry_promise = test_driver.bidi.log.entry_added.once(); // Emit a console.log message. console.log(some_message) @@ -76,10 +71,9 @@ The BiDi actions are functionally equivalent to the classic actions. They are pl ###### Events The exposed events API in `test_driver` provides the following methods for each event: -* `test_driver.bidi.{module}.{event}.subscribe(): Promise`. The async method subscribes to the given WebDriver BiDi event, but does not add event listeners yet. -* `test_driver.bidi.{module}.{event}.unsubscribe(): Promise`. The async method unsubscribes from the given WebDriver BiDi event. +* `test_driver.bidi.{module}.{event}.subscribe(): Promise`. The async method subscribes to the given WebDriver BiDi event, but does not add event listeners yet. The unsubscription is done by the wptrunner after each test. * `test_driver.bidi.{module}.{event}.on(handler: (event)=>void): ()=>void`. The method makes the handler be called each time the event emitted. The callback argument is the event params. It returns a handle for removing this listener. -* (optional) `test_driver.bidi.{module}.{event}.once(): Promise`. Creates a listener and returns a promise, which will be resolved after the required event is emitted for the first time. Removes the listener after the first event. This method is not required, but allows easier writing of some tests (e.g. it is used in the example above). It can be easily implemented via the `on` method. Note: this wrapper does not unsubscribe from the event. +* (optional) `test_driver.bidi.{module}.{event}.once(): Promise`. Creates a listener and returns a promise, which will be resolved after the required event is emitted for the first time. Removes the listener after the first event. This method is not required, but allows easier writing of some tests (e.g. it is used in the example above). It can be easily implemented via the `on` method. ###### Alternatives @@ -90,7 +84,7 @@ The exposed events API in `test_driver` provides the following methods for each ##### Event processing -The proposal extends the event processing implemented in [`testdriver-extra.js`](https://github.com/web-platform-tests/wpt/blob/master/tools/wptrunner/wptrunner/testdriver-extra.js) with a new message type [“testdriver-event”](https://github.com/web-platform-tests/wpt/pull/44649/files#diff-46aeeb40b0a0c13031b151392ca70a17614295533d3e890c0cb4360cb3b91542R43). This message is sent by the wptrunner to notify the testdriver about new events. It uses the `message` field to store JSON-serialized `params` and `method`. The `testdriver-extra.js` script overrides `window.test_driver_internal.bidi.{modules}.{event}` object methods `subscribe`, `unsubscribe`, and `on`. +The proposal extends the event processing implemented in [`testdriver-extra.js`](https://github.com/web-platform-tests/wpt/blob/master/tools/wptrunner/wptrunner/testdriver-extra.js) with a new message type [“testdriver-event”](https://github.com/web-platform-tests/wpt/pull/44649/files#diff-46aeeb40b0a0c13031b151392ca70a17614295533d3e890c0cb4360cb3b91542R43). This message is sent by the wptrunner to notify the testdriver about new events. It uses the `message` field to store JSON-serialized `params` and `method`. The `testdriver-extra.js` script overrides `window.test_driver_internal.bidi.{modules}.{event}` object methods `subscribe`, and `on`. #### Wptrunner @@ -156,7 +150,10 @@ If `BidiEventsProtocolPart` is available, wptrunner sets a callback via [`add_ev ###### `BidiEventsProtocolPart` -To provide events from wptrunner to testdriver, the [`BidiEventsProtocolPart`](https://github.com/web-platform-tests/wpt/pull/44649/files#diff-47192f72de48b834a50992ada793229686299b5165612c2d97af6811764514c7R325) implements [`add_event_listener`](https://github.com/web-platform-tests/wpt/pull/44649/files#diff-47192f72de48b834a50992ada793229686299b5165612c2d97af6811764514c7R345). `TestExecutor` will subscribe and forward events to testdriver. +[`BidiEventsProtocolPart`](https://github.com/web-platform-tests/wpt/pull/44649/files#diff-47192f72de48b834a50992ada793229686299b5165612c2d97af6811764514c7R325) is responsible for subscribing to events, forwarding them to testdriver, and cleaning up the subscription state between test runs. It implements the following methods: +* [`add_event_listener`](https://github.com/web-platform-tests/wpt/pull/44649/files#diff-47192f72de48b834a50992ada793229686299b5165612c2d97af6811764514c7R345). `TestExecutor` will subscribe and forward events to testdriver using this method. +* [`subscribe(events, contexts)`](https://github.com/web-platform-tests/wpt/pull/44649/files#diff-e5a8911dd97e0352b1b26d8ce6ef0a92b25378f7c9e79371a1eb1b1834bc9a8dR123). Subscribe to BiDi events. +* [`unsubscribe_all`](https://github.com/web-platform-tests/wpt/pull/44649/files#diff-e5a8911dd97e0352b1b26d8ce6ef0a92b25378f7c9e79371a1eb1b1834bc9a8dR128). This method removes all the subscriptions that were added before. `TestExecutor` will call it between tests. ###### `BidiScriptProtocolPart` @@ -248,13 +245,11 @@ In this alternative, the `test_driver.bidi.{module}.{event}.subscribe()` method ### Use `EventTarget`-like events API In this alternative, an [`EventTarget`](https://developer.mozilla.org/en-US/docs/Web/API/EventTarget)-like API is exposed for BiDi events: -`addEventListener(type, listener, options)`. -Options: -* `once: bool` Optional. A boolean value indicating that the listener should be invoked at most once after being added. If true, the listener would be automatically removed when invoked. If not specified, defaults to false. -* `subscribe: bool` indicates if the subscription should be set, if not yet. Defaults to `true`. -* `removeEventListener(type, listener, options)`. +* `addEventListener(type, listener, options)`. Options: - * `unsubscribe: bool` indicates if the subscription should be stopped. Defaults to `true`. + * `once: bool` Optional. A boolean value indicating that the listener should be invoked at most once after being added. If true, the listener would be automatically removed when invoked. If not specified, defaults to false. + * `subscribe: bool` indicates if the subscription should be set, if not yet. Defaults to `true`. +* `removeEventListener(type, listener, options)`. This API is more common. However, more workarounds are required for it to be used in tests. Also, the subscription handling is harder and less explicit. The alternative was declined in order to prioritize test driver ergonomics. From c39e1db7393e4fd83f8f5a9028ef5c27cab49827 Mon Sep 17 00:00:00 2001 From: Maksim Sadym <69349599+sadym-chromium@users.noreply.github.com> Date: Fri, 12 Apr 2024 10:15:31 +0200 Subject: [PATCH 08/16] Update README.md Co-authored-by: Alex Rudenko --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 81178036..f74e81a5 100644 --- a/README.md +++ b/README.md @@ -21,7 +21,7 @@ or users of web-platform-tests. Examples of where it is likely to be useful: Cases where the RFC process need *not* be used: - Introducing a new top-level directory for a new specification. - - Extending testdriver.js with a method that closely matches a [WebDriver Classic](https://w3c.github.io/webdriver/) or [WebDriver BiDi](https://w3c.github.io/webdriver-bidi) endpoints. To notify maintainers of testdriver.js vendor integration, label the pull request `testdriver.js`. (This exemption was introduced by [RFC 127](https://github.com/web-platform-tests/rfcs/blob/master/rfcs/rfc_exemption_testdriver_method.md) and [RFC 185](https://github.com/web-platform-tests/rfcs/blob/master/rfcs/testdriver_bidi.md).) + - Extending testdriver.js with a method that closely matches a [WebDriver Classic](https://w3c.github.io/webdriver/) endpoint or a [WebDriver BiDi](https://w3c.github.io/webdriver-bidi) command or an event. To notify maintainers of testdriver.js vendor integration, label the pull request `testdriver.js`. (This exemption was introduced by [RFC 127](https://github.com/web-platform-tests/rfcs/blob/master/rfcs/rfc_exemption_testdriver_method.md) and [RFC 185](https://github.com/web-platform-tests/rfcs/blob/master/rfcs/testdriver_bidi.md).) - Minor changes in behavior in where all call sites are known and accounted for. - Behavior-preserving refactoring with a low risk of regressions. From d75df88332b2908a23f3e2d604bfcc0005dbef51 Mon Sep 17 00:00:00 2001 From: Maksim Sadym Date: Mon, 15 Apr 2024 16:24:16 +0200 Subject: [PATCH 09/16] update text --- rfcs/testdriver_bidi.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rfcs/testdriver_bidi.md b/rfcs/testdriver_bidi.md index 7b7bd1aa..b127cfe9 100644 --- a/rfcs/testdriver_bidi.md +++ b/rfcs/testdriver_bidi.md @@ -50,7 +50,7 @@ The proposal changes the following parts: #### Testdriver -The testdriver requires support of event processing, and exposing it as an API. +The testdriver requires support of bidi actions, aevent processing, and exposing it as an API. ##### Testdriver API From 3a405fc65a33154bec7949e3e1e9a253e8d8e37e Mon Sep 17 00:00:00 2001 From: Maksim Sadym Date: Mon, 15 Apr 2024 16:43:10 +0200 Subject: [PATCH 10/16] Add `Wrap event into a class` alternative --- rfcs/testdriver_bidi.md | 36 ++++++++++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+) diff --git a/rfcs/testdriver_bidi.md b/rfcs/testdriver_bidi.md index b127cfe9..fffbb3ff 100644 --- a/rfcs/testdriver_bidi.md +++ b/rfcs/testdriver_bidi.md @@ -77,6 +77,7 @@ The exposed events API in `test_driver` provides the following methods for each ###### Alternatives +* [Wrap event into a class](#wrap-event-into-a-class). * [Make generic event emitting in testdriver](#make-generic-event-emitting-in-testdriver). * [Expose generic `session.subscribe` method in `test_driver` instead of `test_driver.bidi.{module}.{event}.subscribe()`](#expose-generic--sessionsubscribe-method-in-test_driver-instead-of-test_driverbidimoduleeventsubscribe) * [Do not expose `test_driver.bidi.{module}.{event}.subscribe()`, do it silently in the `on` method](#do-not-expose-test_driverbidimoduleeventsubscribe-do-it-silently-in-the-onmethod) @@ -220,6 +221,41 @@ No need in introducing a new namespace, and continue using the established patte * Harder to determine if a specific testdriver method requires WebDriver BiDi support: Developers may need to consult the documentation or code to determine if a particular method requires WebDriver BiDi support. * Can be done later after WebDriver BiDi is widely adopted, the `test_drive`’s root-level methods can be silently shifted to use BiDi under the hood. +### Wrap event into a class + +As an alternative, the event is encapsulated within a class to enhance usability. +```javascript +promise_test(async () => { + const some_message = "SOME MESSAGE"; + // Get `log.entryAdded` BiDi event handler. + const entry_added_event_handler = await test_driver.bidi.log.entry_added.activate(); + // Clean the `entry_added_event_handler`, as it can contain buffered events. + entry_added_event_handler.clean(); + // Emit a console.log message. + console.log(some_message) + // Wait for the log.entryAdded event to be received. + const event = await entry_added_event_handler.next(); + // Assert the log.entryAdded event has the expected message. + assert_equals(event.args.length, 1); + const event_message = event.args[0]; + assert_equals(event_message.value, some_message); +}, "Assert log event is logged"); +``` + +The event handler class has the following interface: +```javascript +class EventHandler{ + // Create and activate the event handler for a specific event. + static activate(): Promise; + // Clean the event queue, as some events can be buffered. + clean(): EventEntry[]; + // Return a promise which is resolved with the next log entry once one is received. + next(): Promise; + // Set a custom event handler. Returns a remove handler. + on(callback: (EventEntry)=>void): ()=>void; +} +``` + ### Make generic event emitting in testdriver In this alternative, `test_driver` exposes an `EventTarget`, or methods `addEventListener(type, listener)`, and `removeEventListener(type, listener)`. Test will be able to subscribe to any WebDriver BiDi event. From 82ccd50f00cfc4b604cbdde25367bab686d70144 Mon Sep 17 00:00:00 2001 From: Maksim Sadym Date: Mon, 15 Apr 2024 17:17:57 +0200 Subject: [PATCH 11/16] Remove `once` --- rfcs/testdriver_bidi.md | 22 ++++++++++++++-------- 1 file changed, 14 insertions(+), 8 deletions(-) diff --git a/rfcs/testdriver_bidi.md b/rfcs/testdriver_bidi.md index fffbb3ff..ff519c56 100644 --- a/rfcs/testdriver_bidi.md +++ b/rfcs/testdriver_bidi.md @@ -32,7 +32,9 @@ promise_test(async () => { await test_driver.bidi.log.entry_added.subscribe(); // Add a listener for the log.entryAdded event. This will not subscribe to the event, so the subscription is // required before. The cleanup is done automatically after the test is finished. - const log_entry_promise = test_driver.bidi.log.entry_added.once(); + let on_log_entry; + const log_entry_promise = new Promise(r => on_log_entry = r); + test_driver.bidi.log.entry_added.on(on_log_entry); // Emit a console.log message. console.log(some_message) // Wait for the log.entryAdded event to be received. @@ -73,7 +75,6 @@ The BiDi actions are functionally equivalent to the classic actions. They are pl The exposed events API in `test_driver` provides the following methods for each event: * `test_driver.bidi.{module}.{event}.subscribe(): Promise`. The async method subscribes to the given WebDriver BiDi event, but does not add event listeners yet. The unsubscription is done by the wptrunner after each test. * `test_driver.bidi.{module}.{event}.on(handler: (event)=>void): ()=>void`. The method makes the handler be called each time the event emitted. The callback argument is the event params. It returns a handle for removing this listener. -* (optional) `test_driver.bidi.{module}.{event}.once(): Promise`. Creates a listener and returns a promise, which will be resolved after the required event is emitted for the first time. Removes the listener after the first event. This method is not required, but allows easier writing of some tests (e.g. it is used in the example above). It can be easily implemented via the `on` method. ###### Alternatives @@ -227,14 +228,17 @@ As an alternative, the event is encapsulated within a class to enhance usability ```javascript promise_test(async () => { const some_message = "SOME MESSAGE"; - // Get `log.entryAdded` BiDi event handler. - const entry_added_event_handler = await test_driver.bidi.log.entry_added.activate(); - // Clean the `entry_added_event_handler`, as it can contain buffered events. - entry_added_event_handler.clean(); + // Subscribe to `log.entryAdded` BiDi events in the current window. This will not add a listener to the page. + await test_driver.bidi.log.entry_added.subscribe({contexts: [window]}); + // Add a listener for the log.entryAdded event. This will not subscribe to the event, so the subscription is + // required before. The cleanup is done automatically after the test is finished. + let on_log_entry; + const log_entry_promise = new Promise(r => on_log_entry = r); + test_driver.bidi.log.entry_added.on(on_log_entry); // Emit a console.log message. console.log(some_message) // Wait for the log.entryAdded event to be received. - const event = await entry_added_event_handler.next(); + const event = await log_entry_promise; // Assert the log.entryAdded event has the expected message. assert_equals(event.args.length, 1); const event_message = event.args[0]; @@ -276,7 +280,9 @@ In this alternative, the `test_driver.session.subscribe` method is exposed in th In this alternative, the `test_driver.bidi.{module}.{event}.subscribe()` method is not exposed in the testdriver API. Instead, each `on` handler subscribes to the required event. The API becomes simpler, as test writers would not require care about the concept of WebDriver BiDi subscriptions at all. Nonetheless, this option was rejected because: * Event enabling API in BiDi is done pre-session. Meaning if `event.on()` is called twice, and after that the first listener is removed, should the subscription be removed as well? -* The `once` handler cannot be implemented since subscription is an asynchronous process. +* The API is less explicit, as the subscription is done silently. +* It's unclear how to handle the subscription properties. +* It's unclear how to handle event buffer (events are emitted before the `subscribe` command is done). ### Use `EventTarget`-like events API From a44d0e619749555c4b438f024c1d41c67bb5070e Mon Sep 17 00:00:00 2001 From: Maksim Sadym <69349599+sadym-chromium@users.noreply.github.com> Date: Tue, 16 Apr 2024 08:18:35 +0200 Subject: [PATCH 12/16] Update rfcs/testdriver_bidi.md Co-authored-by: Alex Rudenko --- rfcs/testdriver_bidi.md | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/rfcs/testdriver_bidi.md b/rfcs/testdriver_bidi.md index ff519c56..a6a811d2 100644 --- a/rfcs/testdriver_bidi.md +++ b/rfcs/testdriver_bidi.md @@ -32,9 +32,7 @@ promise_test(async () => { await test_driver.bidi.log.entry_added.subscribe(); // Add a listener for the log.entryAdded event. This will not subscribe to the event, so the subscription is // required before. The cleanup is done automatically after the test is finished. - let on_log_entry; - const log_entry_promise = new Promise(r => on_log_entry = r); - test_driver.bidi.log.entry_added.on(on_log_entry); + const log_entry_promise = new Promise(resolve => test_driver.bidi.log.entry_added.on(resolve)); // Emit a console.log message. console.log(some_message) // Wait for the log.entryAdded event to be received. From 8fcab7d407b209fa1b2ea25eb2387a1fd861a9e1 Mon Sep 17 00:00:00 2001 From: Maksim Sadym <69349599+sadym-chromium@users.noreply.github.com> Date: Tue, 16 Apr 2024 08:18:41 +0200 Subject: [PATCH 13/16] Update rfcs/testdriver_bidi.md Co-authored-by: Alex Rudenko --- rfcs/testdriver_bidi.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rfcs/testdriver_bidi.md b/rfcs/testdriver_bidi.md index a6a811d2..8d56ef7c 100644 --- a/rfcs/testdriver_bidi.md +++ b/rfcs/testdriver_bidi.md @@ -50,7 +50,7 @@ The proposal changes the following parts: #### Testdriver -The testdriver requires support of bidi actions, aevent processing, and exposing it as an API. +The testdriver requires support of bidi actions, event processing, and exposing it as an API. ##### Testdriver API From d41cea24e8565366c3db8d4b9a734e1214c6bf4d Mon Sep 17 00:00:00 2001 From: Maksim Sadym <69349599+sadym-chromium@users.noreply.github.com> Date: Tue, 16 Apr 2024 08:19:07 +0200 Subject: [PATCH 14/16] Update rfcs/testdriver_bidi.md Co-authored-by: Alex Rudenko --- rfcs/testdriver_bidi.md | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/rfcs/testdriver_bidi.md b/rfcs/testdriver_bidi.md index 8d56ef7c..af706e44 100644 --- a/rfcs/testdriver_bidi.md +++ b/rfcs/testdriver_bidi.md @@ -230,9 +230,7 @@ promise_test(async () => { await test_driver.bidi.log.entry_added.subscribe({contexts: [window]}); // Add a listener for the log.entryAdded event. This will not subscribe to the event, so the subscription is // required before. The cleanup is done automatically after the test is finished. - let on_log_entry; - const log_entry_promise = new Promise(r => on_log_entry = r); - test_driver.bidi.log.entry_added.on(on_log_entry); + const log_entry_promise = new Promise(resolve => test_driver.bidi.log.entry_added.on(resolve)); // Emit a console.log message. console.log(some_message) // Wait for the log.entryAdded event to be received. From 753f5881e6530d38f21140cd5e8661187d3737ea Mon Sep 17 00:00:00 2001 From: Maksim Sadym Date: Tue, 16 Apr 2024 09:32:01 +0200 Subject: [PATCH 15/16] Remove `once` --- rfcs/testdriver_bidi.md | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-) diff --git a/rfcs/testdriver_bidi.md b/rfcs/testdriver_bidi.md index af706e44..8baa3b38 100644 --- a/rfcs/testdriver_bidi.md +++ b/rfcs/testdriver_bidi.md @@ -226,15 +226,14 @@ As an alternative, the event is encapsulated within a class to enhance usability ```javascript promise_test(async () => { const some_message = "SOME MESSAGE"; - // Subscribe to `log.entryAdded` BiDi events in the current window. This will not add a listener to the page. - await test_driver.bidi.log.entry_added.subscribe({contexts: [window]}); - // Add a listener for the log.entryAdded event. This will not subscribe to the event, so the subscription is - // required before. The cleanup is done automatically after the test is finished. - const log_entry_promise = new Promise(resolve => test_driver.bidi.log.entry_added.on(resolve)); + // Get `log.entryAdded` BiDi event handler. + const entry_added_event_handler = await test_driver.bidi.log.entry_added.activate(); + // Clean the `entry_added_event_handler`, as it can contain buffered events. + entry_added_event_handler.clean(); // Emit a console.log message. console.log(some_message) // Wait for the log.entryAdded event to be received. - const event = await log_entry_promise; + const event = await entry_added_event_handler.next(); // Assert the log.entryAdded event has the expected message. assert_equals(event.args.length, 1); const event_message = event.args[0]; @@ -244,17 +243,20 @@ promise_test(async () => { The event handler class has the following interface: ```javascript + class EventHandler{ // Create and activate the event handler for a specific event. - static activate(): Promise; - // Clean the event queue, as some events can be buffered. + static activate(preserver_buffer: bool): Promise; + // Clean the event queue. Can be used to ignore buffered events. Returns the cleaned events. clean(): EventEntry[]; // Return a promise which is resolved with the next log entry once one is received. next(): Promise; // Set a custom event handler. Returns a remove handler. on(callback: (EventEntry)=>void): ()=>void; } + ``` +This alternative is declined for the moment, as it can be implemented over proposed `subscribe` + `on` API. ### Make generic event emitting in testdriver From 4122e8c97a58dc1ed222e714602272bcf311c30d Mon Sep 17 00:00:00 2001 From: Maksim Sadym Date: Mon, 22 Apr 2024 12:04:39 +0200 Subject: [PATCH 16/16] Rename `async_call_function` -> `call_function` --- rfcs/testdriver_bidi.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/rfcs/testdriver_bidi.md b/rfcs/testdriver_bidi.md index 8baa3b38..4e0e8682 100644 --- a/rfcs/testdriver_bidi.md +++ b/rfcs/testdriver_bidi.md @@ -132,7 +132,7 @@ In this example we consider changes in [`WebDriverTestharnessExecutor.do_testhar ###### Messages from testdriver to wptrunner -The communication channel between testdriver and wptrunner is set in the `do_testharness` method. Currently the messages from testdriver to wptrunner is done via long poll by calling `BaseProtocolPart.execute_async_script` which is synchronous (`async` in the name indicates that the script is wrapped in a special way before sent to the browser), and blocks processing the event loop in wptrunner, which in turn blocks the event processing while waiting for the next message form testdriver to wptrunner. To overcome this limitation, this RFC changes the wptrunner - testdriver communication channel to asynchronous non-blocking [`BidiScriptProtocolPart.async_call_function`](https://github.com/web-platform-tests/wpt/pull/44649/files#diff-e5a8911dd97e0352b1b26d8ce6ef0a92b25378f7c9e79371a1eb1b1834bc9a8dR761) if available. `WebDriverBidiProtocol`’s event loop should be used for message processing. This behavioral change is done only for protocols supporting BiDi. +The communication channel between testdriver and wptrunner is set in the `do_testharness` method. Currently the messages from testdriver to wptrunner is done via long poll by calling `BaseProtocolPart.execute_async_script` which is synchronous (`async` in the name indicates that the script is wrapped in a special way before sent to the browser), and blocks processing the event loop in wptrunner, which in turn blocks the event processing while waiting for the next message form testdriver to wptrunner. To overcome this limitation, this RFC changes the wptrunner - testdriver communication channel to asynchronous non-blocking [`BidiScriptProtocolPart.call_function`](https://github.com/web-platform-tests/wpt/pull/44649/files#diff-e5a8911dd97e0352b1b26d8ce6ef0a92b25378f7c9e79371a1eb1b1834bc9a8dR761) if available. `WebDriverBidiProtocol`’s event loop should be used for message processing. This behavioral change is done only for protocols supporting BiDi. ###### Processing asynchronous actions @@ -157,7 +157,7 @@ If `BidiEventsProtocolPart` is available, wptrunner sets a callback via [`add_ev ###### `BidiScriptProtocolPart` -[`BidiScriptProtocolPart`](https://github.com/web-platform-tests/wpt/pull/44649/files#diff-47192f72de48b834a50992ada793229686299b5165612c2d97af6811764514c7R358) is required for implementing asynchronous poll messages from testdriver. It implements async [`async_call_function`](https://github.com/web-platform-tests/wpt/pull/44649/files#diff-e5a8911dd97e0352b1b26d8ce6ef0a92b25378f7c9e79371a1eb1b1834bc9a8dR142) action. +[`BidiScriptProtocolPart`](https://github.com/web-platform-tests/wpt/pull/44649/files#diff-47192f72de48b834a50992ada793229686299b5165612c2d97af6811764514c7R358) is required for implementing asynchronous poll messages from testdriver. It implements async [`call_function`](https://github.com/web-platform-tests/wpt/pull/44649/files#diff-e5a8911dd97e0352b1b26d8ce6ef0a92b25378f7c9e79371a1eb1b1834bc9a8dR142) action. ###### Alternative