Skip to content

Commit ed83929

Browse files
authored
Refactor "pressKeys" as one of many "user intents" (#76)
In conversation with assistive technology vendors, we have learned that automating arbitrary key presses does not necessarily align with the security model of all platforms. Separately, we anticipate introducing means to simulate more specific types of user interaction (e.g. "move to next heading"). Because these may themselves be implemented at the vendor's discretion (and because we wish to allow for vendors to experiment with the development of still more kinds of interaction), the space for simulating interaction may be somewhat fragmented, particularly in the initial deployments of this protocol. We have decided to design a command which accommodates this variability so that the presence/absence of interactions can be communicated consistently (note the "unknown user intent" error) and so that additional interactions can be introduced without the addition of entirely new commands (or entirely new extension commands, as the case may be).
1 parent 44df4b8 commit ed83929

File tree

2 files changed

+91
-33
lines changed

2 files changed

+91
-33
lines changed

index.bs

+75-27
Original file line numberDiff line numberDiff line change
@@ -337,6 +337,10 @@ The following table lists each <dfn>error code</dfn>, its associated JSON `error
337337
<td><dfn for="error code">session not created</dfn>
338338
<td>`session not created`
339339
<td>A new [=session=] could not be created.
340+
<tr>
341+
<td><dfn for="error code">unknown user intent</dfn>
342+
<td>`unknown user intent`
343+
<td>The remote end does not support a user intent with the provided name.
340344
<tr>
341345
<td><dfn for="error code">cannot simulate keyboard interaction</dfn>
342346
<td>`cannot simulate keyboard interaction`
@@ -824,12 +828,53 @@ Issue: Do we need a "setting changed" event?
824828
The Interaction Module {#module-interaction}
825829
--------------------------------------------
826830

831+
The following <dfn>table of standard user intents</dfn> enumerates the user intents each implementation must support.
832+
833+
<table>
834+
<caption>Standard user intents</caption>
835+
<tr>
836+
<th>Name
837+
<th>Algorithm
838+
<tr>
839+
<td>"`pressKeys`"
840+
<td>[=press keys=]
841+
</table>
842+
843+
A [=remote end=] has a <dfn for="remote end">table of extension user intents</dfn>, which is a mapping of zero or more string names of user intents and algorithms for simulating the named intents. Extension user intents' names must contain a "`:`" (colon) character, denoting an implementation specific namespace.
844+
845+
<div algorithm>
846+
847+
Note: Each string in `KeyCombination` represents a "raw key" consisting of a
848+
single code point with the same meaning as in <a
849+
href="https://w3c.github.io/webdriver/#keyboard-actions">WebDriver's keyboard
850+
actions</a>. For example, `["\uE008", "a"]` means holding the left shift key
851+
and pressing "a", and then releasing the left shift key. [[WEBDRIVER]]
852+
853+
Issue(34): This algorithm does not yet have a means for indicating a screen-reader specific modifier key (or keys).
854+
855+
Issue(51): This algorithm only supports one specific kind of press/release sequence, and it is not clear if that is sufficient to express all keyboard commands in all implementations.
856+
857+
To <dfn>press keys</dfn> given |command parameters|:
858+
859+
1. [=Try=] to [=check that keyboard interaction can be simulated=].
860+
2. [=Try=] to [=check that one of the expected applications has focus=].
861+
3. Let |keys| be the value of the <code>keys</code> field of |command
862+
parameters|.
863+
4. [=list/For each=] |key| of |keys|:
864+
1. Run [=implementation-defined=] steps to simulate depressing |key|.
865+
5. [=list/For each=] |key| of |keys| in reverse [=List=] order:
866+
1. Run [=implementation-defined=] steps to simulate releasing |key|.
867+
6. Let |body| be a new [=map=].
868+
7. Return [=success=] with data |body|.
869+
870+
</div>
871+
827872
### Definition ### {#module-interaction-definition}
828873

829874
[=Remote end definition=]:
830875

831876
<xmp class="cddl remote-cddl">
832-
InteractionCommand = (InteractionPressKeysCommand)
877+
InteractionCommand = (InteractionUserIntentCommand)
833878
</xmp>
834879

835880
[=Local end definition=]:
@@ -851,32 +896,38 @@ InteractionCapturedOutputParameters = {
851896

852897
### Commands ### {#module-interaction-commands}
853898

854-
#### The interaction.pressKeys Command #### {#module-interaction-presskeys}
855-
856-
The <dfn>interaction.pressKeys</dfn> command simulates pressing a key combination on a keyboard.
899+
#### The interaction.userIntent Command #### {#module-interaction-userintent}
857900

858-
Issue(34): This command does not yet have a means for indicating a screen-reader specific modifier key (or keys).
859-
860-
Issue(51): This algorithm only supports one specific kind of press/release sequence, and it is not clear if that is sufficient to express all keyboard commands in all implementations.
901+
The <dfn>interaction.userIntent</dfn> command simulates pressing a key combination on a keyboard.
861902

862903
<dl>
863904
<dt>[=Command Type=]
864905
<dd>
865906

866907
<pre class="cddl remote-cddl">
867-
InteractionPressKeysCommand = {
868-
method: "interaction.pressKeys",
869-
params: InteractionPressKeysParameters
908+
InteractionUserIntentCommand = {
909+
method: "interaction.userIntent",
910+
params: InteractionUserIntentParameters
870911
}
871912

872-
InteractionPressKeysParameters = {
913+
InteractionUserIntentParameters = (
914+
PressKeysIntentParameters /
915+
ExtensionIntentParameters
916+
)
917+
918+
PressKeysIntentParameters = {
919+
"name" => "pressKeys",
873920
"keys" => KeyCombination,
874-
Extensible,
875921
}
876922

877923
KeyCombination = [
878924
1* text
879925
]
926+
927+
ExtensionIntentParameters = {
928+
"name" => text,
929+
Extensible,
930+
}
880931
</pre>
881932

882933
<dt>Result Type
@@ -888,24 +939,21 @@ Issue(51): This algorithm only supports one specific kind of press/release seque
888939

889940
</dl>
890941

891-
Note: Each string in `KeyCombination` represents a "raw key" consisting of a
892-
single code point with the same meaning as in <a
893-
href="https://w3c.github.io/webdriver/#keyboard-actions">WebDriver's keyboard
894-
actions</a>. For example, `["\uE008", "a"]` means holding the left shift key
895-
and pressing "a", and then releasing the left shift key. [[WEBDRIVER]]
942+
<div algorithm="remote end steps for interaction.userIntent">
896943

897-
The [=remote end steps=] given |session| and |command parameters| are:
944+
The [=remote end steps=] given <var ignore>session</var> and |command parameters| are:
898945

899-
1. [=Try=] to [=check that keyboard interaction can be simulated=].
900-
2. [=Try=] to [=check that one of the expected applications has focus=].
901-
3. Let |keys| be the value of the <code>keys</code> field of |command
946+
1. Let |name| be the value of the <code>name</code> field of |command
902947
parameters|.
903-
4. [=list/For each=] |key| of |keys|:
904-
1. Run [=implementation-defined=] steps to simulate depressing |key|.
905-
5. [=list/For each=] |key| of |keys| in reverse [=List=] order:
906-
1. Run [=implementation-defined=] steps to simulate releasing |key|.
907-
6. Let |body| be a new [=map=].
908-
7. Return [=success=] with data |body|.
948+
2. If there is an entry in the [=table of standard user intents=] with name |name|:
949+
1. Let |algorithm| be the algorithm associated with user intent |name| in the [=table of standard user intents=].
950+
3. Otherwise, if there is an entry in the [=table of extension user intents=] with name |name|:
951+
1. Let |algorithm| be the algorithm associated with user intent |name| in the [=table of extension user intents=].
952+
4. Otherwise:
953+
1. Return an [=error=] with [=error code=] [=unknown user intent=].
954+
5. Return the result of evaluating |algorithm| given |command parameters|.
955+
956+
</div>
909957

910958
### Events ### {#module-interaction-events}
911959

schemas/at-driver-remote.cddl

+16-6
Original file line numberDiff line numberDiff line change
@@ -74,18 +74,28 @@ SettingsGetSupportedSettingsCommand = {
7474
params: EmptyParams
7575
}
7676

77-
InteractionCommand = (InteractionPressKeysCommand)
77+
InteractionCommand = (InteractionUserIntentCommand)
7878

79-
InteractionPressKeysCommand = {
80-
method: "interaction.pressKeys",
81-
params: InteractionPressKeysParameters
79+
InteractionUserIntentCommand = {
80+
method: "interaction.userIntent",
81+
params: InteractionUserIntentParameters
8282
}
8383

84-
InteractionPressKeysParameters = {
84+
InteractionUserIntentParameters = (
85+
PressKeysIntentParameters /
86+
ExtensionIntentParameters
87+
)
88+
89+
PressKeysIntentParameters = {
90+
"name" => "pressKeys",
8591
"keys" => KeyCombination,
86-
Extensible,
8792
}
8893

8994
KeyCombination = [
9095
1* text
9196
]
97+
98+
ExtensionIntentParameters = {
99+
"name" => text,
100+
Extensible,
101+
}

0 commit comments

Comments
 (0)