Data-navigator: Enabling navigable chart elements with alt text #483
Conversation
|
While this PR isn't ready to be merged, what I hope for is some feedback on the general approach before populating the prototype's approach across all other visualization types. Right now the navigation capabilities are pretty cool, even with a never-before accomplished navigation type:
The key bindings are set in the constants file, but obviously we can change these across charts and even allow end-users to set their own keybinds too (outside of the scope of this PR but worth thinking about!). |
|
@frankelavsky sounds good. @c-lamoureux and I will try and provide feedback in the next couple of days. I'll try it out today and look through the approach. |
|
@marshallpete sounds good and thanks! Note that I recommend entering the chart via keyboard. It'll show a big group indicator and I've got some design/ui ideas to help with wayfinding and keyboard commands we can think about too. And any of this, from nav structure to keybinds to code quality can all be points of feedback. Nothing is sacred, it's a prototype after all. |
I think we agree here. I lean towards using left / right / up / down for navigation, Space/Enter for drilling in, and Escape for drilling out. The fewer unfamiliar interaction patterns, like |
|
I like what you have here. I think the approach makes sense. As this is intended to be a POC/Prototype, I'm not too worried about code structure and cleanliness. The behavior lines up with what we had discussed and what I understood from our discussions. The styling of things will probably be tweaked for a production version. We will have our designers provide feedback but once again, since this is a POC/prototype, I think it serves it's purpose. |
|
@frankelavsky We have to account for the mobile/touch screen reader experience when navigating the chart. Javascript cannot handle swipe gestures to navigate from a mobile screen reader in the same way that it can handle keyboard or pointer events when the screen reader is not running, it's a privacy issue. The data navigator will need to include both the focused element and any item before or after it in the navigation order, so that as the user swipes to navigate, there is an element within the chart to navigate to. Also, since javascript cannot interpret complicated gestures with a screen reader running, we should make sure that any ways to change the navigation mode are available to mobile screen reader users using simple double tap to activate gestures. |
@majornista - heck yes, absolutely agree. We used the 1-before, 1-after approach at Visa specifically for this reason (rendering these gives fallback support). I reckon we should default to the previous/next element that exists in the same dimension that we are assigning to the left/right directions.
Also totally agree. This means that we will limit dimensional navigation to 2 dimensions. So for cases like a color-encoded scatterplot we might run into some trouble! The left/right would move along x values, and up/down along y values.... but what do we do if there is also a legend that shows color categories? In the Zong et al research project ("rich screen reader experiences", figure below), they had separate navigation spaces that were completely separated from each other: one was up/down/left/right and drill in/out (for x and y) and the other was via categories with left/right and drill in/out. Maybe we could let users choose which path at the start? Or maybe have a key command (like pressing X or something) that swaps between "modes?"
Also (in additional to the above stuff) I think that we need at least 2 things to make this prototype really usable:
|
|
Just a reflection of the coding to-dos before this will be hand-off ready:
We have a few to-dos that I didn't assign to myself, because someone else could definitely build them out in parallel. If anyone wants to do new work, I think that adding the signal for focusing to the spec would be the most helpful. And feel free to mention if there is anything I've forgotten here! |
| signals[SELECTED_ITEM] = selectedData?.[idKey] ?? null; | ||
| signals[SELECTED_SERIES] = selectedData?.[SERIES_ID] ?? null; | ||
|
|
||
| return signals; | ||
| }, [colorScheme, idKey, legendHiddenSeries, legendIsToggleable]); | ||
|
|
||
| const navigationEventCallback = (navData: NavigationEvent) => { | ||
| console.log("RSC chart can use this navData to set signals", navData) | ||
| // set signals here! |
There was a problem hiding this comment.
Update for @marshallpete:
I solved our valid ID issue and am emitting events from the Navigator. (This function in RSCChart is passed down to our Navigator and is where we want to now call a function to set a signal.)
So basically, we now just need to create a new signal and manipulate it here.
The NavigationEvent type sends the following:
eventType: "focus" | "blur" | "selection" | "enter" | "exit" | "help";
// the data navigator ID of the node being focused, blurred, exited from, selected, etc
nodeId: string;
// the vega-compatible ID of the node being focused, blurred, exited from, selected, etc
vegaId: string;
nodeLevel: "dimension" | "division" | "child";This allows us to turn our soon-to-be-made focus signal on/off, activate a "selection" signal (and run functions related to selection, if those exist?), as well as watch for enter, exit, and help events (which I am sure will matter in the future, but right now probably don't matter).
There was a problem hiding this comment.
Awesome! I've cleared my schedule to work on this today.
There was a problem hiding this comment.
Woohoo! Awesome. Stoked to see what you assemble. Let me know if you have questions or anything, otherwise I'll wait to see how I can build off of this tomorrow morning.
Also as a note: "focused" has 1 s (not "focussed").
There was a problem hiding this comment.
Fun fact about me. I exclusively commit code with typos. 😂
|
Ok I think I added all the needed focus rings as well as the signals and calls to control them. It aint pretty but we are in POC mode. |
Heck yeah! Awesome. The focus indication looks great. It looks like I've got a little bug to fix now on where the invisible element is actually being rendered though. I'll tackle that tomorrow (unless you know what caused this, you can do it). I think one of the last steps will be to visually turn off focus indication for the navigator element too, so only yours shows. 
|
|
Okay, so my final major to-do has been to explore the 3-element strategy as a fallback for mobile. Unfortunately, I don't think we will be able to get the prototype working in time. We just need more testing. This is a pretty tricky thing to design for and I'm not satisfied with how the UX is right now. I'll be able to work some more on this this week, but we might have to move this to a stretch goal, since my time on this project wraps up this Friday. If anyone wants to give the project a look in the current status, feel free to do so (@majornista, @marshallpete, @c-lamoureux, etc). I'll have some time set aside Friday as my final day, so I'll be prioritizing any items that we think I can tackle that day (feedback, tweaks, more explanations added for things, etc). |
|
Heads up, I'll have another commit coming in - I've been wrangling the UX (and ARIA) for our mobile fallback and I think I may have settled on something that works. Fingers crossed that gets in today, but if not it might bubble over into Monday. |
|
While doing the last bit of work on this, I noticed that directional nav wasn't correct on the childmost elements of stack. Originally, I thought "oh, the left/right and up/down will change based on chart orientation/etc so we just need to set those props." But then I realized that I actually needed to add a new feature to data navigator itself because the division navigation was correct but it just didn't match the children. I was baffled at first, but then I realized that it was just because of the way data navigator groups dimensions by default: you always navigate within divisions but in a stack you want to nav across them. Anyway, you can navigate the difference between two stacked bars (represented as their graphs) in DN's little demo test page. Note: The diagrams in our v2.2.0 release notes communicate this new feature in simpler terms: Default/previous behavior, or using new "within"
|
|
Okay folks, here is my last official set of commits, I reckon. Thanks for being patient! Notes:
Again (on that last point), I just can't settle on a good UX for this. And I can get a halfway decent experience working in plain html and vanilla js, but this doesn't translate over to our react environment for some reason. I have a feeling that focus handling and other things in react ruins this for us. I'd need time to really dig into this. I'd probably sleep at night if I could somehow replicate my plain environment in ours, but I wouldn't be happy with a mobile user experiencing what we had currently. It was just buggy and strange. I'm sure mobile use is nearly non-existent, but still: I'll be thinking about how to solve this. It's possible that a small "control" panel (of buttons) that had aria live announcements would solve this (like my messy sketch below). 
|
| correspondingNode.semantics = { | ||
| label: describeNode(datum, { | ||
| semanticLabel: NAVIGATION_SEMANTICS[semanticKey].CHILD + '.', | ||
| }), | ||
| }; |
There was a problem hiding this comment.
datum which is currently derived from the dimensions contained in keysToMatch, does not include item value for the dimension. So in the case of a simple bar chart, like the storybook example ?path=/story/rsc-bar--basic, the bar is only labelled by the browser, with no indication of the number of downloads or the percentage of the whole that the number of downloads represents.
This can be fixed with something like:
| correspondingNode.semantics = { | |
| label: describeNode(datum, { | |
| semanticLabel: NAVIGATION_SEMANTICS[semanticKey].CHILD + '.', | |
| }), | |
| }; | |
| Object.keys(correspondingNode.data) | |
| .filter((key) => key !== NAVIGATION_ID_KEY && !Object.hasOwn(datum, key)) | |
| .forEach((key) => { | |
| datum[key] = correspondingNode.data[key]; | |
| }); | |
| correspondingNode.semantics = { | |
| label: describeNode(datum, { | |
| semanticLabel: NAVIGATION_SEMANTICS[semanticKey].CHILD + '.', | |
| }), | |
| }; |
I use correspondingNode.data here, rather than i.datum, to retrieve the additional parameters, downloads and percentLabel, because the i.datum includes additional parameters like the start and end values for the item, downloads0 and downloads1.
There are a few additional problems with this approach that we will need to solve.
- The
keyfor each parameter in thedatumobject passed to data navigator'sdescribeNodeutility should be a localized string, which we currently define as thetitlefor each Axis component, but I'm not quite sure how we should access that here. - There may still be parameters that we wish either omit or transform into other key/value pairs for inclusion within an item label. For example, the stacked bar storybook examples use
"value"as thekeywhere the axistitleis"Downloads", and each node includes the parameter"order"which corresponds to the order in which items within each stacked column are stacked. We should be able to transform the value parameter into a localized string for "Downloads," and omit the "order" parameter if it is not meaningful as part of the accessibility name.
There was a problem hiding this comment.
Great catch! 2 things:
- We can pull axis labels from the spec we generate (which we use to make dimensions/divisions)
- We may also want to consider some developer control over how the descriptions work too (change the order of things, include key names or not, etc)
The first point (and your comment in general), I can look into when I get more time in a couple weeks or so. The second comment is more of a bigger picture question about thinking of ways developers can specify/influence the design (like an API design/dev experience sort of question). Right now RSC doesn't really expose any props to developers for accessibility and this is one area where I know it can be nice to have some design control.
|
Working storybook link: https://opensource.adobe.com/react-spectrum-charts/PR-483/?path=/story/rsc-bar-dodged-bar--color |
Description
This work is part of an active consultation on the accessibility of React Spectrum Charts. Under the hood we want to enable accessible interaction and navigation of every chart in RSC, regardless of whether canvas or svg are used. In addition, we want to provide an API that enables core RSC developers to have opinionated design control over the navigation and interaction experience of each visualization type.
We are targeting smart defaults that make it easy for downstream developers to get accessible, rich, interactive data visualizations out of the box. But we also want to enable overrides for complex or advanced (but valid) use cases.
In general, this is a PR that is intended to demonstrate a working prototype (at minimum) and at best actually contribute working code into RSC across the ecosystem.
Related Issue
#482
Motivation and Context
If a chart can be hovered, clicked: a user should be able to do this with just a keyboard.
If a chart can't easily be summarized in just a couple of sentences: a screen reader user should be able to navigate the data and hear alt text for each element.
If a chart is being used for important decisions or is in an exploratory context: a user should be able to navigate and explore the data in a meaningful, structured way without requiring the use of a mouse or eyesight.
Presently, these experiences are not possible.
How Has This Been Tested?
Screenshots (if appropriate):
Element-level navigation
Alt text at the element-level
Group-level navigation
The goal will be to use Vega's
signalsto actually show the focus indication, while my newNavigatorcomponent will handle actually making accessible navigation possible (even if the visualization is rendered using canvas).Types of changes
Checklist:
Current status
Navigation (tested with screen reader and keyboard-only)
Semantics/labels
Interactivity (clicking/tooltips)
Focus Indication w/
signalsBonus