Data is crucial when making decisions about how features are identified, prioritized, implemented, rolled out and removed, but when we look at the web platform ecosystem we don't have robust access to quantitative and qualitative data for any part of the web platform feature life cycle.
Interoperability is key for new and existing features of the platform, but we don’t have a shared understanding of developer needs between browser vendors, and even when we do, we might not use that data when we assess and prioritize solutions. As a result some developer pain points go unaddressed for a long time, because no vendor can individually address them.
Even if we had a shared understanding of problems between vendors, we have no prioritization for solutions to address them. Consequently, solutions that are moved forwards in the standards process and implementations don’t necessarily target the biggest needs, but follow some other prioritization. The Interop project provides an avenue to align implementation priorities for a given year, but it’s not suited to act on developer needs or potential solutions, it’s focused on delivering interoperable implementations for solutions that have gone through the standardization process already.
Upstream from implementation, the working groups usually don’t have reliable access to user research to inform the decision making process, so decisions take longer and are done with less input from developers who are the target audience.
Downstream from implementation, there is no well supported process to close the feedback loop from web developers who either adopt or don’t adopt those features.
The guiding principle for the WebDX CG research is to make available quantitative and qualitative data to all aspects of the web platform feature life cycle to better address the needs of the web platform ecosystem. Quantitative data can involve surveys, but also metrics like use counters, bug metrics, adoption data, etc. Qualitative data can involve user interviews, summaries of free form data, etc. We will strive to make useful data and information available to aspects like identifying what large numbers of developers find challenging about the platform, as well as how well identified solutions map to user needs, and how solutions perform once implemented.
The purpose of this document is to clarify how we’re going to address the research needs of the platform through 3 objectives:
- Driving broad research on web platform to create shared understanding of developer needs
- Driving in-depth research on specific topics to generate actionable insights
- Providing developer research as a service for SDOs to accelerate and improve decision making
The principle for this objective is to have all major web ecosystem stakeholders involved in designing and evaluating research done to identify developer needs and pain points and to be broad in identifying anything that is forcing people off of the platform as well as what’s keeping people from joining the platform. It’s concerned as much with opportunities as with problems. This research ensures we don’t have any major blind spots when it comes to how people use the platform and what they need from it.
Potential issues and mitigations:
- There is already existing qualitative and quantitative data at browser vendors and publicly, but it might be seen as not valid by other vendors, the most common criticism there is that the target audience might not be representative of web developers at large.
- Provide guidance on best practices for publishing research and data, with a focus on common issues with research on the web platform
- Specifically focus on audience definitions and how to sample audiences.
- Generate a prioritized list of top developer pain points on a regular basis
- Identify developer satisfaction with the web platform (ideally benchmarked against alternative platforms)
- Identify new opportunities for the platform where people don’t see their needs addressed well. These won’t show up as top pain points of web developers yet, because they represent an unaddressed audience
- Identify needs that are based on location, language or some other characteristic
The principle for this objective is to have the target audience – the people who will act on the research – to be involved in the research design and the expectation that the results will be acted upon.
- Do a follow up on top needs and identify solutions with stakeholders
- Follow up on recently launched features and their adoption by developers
- Publish a report on developer needs and existing solutions in flight as well as which ones are currently not being addressed.
- Propose a backlog once a year as input for Interop and to identify solutions that should be accelerated.
The principle for this objective is to make available research that is available when WGs need it, with expert UR guidance and short turnaround times.
Provide standards groups with low threshold research opportunities to accelerate and improve the decision making process
- Make surveys available to SDO on regular basis
- Provide opportunities for SDOs to request data on features
Potential issues and mitigations:
- When there's strong disagreement about some detail, requesting research could lead to significant delays in making a decision, and also much energy would be spent on the framing of the question
- An important safeguard is that there's some cost or quota, so that research isn't done when the value of it would be low.