- What is the Underlay?
- Why is this important?
- How does the Underlay work?
- Where does the Underlay get its data?
- How can we deal with misinformation and bad actors?
- How can I access the Underlay?
- Can I contribute to The Underlay’s development?
The Underlay is a distributed knowledge graph. It is an emergent structure that arises from a network, similar to the World Wide Web. The Underlay's purpose is to make knowledge accessible, connectable, and inspectable in service of the public good. Like the web, it is an emergent system and concept. It is built on a set of open source protocols and tools for structuring, storing, and aggregating distributed graph data.
Knowledge is often exchanged in formats optimized for computers, and then used to render webpages, maps, diagrams, tables and text for human consumption. It is also used directly by machines to navigate vehicles, trade stocks, control appliances, design structures, formulate scientific hypotheses, order search results, and much more. Because this machine-readable data is not in a specific human language, it easier to share it globally. But today, most of these machine-readable data sources are privately held and controlled, and the ones that do exist publicly are fragmented and don’t work well with each other. The goal of the Underlay is to improve the way that machine-readable knowledge is shared and used for the benefit of the entire world.
The Underlay represents knowledge in the form of relationships between entities, for example the relationship between an author and a book, or a location and its current temperature. This method of representing knowledge is call a knowledge graph, and it can be used to represent any type of machine-readable information. The Underlay is premised on the idea that a knowledge graph can be constructed from a series of distributed transactions called assertions. Multiple assertions are combined through a process called reduction and can be curated into useful groupings using collections.
The Underlay differs from most other knowledge graphs in that it represents not only the information, but the provenance of that information: where came from and what others have to say about it.
The Underlay is not a single piece of software, but rather a suite of interfaces that allows anyone to publish data for others to use, and to connect data to other databases, which can then be queried and used in applications. Users of the Underlay can pull specific information from the Underlay into databases that contain only the information that they judge to be trusted and relevant for their specific needs. Thus, the Underlay does not replace other databases, it makes it easier to build databases and keep them up to date.
The Underlay draws its content from public information sources, including public records, scientific publications, commercial catalogs, the web, and open databases created for the public good. The Underlay also provides a convenient way for for-profit, not-for-profit, and government organizations to publish machine-readable information that they wish to make easily available, including schedules, catalogs, scientific datasets, public disclosures, standards, and more.
The Underlay works by allowing anyone to make assertions about the world, including false ones. Critically, the Underlay provides a method for identifying the source, or provenance, of these assertions. Unlike the explicit existence of a data file, knowledge is nuanced, dynamic, and open to qualitative judgment. As such, the Underlay is designed as much around keeping track of who asserted a particular bit of knowledge as it is the knowledge itself.
Knowledge in the Underlay is not assumed to be true because it has been asserted, but rather, is designed to be curated by an external group of humans or algorithms that organize assertions into a trustworthy set (for any relevant definition of trustworthy). When querying data or building applications, users of the Underlay are able to see exactly who asserted the information they’re relying on, and how many other people agree with those assertions, allowing them to make informed judgements about the quality and veracity of the data they encounter and use.
We are currently developing the first specifications and implementations for the Underlay. To stay in the loop, subscribe to our very infrequent newsletter.
Yes. We are currently early in the development phase, but could use help from developers who want to provide feedback on our initial specifications and code. To get involved, send an email to [email protected].