Improve external vocabulary referencing and move it to an "Extensions" section

[maximiliani]

Problem

Currently, the external vocabulary mechanism requires that for every vocabulary, a separate attribute and syntax definition must be created. If this attribute definition (e.g., 0.FDO/SchemaOrgTerm) is used, the value describes an URL prefix of the vocabulary terms that describe other attributes in the same FDO (see example in the specifications). The value 0.FDO/SchemaOrgTerm attribute in the FDO is always https://schema.org and does therefore provide not much information, except that this vocabulary may be used. Also, this approach does not easily translate to other vocabularies since for each, new attribute and syntax definitions must be created.

Proposed solution

Introduce an attribute definition 0.FDO/ImportsVocabulary which values refer to FDOs that use the profile 0.FDO/VocabularyDef (using the profile reference mechanism). The 0.FDO/VocabularyDef profile describes vocabularies in general, such that they are also represented as FDOs themselves. It specifies the following attributes:

• name of the vocabulary (mandatory) --> e.g., schema.org
• unique prefix of vocabulary terms in an FDO (mandatory) --> e.g., https://schema.org
• URL to download the vocabulary (mandatory) --> e.g., https://schema.org/version/latest/schemaorg-current-https.jsonld
• link to human-readable landing page (optional) --> e.g., https://schema.org
• sparql query for validation (optional) --> checks whether a term is included in the vocabulary
• description (optional)
• contact (mandatory)
• and possibly others (such as related FDO operations)

In this image, you can see four FDOs: 1. The `0.FDO/VocabularyDef` profile defines which attributes SHOULD/MUST be included in a profile. Note that the attribute definitions are not included for brevity. 2. The `0.FDO/ImportVocabulary` attribute definition uses the [profile reference mechanism](https://fairdo-org.github.io/fdo-architecture-spec/#profile-referencing) to ensure every value refers an FDO that complies with the `0.FDO/VocabularyDef` profile. Multiple of these attributes can be used to import multiple repositories. 3. The `21.abc/SchemaOrg` FDO instantiates `0.FDO/VocabularyDef` and describes the schema.org vocabulary. It also includes a SPARQL query to validate whether a certain term is included in the vocabulary. A landing page and download link are provided. There are multiple prefixes available to be used by attributes in an FDO to indicate it stems from the schema.org vocabulary. 4. `234567/example` is an example of how to use vocabulary referencing. It uses `0.FDO/ImportVocabulary` to "import" `21.abc/SchemaOrg`. This means that all attributes with a key that does resolve to an attribute definition will be matched against all known prefixes (in this example `schema` and `https://schema.org/`). These can now optionally be validated using the information of `21.abc/SchemaOrg` by clients that support this. If it is not validated, it is evaluated as an untyped community attribute.

This approach has three main advantages: 1) It does not require users to create attribute and syntax definitions for every vocabulary, which reduces bloat in the typing infrastructure. 2) We "automatically" get some FDOs for vocabularies. 3) This could probably be applied to ontologies as well.

I also think that we should move this out of the existing "advanced referencing" section and introduce an "extensions" section where we can manage such extensions that do not directly affect the functioning of the type system. These could then be clearly marked as "optional" features that some implementations support and others don't. FDO operations, Vocabulary references, and Ontology references are first candidates for this section.

[jboehm1911]

Dear Max,
thank you very much for this draft! I think this approach makes perfectly sense. A minor issue I spotted in the diagram:

• In the 0.FDO/ImportVocabulary, the value of 0.FDO/Description should not contain schema.org directly. Change it to Value references an FDO that describes an external vocabulary. or something similar.

Please feel free to create a pull request such that we can include the changes.

[jboehm1911]

After inspecting again, I have a question regarding this construction:

The attribute definition 0.FDO/ImportVocabulary is said to follow the 0.FDO/ValidationMechanism ProfileReferencing.

If now a FDO would contain the attribute (0.FDO/ImportVocabulary, 21.abc/schemaorg), a validator would check that 21.abc/schemaorg follows the 0.FDO/VocabularyDef profile. And then the validator would stop. How should the validator know whether or not he should continue validating the single schema.org terms against the SPARQL query? Or would he by default not validate anything?

[maximiliani]

Well spotted. My thinking when writing was, that since this is an extension, we use one of the existing validation mechanisms and then decide based on "special rules". This doesn't scale and violates one of the core principles of the new specification document (No special rules!). Thus, we need to use 0.FDO/ValidationMechansim "ExternalVocabulary Referencing". Thanks for spotting this!