fix(schema): validate extension files against their entity metaschema

[ncdingari]

fix(schema): validate extension files against their entity metaschema

Problem

schema/test/schema_test.go validates every .json file under schema/extensions/ against metaschema/extension.schema.json:

{Dir: filepath.Join(schemaDir, "extensions"), Schema: filepath.Join(metaschemaDir, "extension.schema.json")},

The match is a recursive path-prefix match, so it reaches every file in every subdirectory of an extension. But extension.schema.json requires description, caption, name and uid and sets additionalProperties: false — it describes extension.json and nothing else.

An extension mirrors the layout of the core schema directory, as described in CONTRIBUTING.md. Its dictionary.json, domains/, skills/, modules/, objects/ and profiles/ files are governed by the same metaschemas as their core counterparts. Validating them against extension.schema.json makes every one of them fail, so task test:schema cannot pass for any in-tree extension.

The bug is currently invisible because schema/extensions/ does not exist on main. It surfaces as soon as anyone follows the documented extension workflow.

Reproduced against main by dropping a minimal extension into schema/extensions/example/:

File ../extensions/example/dictionary.json failed validation:
  (root): uid is required
  (root): Additional property attributes is not allowed
File ../extensions/example/domains/example_verticals/example_verticals.json failed validation:
  (root): Additional property extends is not allowed
  (root): Additional property category is not allowed
  (root): Additional property attributes is not allowed
File ../extensions/example/modules/example_telemetry.json failed validation: ...
File ../extensions/example/objects/example_telemetry_data.json failed validation: ...

Change

extensions is removed from the flat directory list and gets its own spec. Discovery mirrors the schema server: a directory containing extension.json is an extension root and its subdirectories belong to it — the same rule as find_extensions/2 in server/lib/schema/json_reader.ex. Each file is then validated against the metaschema for its entity type:

Location within an extension	Metaschema
extension.json	extension.schema.json
dictionary.json	dictionary.schema.json
skills/, domains/, modules/	class.schema.json
objects/	object.schema.json
profiles/	profile.schema.json

Files in locations with no defined metaschema are reported through the existing AddWarning mechanism rather than failing the suite, so an extension carrying additional JSON does not break the build.

Verification

Run from schema/test:

Scenario	Result
Fix + an in-tree extension	ok schema — 6 of 6 specs pass
main's test file + the same extension	1 spec fails, 5 files rejected (output above)
Fix + no schema/extensions/ directory	ok schema — unchanged behaviour for the current tree

The Go diff introduces no gofmt drift beyond what is already on main.

Notes

This PR is the test fix alone. The reference example extension it was originally bundled with is now a separate PR, stacked on this one, since it is only mergeable once this lands.

One known gap, called out for the record: a directory under extensions/ that lacks extension.json is not discovered, so its files are neither validated nor warned about — a contributor who misspells the filename sees a green suite. That matches the schema server's own behaviour (it registers an extension only on finding extension.json), so this PR reproduces it deliberately rather than inventing a second discovery rule. Note that before this change such a directory was not validated either: every file in it hard-failed against extension.schema.json. Happy to add a warning for undiscovered directories under extensions/ if you would like the signal.

The commit is signed off per the DCO.

[ramizpolic]

LGTM, thank you for a valuable contribution @ncdingari!

[akijakya]

The test fix in commit 1 looks correct — discovery matches find_extensions/2, the metaschema map covers exactly the directories json_reader.ex reads, and I confirmed the problem reproduces on main.

The example extension in commit 2 doesn't load cleanly, though. Baseline mix test is 366 passed; loaded the way server/README.md documents (SCHEMA_EXTENSION=extensions), it's 2 failures. CI never sets SCHEMA_EXTENSION, so nothing catches it.

1. dictionary.json has no entry for example_telemetry_data, so the module payload override silently degrades to a string (Missing data type for: example/example_telemetry_data, will use string_t type) and Schema.CompilationTest fails. Core modules work because schema/dictionary.json carries a self-typed entry, e.g. "observability_data": {..., "type": "observability_data"}. Adding the equivalent here fixes it.

oasf/schema/extensions/example/dictionary.json

Lines 4 to 16 in 3f4695a

	"name": "dictionary",
	"attributes": {
	"example_endpoint": {
	"caption": "Example Endpoint",
	"description": "URL of the endpoint that the example telemetry module reports to.",
	"type": "string_t"
	},
	"example_sample_rate": {
	"caption": "Example Sample Rate",
	"description": "Percentage of events reported by the example telemetry module, from 0 to 100.",
	"type": "integer_t"
	}
	}

2. "extends": "example_verticals" doesn't resolve, so the class loses its category and Schema.SchemaIntegrityTest fails with domains with invalid extends: [:"example/cold_chain_logistics"]. find_category_class uses a bare String.to_atom(extends) while extension entities are keyed :"example/example_verticals". Scoping the extends clears both failures (366 passed). The cache.ex gap is pre-existing — Utils.find_parent/3 handles scoping and this doesn't — the example is just the first thing to trip it.

oasf/schema/extensions/example/domains/example_verticals/cold_chain_logistics.json

Lines 3 to 7 in 3f4695a

	"caption": "Cold Chain Logistics",
	"description": "Temperature-controlled storage and transport of perishable goods.",
	"extends": "example_verticals",
	"name": "cold_chain_logistics",
	"attributes": {}

3. The README's worked example isn't achievable as written: as submitted the uid is 9980001, not 9980101. Scoping the extends gives 9980101, but then the name becomes example/example_verticals/example_cold_chain_logistics. The name and id can't both be right in either configuration.

oasf/schema/extensions/example/README.md

Lines 39 to 45 in 3f4695a

	```json
	{ "name": "example_verticals/example_cold_chain_logistics", "id": 9980101 }
	```
	That is, the parent category path followed by `<extension_name>_<class_name>`. The
	identifier is derived as `(extension_uid * 100 + category_uid) * 100 + class_uid`.

Minor: a directory under extensions/ without extension.json gets no signal at all — not validated, not warned — since the loop only visits discovered roots.

Given the example needs two fixes plus the README correction, splitting the commits as you offered seems worthwhile.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[ncdingari]

@akijakya — all three reproduce, all three are fixed, and I've taken the split.

This PR is now the test fix alone. Force-pushed to 7b2badb, the commit you said stands on its own — unchanged, not rebased, not amended. The PR body no longer describes the example. Apologies to @ramizpolic, whose approval was against the old head.

The example extension is on ncdingari:feat/example-extension. I haven't opened it as a PR yet: it only applies on top of this one, and a cross-fork PR can't use this branch as its base, so opening it now against main would put this commit in two open PRs at once. I'll open it the moment this merges — or now, if you'd rather review them side by side and don't mind the duplicated commit.

Reproduction

Fresh clone at e856537, Elixir 1.20.3, Go 1.27.0:

Run	Result
mix test	366 passed
SCHEMA_DIR=../schema SCHEMA_EXTENSION=extensions mix test, extension as submitted	364/366 — your two failures, verbatim
same, after the fixes	366 passed
cd schema/test && go test ./...	ok schema

1. Missing example_telemetry_data dictionary entry

Fixed with the self-typed entry. data now resolves to object_t / example/example_telemetry_data instead of String.

Confirmed the mechanism, not just the symptom: cache.ex:287 resolves the reference through find_attribute/3, which tries :"example/example_telemetry_data" and then :example_telemetry_data (cache.ex:396-409). With neither present the attribute reaches utils.ex:220 carrying no :type and falls back to string_t — which is exactly why the core observability module depends on dictionary.json:716-720.

I've documented the requirement in the extension README too. It's load-bearing, written down nowhere, and fails quietly.

2. "extends": "example_verticals"

Fixed, scoped to example/example_verticals. Both failures clear.

Your read of the cache.ex gap matches what I see: Utils.find_parent/3 (utils.ex:455-475) falls back to the extension-scoped key, find_category_class/2 (cache.ex:542) does a bare String.to_atom, and schema_integrity_test.exs:108 does the same. So the unqualified form is wrong in the example regardless — but the next extension author hits the same wall, and the failure (category: nil, then a uid computed against category 0) points nowhere near the cause.

I've left cache.ex alone. Making find_category_class/2 go through Utils.find_parent/3 is a server fix, not an example, and I'd rather not bundle it into either PR uninvited. Happy to open it as an issue or a separate PR — your call which.

3. README worked example

Fixed, and you're right that neither line survived as written. With the scoped extends, Utils.class_name_with_hierarchy/2 (utils.ex:712-726) slash-replaces only the leaf, so the parent stays example/example_verticals.

Rather than trusting the derivation I read the values off a loaded schema:

domain example/cold_chain_logistics
  uid            9980101
  category       example_verticals  ("Example Verticals")
  name enum      example/example_verticals/example_cold_chain_logistics

module example/example_telemetry
  data           object_t -> example/example_telemetry_data

The README now carries exactly those, with the arithmetic spelled out — (998 * 100 + 1) * 100 + 1 — and a note that an unqualified extends is what produces the 9980001 you saw.

On CI giving no signal

Confirmed — nothing in .github/workflows or Taskfile.yml sets SCHEMA_EXTENSION, so the suite never loads an extension regardless of what's in the tree. The smallest fix is a second mix test invocation with it set. I haven't included it: it's a CI change rather than a schema one, and it belongs to whichever of these two PRs you'd rather have carry it. Say which and I'll add it.

On the undiscovered-directory gap

Also correct, and deliberate. Discovery mirrors find_extensions/2, which registers an extension only on finding extension.json; a second, stricter rule in the test would mean the test and the server disagree about what an extension is. Worth noting the previous behaviour wasn't better — such a directory wasn't validated then either, every file in it hard-failed against extension.schema.json.

If you want the signal, a warning for any directory under extensions/ that no root claims is a few lines and I'll add it here.

[akijakya]

Verified, and this is byte-identical to what I said stands on its own — 7b2badb1 is the same SHA as before the force-push, which is only possible if content, parent, message and timestamp are all unchanged. One file in the diff, and go test -count=1 at head gives ok schema.

I loaded the example branch and measured rather than reading the diff:

Claim	Measured
mix test with the extension	366 passed, 0 failures
domain uid	9980101
category	example_verticals ("Example Verticals")
name enum	example/example_verticals/example_cold_chain_logistics
module data	object_t → example/example_telemetry_data

All four match the new README exactly, and both failures are gone. You fixed the mechanism rather than the symptom, and documenting the two traps that made these fail quietly — the unqualified extends and the self-typed dictionary entry — is more than I asked for. Your read of the find_category_class/2 gap matches mine, and I agree it shouldn't ride along in either PR.

One consequence of the split I should have flagged when I suggested it: this PR's new spec is dormant on main. With no schema/extensions/ directory it takes the "does not exist" warning path and cannot fail, so CI can't demonstrate the fix — the evidence lives on a branch that isn't up for review. Not a defect, and the guard activates the moment anyone adds an extension, which is the point. But it does argue for your side-by-side option.

On your four questions:

• Open the example PR now. The duplicated commit collapses to nothing when this merges, and it closes the gap above.
• find_category_class/2: separate issue please. It will hit every extension author and deserves its own visibility rather than being buried in an example.
• The SCHEMA_EXTENSION CI run belongs in the example PR — here it would have nothing to load.
• Take the undiscovered-directory warning, in this PR. Your reasoning about not diverging from the server's definition of an extension is sound, but a warning doesn't redefine anything; it just reports a directory no root claimed.

Approving. Happy to merge once you've decided on the warning — it's small enough that I'd rather it land here than as a follow-up.

[ncdingari]

All four done.

1. Undiscovered-directory warning — taken, in this PR. 0d89895, pushed here. You're right that a warning doesn't redefine anything, and your framing is the one I should have used: discovery still mirrors find_extensions/2, the warning only reports a directory no root claimed.

7b2badb is untouched — the warning is a second commit rather than an amend, so the commit you approved is still byte-identical.

Two details worth flagging:

• The check runs before the no-extensions-found early return. The case that actually bites is not an empty tree, it's one valid extension alongside a misspelled one: roots is non-empty, so nothing would have warned. Verified both — a lone typo/ dir warns, and typo/ beside a valid example/ warns while example/ still validates clean.
• It only reports directories holding .json files directly. A directory that merely groups extensions has no JSON of its own and stays quiet, so this doesn't become noise for anyone nesting.

2. find_category_class/2 — separate issue: #490. Agreed on the reasoning: it hits every extension author with a category of their own, and the failure mode points nowhere near the cause. The issue has the repro, both call sites, and the observation that schema_integrity_test.exs:108 makes the same bare-atom assumption, so it reports the child as invalid rather than reporting the resolution gap. Happy to send the PR if you want it.

3. SCHEMA_EXTENSION in CI — in the example PR, as test:server:extensions wired into the aggregate test task and the CI job. Made it a separate task rather than a change to test:server so the no-extensions run keeps its own signal; the two catch different failures.

I checked it can fail rather than assuming it — reverting the example's extends to the unqualified form makes the task fail with domains with invalid extends: [:"example/cold_chain_logistics"], back to 366 once restored. A CI step that can't fail would just have reproduced the gap it exists to close, one layer up.

4. Example PR opened now: #489. It's rebased onto 0d89895, so the commits it shares with this PR are the same objects and collapse cleanly when this merges.

On the dormant-spec point — you didn't need to flag it, but I'm glad you did, because it's the sharper version of what this PR is. Worth stating plainly: on main this spec cannot fail. No schema/extensions/, so it takes the "does not exist" path and returns. The guard is real but unexercised, and #489 is what puts it under load — task test:schema there validates six files across five metaschemas. Reviewed side by side, the pair is one change: #488 makes extension validation correct, #489 makes it actually run.

Current state:

task test:schema	7 of 7 specs, SUCCESS
task test:server	366 passed
task test:server:extensions	366 passed

Both commits here signed off.