Skip to content

fix(schema-guard): see an extension's responses, and declare what they render (#1536) - #1540

Merged
jhgaylor merged 1 commit into
mainfrom
fix/1536-schema-guard-extensions
Sep 4, 2026
Merged

fix(schema-guard): see an extension's responses, and declare what they render (#1536)#1540
jhgaylor merged 1 commit into
mainfrom
fix/1536-schema-guard-extensions

Conversation

@jhgaylor

@jhgaylor jhgaylor commented Sep 4, 2026

Copy link
Copy Markdown
Collaborator

Closes #1536. The first gate of tracker #1538, and the correctness gate the tracker puts before repository graduation.

FountainWeb.SchemaGuard (#1427) validates every response the suite renders against the schema its operation declares. It had never validated one of an extension's, and nothing failed — because a guard that checks nothing looks exactly like a guard that finds nothing.

Why it saw nothing

template/1 resolves through Phoenix.Router.route_info(FountainWeb.Router, ...), and the /api scope ends with forward "/", ExtensionDispatch. A forward is opaque to route_info/4: the router reports the forward's own route, /api, which matches no documented path, so operation/2 returned {:skip, :undocumented} for every extension response.

This is the same opacity that made openapi_paths/0 necessary (ADR 0043 decision 3). The described-vs-served half was compensated for; the rendered-vs-declared half was not.

template/1 now re-resolves through the mount, reading Fountain.Extensions — the host's own registry, so this crosses no ADR 0043 boundary and Fountain.ExtensionGuardTest stays green. Each extension's test_helper.exs attaches the guard too, because scripts/test-libraries.sh runs those suites from their own directories where apps/fountain's helper never runs. attach/0 is idempotent now: a root mix test calls it three times, and the second keeper used to die on :ets.new against a table the first already owns.

What it found on the first run

Measured, not inferred. Every documented extension operation resolved, and two rendered a status nothing declared:

POST /api/support/reports 401 TenantAPIAuth, undeclared
POST /api/buzz/agents 404 environment_not_found, undeclared

The 401 was {"POST /api/support/reports", 401} on SchemaGuardAllowlist until #1528 deleted it as stale — and it was not stale. SchemaGuardrailTest's staleness check reads the operations apps/fountain serves, and that run installs no extension, so the entry named an operation the run could not see. The defect simply stopped being observable, and the deletion read as a fix landing. The allowlist's own moduledoc calls out the inverse ("a stale allowlist entry is a fix nobody noticed landing"); this was that failure mode running the other way.

The 404 is a real disagreement between AgentController.create/2 and its schema, added with the environment_id support and catchable by nothing until now.

What it does about them

Both extensions declare the statuses rather than going on the allowlist: 401 on all eight operations, plus the 404. That is nine declarations, against the 66 core operations #1432 still owes.

The ratchet therefore stays at 70 and gains no entry it could not evaluate. That rule is written down in SchemaGuardAllowlist's moduledoc and enforced by FountainWeb.ExtensionSchemaGuardCase, from each extension's own suite — the only run that can see an extension's operations at all.

The published document moves, so sdk/contract/contract.json and sdk/typescript/src/generated/openapi.ts are rebuilt here. That is exactly why this could not ride along with #1528, whose gate was a byte-identical spec. All four SDK contract verifiers pass.

The fixture extension is an installed extension in the test VM like any other, and the guard sees it too: :whoami declared no 401 and :deep declared nothing at all. Both declare their responses now. The fixture describes OpenAPI paths only while the suite runs, so the published artifact is unchanged by that half — verified with build.sh --check.

Verified by reverting

A guard that stopped guarding looks exactly like one finding nothing, so both halves were broken on purpose:

  • Removing the ExtensionDispatch clause from template/1 fails the new coverage test, naming all three support operations resolving to GET /api.
  • required: [..., :never_rendered] planted on SupportReportResponse fails 2 tests across 3 operations; the same on BuzzIdentityResponse fails 8. Before this change both suites were 0 failures with the schema broken.

Gate

mix precommit green: 4,206 + 114 + 33 tests, 0 failures; credo clean; dialyzer 0 errors; sobelow clean; prod release assembles. Formatting checked with the pinned 1.19.2.

Left for the tracker

POST /api/mcp/buzz/{conversation_id} stays :undocumentedMcpController declares no operation, the same as core's own /api/mcp/team/... transports. It is a sandbox transport rather than public API, so this is the correct outcome, not a gap.

🤖 Generated with Claude Code

https://claude.ai/code/session_015unQSjdFCfcmt4XJHPHtAY

…y render (#1536)

`FountainWeb.SchemaGuard` validates every response the suite renders against
the schema its operation declares. It had never validated one of an
extension's, since the day the first extension existed, and nothing failed —
because a guard that checks nothing looks exactly like a guard that finds
nothing.

`SchemaGuard.template/1` resolves through
`Phoenix.Router.route_info(FountainWeb.Router, ...)`, and the `/api` scope ends
with `forward "/", ExtensionDispatch`. A forward is opaque to `route_info/4`:
the router reports the forward's own route, `/api`, which matches no
documented path, so `operation/2` returned `{:skip, :undocumented}` for every
extension response. The described-vs-served half of this opacity was
compensated for by `openapi_paths/0` (ADR 0043 decision 3); the
rendered-vs-declared half was not.

`template/1` now re-resolves through the mount, reading `Fountain.Extensions`
— the host's own registry, so this crosses no ADR 0043 boundary. Each
extension's `test_helper.exs` attaches the guard too, because
`scripts/test-libraries.sh` runs those suites from their own directories where
`apps/fountain`'s helper never runs; `attach/0` is idempotent now, since a
root `mix test` calls it three times and the second keeper used to die on
`:ets.new`.

## What it found on the first run

Measured, not inferred — both suites, every documented operation resolved:

* `POST /api/support/reports` renders an undeclared 401. This entry was on
  `SchemaGuardAllowlist` until #1528 deleted it as stale, and it was not
  stale: the staleness check reads the operations `apps/fountain` serves, and
  that run installs no extension. The defect stopped being observable, and the
  deletion read as a fix landing.
* `POST /api/buzz/agents` renders an undeclared 404 (`environment_not_found`),
  a real disagreement nothing could have caught before.

## What it does about them

Both extensions declare the statuses rather than going on the allowlist. The
ratchet stays at 70 entries and gains no extension entry it could not
evaluate: `apps/fountain` installs no extension, so an entry naming one is
reported stale by the run that cannot see it. `SchemaGuardAllowlist`'s
moduledoc now says so, and `FountainWeb.ExtensionSchemaGuardCase` enforces it
from each extension's suite, which is the only run that can.

That is nine declarations across eight operations (401 on all eight, plus the
404), against the 66 core operations #1432 still owes. The published document
moves, so `sdk/contract/contract.json` and the TypeScript generated types are
rebuilt here — which is why this could not ride along with #1528, whose gate
was a byte-identical spec.

The fixture extension is an installed extension in the test VM like any other,
and the guard now sees it too: `:whoami` declared no 401 and `:deep` declared
nothing at all. Both declare their responses now. The fixture describes
OpenAPI paths only while the suite runs, so the published artifact is
unchanged by that half.

## Verified by reverting

* Removing the `ExtensionDispatch` clause from `template/1` fails the new
  coverage test, naming all three support operations resolving to `GET /api`.
* A `required: [..., :never_rendered]` planted on `SupportReportResponse`
  fails 2 tests across 3 operations; the same on `BuzzIdentityResponse` fails
  8. Before this change both were `0 failures`.

`mix precommit` green: 4,206 + 114 + 33 tests, 0 failures, credo clean,
dialyzer 0 errors, prod release assembles.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015unQSjdFCfcmt4XJHPHtAY
@jhgaylor jhgaylor added the sdk-no-release Changes the SDK's published surface deliberately without cutting a release label Sep 4, 2026
@jhgaylor

jhgaylor commented Sep 4, 2026

Copy link
Copy Markdown
Collaborator Author

sdk-no-release, deliberately

The SDK release gate is right: this changes sdk/typescript/src/generated/openapi.ts and leaves the version at 1.20.0. Labelling rather than bumping, for two reasons.

The change is additive types only. Nine new response variants (401 on eight operations, 404 on POST /api/buzz/agents) on operations that already existed. No runtime code, no client behaviour, no removed or narrowed type — regenerating from the spec is the whole diff.

Two bumps are already in flight and must merge in ascending order. #1489 claims 1.21.0 and #1481 claims 1.22.0. sdk-publish.yml runs a bare npm publish, so npm moves the latest dist-tag to whatever it just published regardless of semver; a third number here adds another ordering constraint to that race for a types-only change. Picking SDK versions while other bumps are open is the coordinator's call, not this PR's.

These types publish with whichever of those releases lands next — both regenerate the same file from the same spec, so they carry this change forward automatically.

If you would rather this cut its own release, say so and I will take the next free number and add the four edits (package.json, package-lock.json, USER_AGENT in src/http.ts, CHANGELOG heading).

@jhgaylor
jhgaylor merged commit 7349a82 into main Sep 4, 2026
17 of 18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

sdk-no-release Changes the SDK's published surface deliberately without cutting a release

Projects

None yet

Development

Successfully merging this pull request may close these issues.

The schema guard never sees an extension's responses, and the ratchet under-counts as a result

1 participant