Enforce per-field schema version availability at parse time - #738
Open
Gudge (MGudgin) wants to merge 4 commits into
Open
Conversation
…ng gates
This PR adds the two shared libraries every history-aware versioning gate needs
-- resolving what a pull request is being compared against, and ordering schema
versions -- together with a guard that the versioning job cannot report success
without running its tests.
Both libraries fail closed. A gate that cannot determine its base, or cannot
order two versions, reports failure rather than skipping, because a check that
goes quiet when confused is a check that passes on exactly the inputs it exists
to catch.
Details
* `lib/git-base.js` resolves the pull-request base commit and reads files as
they were at that commit. It returns `null` only when a file genuinely does
not exist there, and throws when git itself fails, so "I could not look" is
never mistaken for "it was deleted".
* Paths are normalised to git's forward-slash syntax, and `ls-tree` is read
NUL-delimited. Git C-quotes any path containing non-ASCII bytes, quotes,
backslashes or control characters, and a Windows caller naturally produces
backslashes; either would miss the literal comparison and make an existing
file read as absent.
* The same normalisation collapses `.` and `..` segments and doubled slashes,
and an absolute path is rebased onto the repository root -- `path.join(repoRoot,
...)` is what a caller naturally writes, and it matches no `ls-tree` entry. A
path that cannot name a file in the repository at all is refused outright
rather than reported as absent, so `null` keeps meaning exactly "not present
at that commit" and a gate cannot read a malformed path as "newly added,
nothing to compare".
* `lib/version.js` parses strict SemVer. Numeric identifiers reject leading
zeros, and the core components reject anything not exactly representable,
since an unbounded digit run converts to `Infinity` and `Infinity - Infinity`
is `NaN`, for which every ordinary ordering check is false. Build metadata is
accepted, kept for round-tripping, and excluded from precedence as the
specification requires.
* Numeric *prerelease* identifiers are instead ordered exactly, without ever
converting them to `Number`: SemVer leaves them unbounded, and the grammar has
already rejected leading zeros, so the longer digit string is the larger value
and equal lengths compare lexically. Converting would collapse distinct values
above 2^53 onto one float and long ones onto `Infinity`, whose difference is
`NaN` -- falsy, so the comparison would report equality.
* The parsers take strings only. `RegExp.exec` coerces its argument, so a
single-element array, a boxed `String` or any object with a `toString` would
otherwise parse, and `raw` would carry the non-string.
* The comparison helpers reject anything that is not a parsed version. Reading
`.major` off a raw string or a failed parse yields `undefined`, falls through
every branch, and reports the two versions as equal. `compareVersions` applies
the stricter check, since a `parseMajorMinor` result carries no `patch` or
`prerelease` and would otherwise subtract `undefined` and yield `NaN`;
comparing version lines is what `compareMajorMinor` is for. Both guards check
the domain the parsers actually produce -- non-negative components, and a
prerelease matching the grammar `parseVersion` applies -- since
`Number.isSafeInteger` alone admits negatives and a bare `typeof` check admits
a prerelease the parser would have refused, either of which would be ordered
rather than rejected.
* `check-tests-present.js` runs as `pretest` and fails when the tests directory
is missing or holds no test files. It reads entry types rather than names, so a
*directory* named `foo.test.js` does not satisfy it. `node --test tests/*.test.js` exits 0 when
the pattern matches nothing, so moving or renaming the directory would leave
the job green while executing nothing.
* The versioning workflow checks out full history so a base ref can be
resolved.
Tests
* 39 unit and integration tests pass. The git helpers are exercised against
real scratch repositories, since fail-closed base resolution and path
handling cannot be tested any other way.
* Path coverage includes a backslash-separated lookup and a non-ASCII filename,
each of which read as absent before normalisation and NUL-delimited output.
* A shallow clone with no reachable base is asserted to raise, not to skip.
* SemVer coverage includes leading zeros, unrepresentable components, build
metadata parsing and its exclusion from ordering, malformed prerelease and
build identifiers, prerelease precedence, and the comparison type guards.
* Numeric prerelease ordering is pinned across the 2^53 boundary and at 400
digits, in both directions and for equal values, together with the full
precedence chain from the specification's own example, which fixes
numeric-before-alphanumeric and numeric-not-lexical ordering at once.
* `compareVersions` is asserted to throw for a version line, in either argument
position, and for hand-rolled objects missing `patch` or `prerelease`, while
`compareMajorMinor` still accepts both shapes.
* The presence check now has its own tests, run as real processes against
scratch layouts so the exit code -- the only thing CI reads -- is what is
asserted: a missing directory, an empty one, a directory named `foo.test.js`
with nothing else, and that same directory alongside a real test file.
* Path coverage adds the absolute `path.join(repoRoot, ...)` form, doubled
slashes and a `..` round trip, each of which read as absent before; a file
that is genuinely missing still reads as absent, and a path outside the
repository raises instead.
* The parsers are asserted to reject arrays, boxed strings, `toString` objects
and numbers, and the comparison guards to reject negative components and every
prerelease spelling `parseVersion` refuses, while still accepting the shapes
it produces.
* `check-schema-versions.js` and `validate-configs.js` both still pass, so the
parser change does not disturb existing consumers.
* The resolver takes its options as a single `{ argv, env }` object, and a test
pins that: passing them positionally silently falls back to `process.argv`
and `process.env`, which would make these tests read whatever the ambient job
happens to set. The suite is run both with and without the workflow's own
`MXC_VERSIONING_BASE_REF` and `GITHUB_ACTIONS` values.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: cd48fff2-bde9-487a-ab67-012e9bbc0796
Generated-with: claude-opus-5
This PR adds `scripts/versioning/lib/schema-compatibility.js`, which reports the
ways a new JSON Schema can reject an instance the old one accepted. It is the
primitive the dev-schema gate enforces with.
The detector fails closed in both directions: anything it cannot model becomes a
manual-review finding rather than silence, and anything it cannot prove is a
restriction is reported as needing proof rather than asserted as breaking.
Details
* Covers roughly thirty categories of tightening: closed objects losing a
property, new `required` entries, narrowed `type`, removed `enum` values,
tightened numeric and length bounds, added `items` / `contains` /
`propertyNames`, and changed combinators.
* Normalises equivalent spellings so a generator's rendering choice never reads
as a structural change: `const` and single-valued `enum`, `{}` and `true`,
draft-04 boolean `exclusiveMinimum` / `exclusiveMaximum`, and a `oneOf` of
singleton enums against a flat `enum`. `{}` is canonicalised during
normalisation rather than only where the diff walk enters a node, so the two
spellings stay interchangeable in positions reached by a keyword comparison as
well as by a recursive descent.
* Reports restrictions only. `integer` -> `number` is a widening, since integer
instances are a subset of number, and an assertion-free `items: true` or
`propertyNames: true` rejects nothing.
* Compares `contains` by effective `minContains` and `maxContains` rather than as
written. `contains` carries an implicit `minContains: 1`, which is what makes
even `contains: true` a restriction -- it rejects the empty array. Reading the
keywords literally would miss both halves of that: dropping an explicit
`minContains: 0` while keeping `contains` restores the default and starts
rejecting arrays with no match, while adding `contains` beside
`minContains: 0` demands nothing at all. A *changed* `contains` subschema is
routed to manual review whenever the next effective maximum is finite, because
the polarity inverts there: widening the subschema lets more elements count
toward `maxContains`, so `{contains: integer, maxContains: 1}` becoming
`{contains: number, maxContains: 1}` newly rejects `[1, 1.5]`, which a
recursive descent would read as safe.
* Preserves assertion keywords sitting beside a `$ref`. Draft 2019-09 applies
them, so returning only the target would drop a real restriction such as an
added `required` or `additionalProperties: false`. Draft-07 ignores them, so
composing is the conservative reading -- it can only ask for a review that a
draft-07 document did not need, never miss a restriction. Annotations beside a
reference are dropped, since no dialect applies them as assertions.
* Reports an unresolved reference even when both sides carry the same one:
matching text says nothing about matching content when neither target was ever
inspected. A recursion marker is treated separately, because it marks a cycle
the walk already entered, so equal markers there do mean equal structure.
* Descends into unmatched `anyOf` branches only for the exact `[T, null]`
nullable idiom, where the null branches match and leave a single possible
correspondence. That is the shape the generator emits for every optional
field, and descending is what names a property removed from inside `T`.
Nothing weaker is sound: one unmatched branch a side does not prove those
branches correspond, because a branch that did match may already cover the
removed one, and `[string, const "x"]` becoming `[string, number]` is a pure
widening. Every other shape reports that containment requires manual proof --
except one that is provable in the opposite direction: if every previous
branch still matches exactly, added branches only widen, since an instance
that matched a branch before still matches it now.
* Compares `additionalItems` only alongside tuple-form `items`, where the
keyword has effect, deciding each side's effective value from that side's own
`items` form.
* Handles hostile property names. Own-property lookups are used throughout, so a
property legitimately named `constructor` or `toString` is not skipped via the
prototype chain, and normalisation accumulates into a null-prototype object,
so a schema keyword named `__proto__` stays an own property instead of
invoking the inherited setter and vanishing from the comparison.
* Reports deterministically. Findings are sorted, and properties are descended
in name order: a normalised `$ref` target is identity-shared and a shared
subschema is reported at the first path that reaches it, so insertion order
would otherwise decide whether a finding reads `$.a` or `$.b`.
* Bounds traversal. Normalisation memoises `$ref` targets; the diff walk and
structural equality memoise node-identity pairs; combinator branches are
bucketed by a fixed-size digest, also memoised on identity. Without these a
`$ref` graph that fans out expands exponentially, and equality that serialises
its operands materialises the tree a shared graph unfolds to. Depth and node
budgets catch what remains -- including deeply nested enum data and the
untraversed payload of an unrecognised keyword, which only the equality walk
ever descends -- and surface it as a finding rather than a crash. A memo entry
seeded to break a cycle is deleted again while unwinding an aborted walk,
since the caches are keyed on node identity and an unrecognised keyword's
payload is the caller's own object: a value left behind would let a later run
clear the very pair that just exhausted the budget. Both memos are scoped to a
single call for the same reason: normalised nodes are rebuilt per call, and the
values they share with the caller -- the untraversed payloads of unrecognised
keywords -- are the caller's own mutable objects.
Tests
* 51 unit tests pass, covering each detection category, the equivalent-spelling
normalisations, `$ref` siblings, unresolved and recursive references,
prototype-named properties and keywords, tuple-only `additionalItems`,
effective `contains` bounds, deterministic ordering, and the traversal
budgets. `npm test` in `scripts/versioning` runs 79 tests across the directory
and is the exact command the Versioning Checks job runs.
* Widenings are pinned as producing no finding -- `integer` -> `number`,
`items: true`, `propertyNames: true`, `{}` in nested positions, `contains`
added beside `minContains: 0`, `contains` removed, and annotations beside a
reference -- alongside their counterparts, which are pinned as still reported:
`number` -> `integer`, `contains: true`, `minContains: 0` dropped, a narrowed
`contains` subschema, and assertion keywords beside a reference.
* Fail-closed behaviour is pinned across repeated invocations: the same objects
compared three times in a row yield the budget finding every time, rather than
clearing on the second call from a stale memo entry, and mutating a payload
between calls is detected rather than answered from the previous call's memo.
* Fan-out at depth 40 completes in 2 ms and 5,000-deep nesting returns a budget
finding instead of overflowing the stack; a 5,000-level enum value and a
20,000-deep unrecognised-keyword payload do the same.
* Combinator matching measured 20 ms at 2,000 branches, 51 ms at 4,000 and 70 ms
at 8,000.
* Regressions pin the two behaviours in tension: a removed property inside a
nullable wrapper is still named, and an equal-count branch replacement yields
one manual-proof finding rather than invented positional restrictions.
* Checked against the committed schemas: every schema compares clean against
itself, 0.6.0-alpha to 0.7.0-alpha yields 5 findings, and 0.7.0-alpha to the
dev schema yields 12.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: a51edbf6-88f0-47f1-83e3-931497800904
This PR adds a CI gate that compares the dev schema at the pull-request base against the dev schema at HEAD and fails when the new one rejects an instance the old one accepted. Every other breaking-change guard compares RELEASED stable schemas, and only at release time. The surface a pull request actually edits -- the dev schema -- is unguarded, so a change can delete a stable field, regenerate the schema and the SDK types, migrate the config corpus, and merge green. PR #676 did exactly that, and was reverted by hand. Details * `scripts/versioning/check-dev-schema-compat.js` resolves the base commit with the fail-closed helper, reads both dev schemas out of git, and reports every structural restriction the compatibility detector finds. * Each side is read at its own declared `devSchemaFile`. Opening a new dev line copies the outgoing one, so the documents stay the same lineage and the comparison holds across that transition. Skipping the comparison when the line moves would let a change disable the gate by editing one line of `schemas/schema-version.json`. * A missing or unparsable schema on either side fails. The gate is only useful if it cannot succeed vacuously. * There is no per-field escape hatch. The supported-version window is what allows surface to end, so until a change moves that window, a config declaring an already-supported version has to keep parsing. * Documented in `.github/copilot-instructions.md` alongside the other schema gates, including how to make a breaking change additively, since this gate is what a contributor meets when they try to remove surface. * Runs ahead of corpus validation, because a change that removes a field also migrates the corpus; validation then passes and the removal is what needs reporting. Tests * 8 end-to-end tests drive the real CLI against throwaway repositories and assert on its exit code: unchanged and additive schemas pass; a removed property, a narrowed type, a missing schema and an unparsable schema all exit 1; a compatible new dev line passes and reports the move; and an incompatible new dev line is still blocked. * Replayed against PR #676: the gate exits 1 and names all six removed `network` fields. * Run against the repository as it stands, the gate passes, as do `check-schema-versions.js` and corpus validation across 195 configs. * Full versioning suite: 71 tests. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: cd48fff2-bde9-487a-ab67-012e9bbc0796 Generated-with: claude-opus-5
This PR adds per-field schema version availability: a wire field can declare the
range of config schema versions it is valid in, and the parser rejects any use
outside that range. It is what makes shape-only support for older schema
versions real — until now such an annotation would have been documentation that
nothing honoured.
Details
* New `mxc_version_derive` proc-macro crate. `#[derive(VersionAvailability)]`
lifts `#[mxc_version(since = "0.8")]` / `until` off `wire.rs` into metadata
normal builds carry, so one declaration feeds both the parser and schema
generation (published as `x-mxc-since` / `x-mxc-until`). A derive rather than
`#[schemars(extend(...))]`, which sits behind the `schema-gen` feature and so
can never be consulted by the parser.
* The mechanism fails **open** if a derived JSON name ever disagrees with what
serde accepts — the range simply never fires — so that case is guarded twice:
the macro compile-errors on every serde construct it cannot model exactly
(`flatten`, split `rename`/`rename_all`, unknown `rename_all` rules,
data-carrying variants), and a conformance test cross-checks all 32 wire types
against the names `schemars` independently derives.
* The gate runs immediately after deserialisation in every entry point, because
`convert_wire_config` moves fields out of the config; state-aware requests are
gated on the original document, not the experimental-masked copy.
* `version` is now **required** — it selects the legal field surface, so an
absent one would silently opt out of every range.
* New `version_incompatible` code across all five surfaces (Rust `MxcErrorCode`,
engine `ErrorCode`, TS, C#, `MXC_STATUS_VERSION_INCOMPATIBLE = 13`) carrying
`details: { field, declaredVersion, since, until }`. The supported-range error
migrates onto it. NOTE: this changes an existing error's observable shape — a
consumer string-matching the old range message is affected.
* Three annotations, each checked against real corpus usage first: `seatbelt`
since 0.7, `processContainer.captureDenials` / `learningMode` since 0.8. The
central subtlety is what is deliberately **not** annotated: schema-first-
appearance is only a lower bound on accepted surface. `experimental` was an
open block before 0.8, and state-aware requests declare 0.6 while carrying
`phase` / `sandboxId` / `correlationVector` — annotating those from schema
data would reject configs that have always worked. (Measured: 66 properties
are unannotated yet absent from the 0.6 schema; 8 are covered transitively by
an annotated ancestor, and the rest legitimately carry no range.)
* New `check-version-availability.js` oracle gate derives each field's true
first appearance from the frozen 0.6/0.7 and dev schemas and fails on
disagreement. It is fail-closed on the surfaces above, which also catches a
range that would leak onto the permissive `experimental` surface via a shared
type.
* Corpus and callers migrated: 61 configs versioned (state-aware to 0.6.0-alpha,
matching what the SDK emits; one-shot to 0.8.0-alpha), ~200 Rust test
literals, the PowerShell lifecycle helpers (stamped centrally), and the SDK
builders, which no longer synthesise a top-level `seatbelt` marker below 0.7.
Tests
* On the final tip: `cargo fmt --all -- --check`,
`cargo check --workspace --all-targets`,
`cargo clippy --workspace --all-targets -- -D warnings`, and the per-package
test suites (`wxc_common` incl. the corpus test, `mxc_schema_gen`,
`mxc_version_derive`, `mxc_engine`, `mxc-sdk`, `mxc_ffi`, `wxc`,
`wxc_e2e_tests::e2e_state_aware`) all clean.
* Feature-gated builds covering every flag this diff can reach, all clean:
`wxc_common` {schema-gen, microvm}, `mxc_ffi` {dotnetsdk}, `mxc_engine`
{isolation_session}, `wxc` {isolation_session, microvm, tier2_bfs, wslc,
hyperlight}.
* `wxc_common` 594 unit tests plus a new corpus test asserting all 195 configs
declare a version and still parse, with the out-of-range fixture pinned as a
negative case by exact code and bounds. Versioning gate tests 71 → 94.
* Node SDK build + 223 tests; C# SDK 35 tests; ErrorCode parity 17 codes;
bindings codegen OK. All 11 CI gates pass.
* Non-regression: the PR #676 replay still yields exactly 6 findings naming
`allowLocalNetwork`, `allowedHosts`, `blockedHosts`, `defaultPolicy`,
`enforcementMode`, `proxy`; detector baselines hold (dev vs dev = 0,
0.6→0.7 = 6, 0.7→dev = 12); the dev-schema gate passes; `SUPPORTED_VERSION`
unchanged at `>=0.6, <=0.8`.
* Converged through a 2-round adversarial review (14 findings; 12 fixed, 1
pushback accepted, 1 pre-existing). Two blockers were genuine test failures an
earlier verification pass had masked with a faulty grep.
* **Not executed on this host** (Windows): the macOS Seatbelt paths, the Windows
Sandbox and IsolationSession PowerShell lifecycle suites, and the host-gated
MicroVM / Hyperlight E2E configs. The macOS code does **cross-compile** —
`cargo check --target aarch64-apple-darwin --all-targets` is clean for
`mxc_engine`, `wxc_common` and `mxc-sdk`, including the new
`cfg(target_os = "macos")` regression tests — but it has not been run.
(`mxc_darwin` cannot be cross-checked at all: pre-existing issue #735.)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 21bb36ae-131a-4ab6-b062-a830ba488428
Generated-with: claude-opus-5
|
Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run. |
| const firstAppearance = (path) => timeline.find((t) => t.paths.has(path)); | ||
| const atLabel = (label) => timeline.find((t) => t.label === label); | ||
|
|
||
| for (const record of declared) { |
Contributor
There was a problem hiding this comment.
checkAvailability only iterates declared (records that already carry an x-mxc-since / x-mxc-until). A brand-new field added after the floor with no annotation produces no record, so it is never checked?
| ); | ||
| } else { | ||
| for (const path of record.paths) { | ||
| if (!at.paths.has(path)) { |
Contributor
There was a problem hiding this comment.
This only confirms the field exists at the until version; it never checks that the field is absent from every later frozen schema. A field still present in a frozen 0.7 schema can declare x-mxc-until:"0.6" and pass here after which the runtime wrongly rejects valid 0.7 configs that use it?
| throw new Error(`schema-version.json: 'min' (${schemaVer.min}) is not a version`); | ||
| } | ||
|
|
||
| const stable = readdirSync(stableDir) |
Contributor
There was a problem hiding this comment.
discoverTimeline reads schemas/stable/* from the HEAD worktree (readdirSync + readJson off disk), so a PR that edits or adds a stable schema shifts the very timeline the since / until claims are validated against. This is asymmetric with #732/#730, which read the comparison baseline from the base commit via git-base?
| "field": "version", | ||
| "declaredVersion": "", | ||
| "since": null, | ||
| "until": null, |
Contributor
There was a problem hiding this comment.
nit: for a missing/empty version, the SDK path emits "since": null, "until": null here, while the parser path emits since: MIN_SUPPORTED / until: MAX_SUPPORTED for the equivalent error in config_parser.rs?
Comment on lines
+19
to
+22
| { path: "experimental", why: "declared no properties before 0.8, so anything under it validated vacuously" }, | ||
| { path: "phase", why: "carried by state-aware requests declaring 0.6 since before the schema described it" }, | ||
| { path: "sandboxId", why: "carried by state-aware requests declaring 0.6 since before the schema described it" }, | ||
| { path: "correlationVector", why: "carried by state-aware requests declaring 0.6 since before the schema described it" }, |
Collaborator
There was a problem hiding this comment.
question (non- blocking): for these, the why contains version numbers. Does this mean that we must always update this file ever release? might need to document this somewhere. Might have to update this later, so we don't have to remember to update these everytime.
Comment on lines
+440
to
+456
| ### The consequence: breaking changes are additive | ||
|
|
||
| Because one dev schema has to validate configs declaring *every* supported | ||
| version, the schema cannot be the thing that retires a field — any surface a | ||
| supported version can use has to stay in it. So a breaking change is made | ||
| **additively**: | ||
|
|
||
| 1. Keep the old field, and mark it `until` the last version it was valid in. | ||
| 2. Add the new shape alongside it, marked `since` the version that introduced it. | ||
| 3. Let the declared version decide which one a given config may use. | ||
|
|
||
| This is also why the dev-schema compatibility gate | ||
| (`check-dev-schema-compat.js`) has no per-field escape hatch: a removal is | ||
| always the wrong shape for a change, not an exception to be waived. Deleting an | ||
| `until`-marked field is legitimate only once the supported floor rises past its | ||
| `until` value. | ||
|
|
Collaborator
There was a problem hiding this comment.
note (non- blocking): I think having this
This is also why the dev-schema compatibility gate (
check-dev-schema-compat.js) has no per-field escape hatch: a removal is always the wrong shape for a change, not an exception to be waive
where we talk about removal being wrong and then saying this right after
Deleting an
until-marked field is legitimate only once the supported floor rises past itsuntilvalue.
is confusing, at least to me. I'd just keep the second part and remove the first since future folks when reading this part will care more about the how does versioning work rather than why do we do versioning this way.
Comment on lines
+489
to
+492
| A derive is required rather than `#[schemars(extend(...))]`: the schemars | ||
| attributes sit behind the `schema-gen` feature, which only `mxc_schema_gen` | ||
| enables, so they are invisible to the parser. An annotation nothing enforces is | ||
| documentation, not a contract. |
Collaborator
There was a problem hiding this comment.
note (non- blocking): probably can remove. Doesn't provide good value like the two paragraphs above. In the future I think we need to separate this doc into 2. One for why versioning is done this way and how to update version. First one would be good for us and new devs coming to the project. and second one would be good for quickly understanding what we need to do at a glance to publish a new version.
Comment on lines
+501
to
+506
| **Put the range on the containing field, not inside a shared struct.** A | ||
| struct reached from two places is *one* node, so a range on its fields applies | ||
| to every path that reaches it. `Seatbelt`, for example, is both the top-level | ||
| `seatbelt` section and `experimental.seatbelt`; the range therefore lives on | ||
| `MxcConfig::seatbelt`. The oracle gate refuses any range that would leak onto | ||
| the `experimental` surface this way. |
Collaborator
There was a problem hiding this comment.
thought (non- blocking): re-read this a few times to wrap my head around it but still not quite sure I understand what happens for fields in experimental.seatbelt since there is only one rust struct MxcConfig::seatbelt.
|
|
||
| | Gate | What it protects | | ||
| |---|---| | ||
| | `check-version-availability.js` | Each `since` matches the field's true first appearance across the frozen 0.6 / 0.7 and dev schemas; each `until` names a version the field really existed in. Fail-closed under `experimental`. | |
Collaborator
There was a problem hiding this comment.
note (non- blocking): for this I think the wording should be updated to be more general e.g maybe instead of "across the frozen 0.6 / 0.7 and dev schemas" we just say "across the stable schemas".
|
|
||
| [Fact] | ||
| public void Spawn_MalformedPolicy_ThrowsMalformedRequest() | ||
| public void Spawn_VersionlessPolicy_ThrowsVersionIncompatible() |
Collaborator
There was a problem hiding this comment.
note (non-blocking): test seems like it's a duplicate of the Run_VersionlessPolicy_ThrowsVersionIncompatible test in sdk/dotnet/Microsoft.Mxc.Sdk.Tests/MxcSandboxTests.cs
Comment on lines
+1203
to
+1209
| // Regression (review F2): the darwin builder synthesises a top-level | ||
| // `seatbelt` block purely as a marker. That block carries a `since: 0.7` | ||
| // range natively, so emitting it for a 0.6 policy made the executor reject a | ||
| // field the caller never supplied — every 0.6 macOS policy failed. | ||
| // | ||
| // Asserted through the exported predicate because buildDarwinProcessConfig | ||
| // only runs on darwin; testing it only there is what let the bug through. |
Collaborator
There was a problem hiding this comment.
note (non-blocking): looks like this comment is still referring to a local AI code review perhaps?
Comment on lines
+130
to
+133
| // Previously this surfaced as the generic `MalformedRequest`, because | ||
| // `build_request` wrapped every loader error. Version problems now carry | ||
| // their own code and structured details so a caller can act on them without | ||
| // parsing the message. |
Collaborator
There was a problem hiding this comment.
note (non-blocking): comments here can be removed since it's talking about what the test did in the past.
Comment on lines
+647
to
+648
| // Preserve the loader's typed error: flattening to `malformed_request` | ||
| // would deny the code and details to every one-shot SDK/FFI/C# caller. |
Collaborator
There was a problem hiding this comment.
note (non-blocking): 50/50 on whether this one is a helpful or not to future folks. It tells us why we're changing it from malformed_request but that's it.
From reading the code I can see that what is being returned by load_request_from_value as an error will get converted to an mxc error and we'll early return from the function. That tells me already we want to preserve the error from load_request_from_value so wouldn't need to have that comment.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR adds per-field schema version availability: a wire field can declare the range of config schema versions it is valid in, and the parser rejects any use outside that range.
The governing requirement is that a breaking config-schema change must be possible without dropping support for an earlier version, where "support" means shape only — an older config keeps parsing and is enforced with today's semantics. #732 blocks the dev schema from accepting less than it did, which forces breaking changes to be additive; this PR supplies the other half. Without parse-time enforcement, an availability annotation is documentation that nothing honours.
Details
mxc_version_deriveproc-macro crate.#[derive(VersionAvailability)]lifts#[mxc_version(since = "0.8")]/untiloffwire.rsinto metadata normal builds carry, so one declaration feeds both the parser and schema generation (published asx-mxc-since/x-mxc-until). It has to be a derive:#[schemars(extend(...))]sits behind theschema-genfeature, which onlymxc_schema_genenables, so it can annotate the schema but can never be consulted by the parser.flatten, splitrename/rename_all, unrecognisedrename_allrules, data-carrying variants, malformed literals), and a conformance test cross-checks all 32 wire types against the property namesschemarsderives independently from the same attributes.convert_wire_configmoves fields out of the config and the document can no longer be checked as a whole. State-aware requests are gated on the original document, not the experimental-masked copy.versionis now required. It selects the legal field surface, so an absent one would silently opt out of every range rather than defaulting to something safe.version_incompatibleerror code across all five coupled surfaces (RustMxcErrorCode, engineErrorCode, TS, C#,MXC_STATUS_VERSION_INCOMPATIBLE = 13) carrying structureddetails: { field, declaredVersion, since, until }. The supported-range error migrates onto it, so one code covers both classes.seatbeltsince 0.7,processContainer.captureDenialsandprocessContainer.learningModesince 0.8.check-version-availability.jsoracle gate derives each field's true first appearance from the frozen 0.6/0.7 and dev schemas and fails when a declaredsincedisagrees.Invoke-StateAware*helper rather than at ~20 call sites), and the SDK config builders.Tests
cargo fmt --all -- --check,cargo check --workspace --all-targets,cargo clippy --workspace --all-targets -- -D warnings, and the per-package test suites all clean on the rebased tip.wxc_common{schema-gen, microvm},mxc_ffi{dotnetsdk},mxc_engine{isolation_session},wxc{isolation_session, microvm, tier2_bfs, wslc, hyperlight}.SUPPORTED_VERSIONunchanged at>=0.6, <=0.8.Notes for reviewers
Annotation is opt-in, and "unannotated" means no claim, not "since 0.6". An unannotated field is unbounded, which today is indistinguishable from
since: 0.6, until: 0.8because the supported-range check already rejects everything outside that window — but it becomes observable when the range moves, and unbounded is what lets fields keep working as a new dev line opens.The most important part of this PR is what is deliberately not annotated. A field's first appearance in the JSON Schema is only a lower bound on how long it has been accepted:
experimentaldeclared no properties before 0.8, so anything under it validated vacuously and has always been accepted.0.6.0-alphawhile carryingphase/sandboxId/correlationVector, which the schema only described from 0.8.Deriving bounds from schema data alone would have approved
since: 0.8onphaseand rejected every state-aware request ever sent. Measured: 66 properties are unannotated yet absent from the 0.6 schema — 8 are covered transitively by an annotated ancestor (the walker checks a field's range before descending, so a rejected parent is never traversed into), and the rest legitimately carry none. The oracle gate is therefore fail-closed on those surfaces: it refuses a declaration it cannot justify rather than checking it against a bound that would be wrong.corpus_parses.rsis the behavioural counter-check the oracle structurally cannot provide.Put the range on the containing field, never inside a shared struct.
Seatbeltis reachable from both the top-levelseatbeltsection andexperimental.seatbelt— it is one node, so a range on its inner fields would leak onto the unconstrained experimental surface. The oracle gate catches this class automatically.One deliberate observable break: migrating the supported-range error onto
version_incompatiblechanges an existing error's shape, so a consumer string-matching the old "older/newer than supported" message is affected. This was flagged and accepted in review.Not executed on this host (Windows): the macOS Seatbelt paths, the Windows Sandbox and IsolationSession PowerShell lifecycle suites, and the host-gated MicroVM / Hyperlight E2E configs. The macOS code does cross-compile —
cargo check --target aarch64-apple-darwin --all-targetsis clean formxc_engine,wxc_commonandmxc-sdk, including the newcfg(target_os = "macos")regression tests — but it has not been run.mxc_darwincannot be cross-checked at all, for the pre-existing reason in #735.Microsoft Reviewers: Open in CodeFlow