diff --git a/CHANGELOG.md b/CHANGELOG.md index 4fb5cc8..0ebfd9b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,62 @@ repository does not provide a supported release-date claim. ## Unreleased +Phase 6F Assurance Bundle interoperability engineering scope: + +- add a local-only preflight plan before download, network, conversion, or GPU work; +- add fixed portable core documents, explicit missing/unknown evidence, and + conservative dimension verdicts reconstructed during verification; +- add concise CLI summaries while retaining canonical machine-readable plans, + manifests, and verification reports; and +- add Phase 5/6A–6E schema-registry interoperability and offline fail-closed + verification for exact file sets, bytes, schemas, and semantic projections; +- add deterministic bounded `.omiv` archives and streaming binary evidence handling; +- add detached Ed25519 signatures with external trust-policy thresholds; and +- add capability negotiation, tracked conformance fixtures, and product-level + `omiv verify bundle.omiv`. + +The Phase 6F engineering scope is complete but not released, and Phase 7 scope +remains unfrozen. + +Phase 7A candidate incubation: + +- add bounded, read-only local discovery of canonical Phase 5/6A–6E evidence; +- conservatively auto-select only one unique final Phase 6A–6E verdict record per + requested dimension, leaving distinct candidates ambiguous; +- emit a deterministic Smart Preflight plan and a standard Phase 6F Assurance + request without changing Phase 6F semantics; and +- prohibit downloads, conversion, remote collectors, runtimes, and GPU work in the + candidate planner. + +This vertical slice is not a frozen Phase 7 scope or release claim. + +Phase 7B candidate incubation: + +- add a provider-neutral local runtime compatibility plan/run/verify workflow with + an initial synthetic llama.cpp-compatible runner profile; +- bind results to explicit executable/artifact digests, runtime version, invocation, + limits, CPU-only environment, and a supplied test vector; +- retain bounded stdout/stderr, five fail-closed stage results, findings, unknowns, + limitations, and canonical JSON evidence; and +- add adversarial orchestration tests and a tracked fully offline example that does + not require a real runtime, model, conversion, or GPU. + +This candidate does not release or freeze Phase 7 or change Phase 6F/7A semantics. + +Phase 7B.1 candidate hardening: + +- replace executable-authored OMIV stage reports with direct execution of the bounded + native argument array and retain raw native stdout/stderr and process observations; +- derive exact OUTPUT status in OMIV while keeping LOAD, TOKENIZER, PREFILL, and + DECODE `UNKNOWN` when the native surface cannot independently establish them; +- embed the complete canonical plan in evidence and verify every duplicated request, + pin, profile, invocation, environment, limit, and test-vector field offline; and +- reject outer-rehashed incoherent plan/evidence mutations while stating explicitly + that canonical integrity is not origin authenticity. + +This hardening remains candidate work. It does not release or freeze Phase 7, +register Phase 7B evidence in Phase 6F, or change Phase 7A selection. + Release infrastructure correction: - pin the approved public release-signing key and verify its digest, typed @@ -15,8 +71,7 @@ Release infrastructure correction: - preserve the immutable `v0.10.0` tag and GitHub pre-release after the initial OIDC workflow stopped at signed-tag verification because no public key was bootstrapped. -This correction does not publish to PyPI, alter release assets, implement Phase 6F/7, -or change runtime package code. +This correction does not publish to PyPI, alter release assets, or implement Phase 7. ## v0.10.0 — Public Preview diff --git a/README.md b/README.md index ffb899e..776d08e 100644 --- a/README.md +++ b/README.md @@ -135,7 +135,20 @@ Released boundaries remain narrow: | Phase 6D | Tokenizer/configuration parity and supplied probes | Finite scope only | | Phase 6E | Runtime resolution, deployment binding, supplied results, output provenance | No implicit live inference | -Phase 6F Assurance Bundle interoperability is planned, not implemented. +Phase 6F remains Assurance Bundle interoperability. Its engineering scope now +provides local preflight, conservative concise verdicts, fixed portable core +documents, deterministic `.omiv` transport, Phase 5/6A–6E schema interoperability, +detached signatures with external trust policy, and offline fail-closed verification. +No Phase 6F release is claimed. Phase 7 scope remains unfrozen. +An isolated [Phase 7A candidate](docs/phase-7a-smart-preflight.md) can now discover +bounded local canonical evidence and generate a Phase 6F request; it does not freeze +Phase 7 scope or bypass the Assurance Bundle preflight. +An isolated [Phase 7B candidate](docs/phase-7b-runtime-compatibility-profiles.md) adds +an explicitly supplied, bounded local runtime profile and five-stage evidence path. +The Phase 7B.1 hardening invokes the native CLI directly, embeds its canonical plan, +and keeps runtime internals `UNKNOWN` when raw output cannot establish them. Its +tracked workflow is synthetic, CPU-only, and offline; it does not freeze Phase 7 +scope or claim general runtime compatibility. See the [roadmap](docs/roadmap.md). ### Reading results @@ -225,6 +238,8 @@ Start with the [documentation index](docs/README.md). - [R1F publication audit](docs/r1f-final-publication-audit.md) - [Release notes](docs/v0.10.0-release-notes.md) - [Roadmap](docs/roadmap.md) +- [Phase 6F Assurance Bundles](docs/phase-6f-assurance-bundle-interoperability.md) +- [Phase 7B Runtime Compatibility Profiles](docs/phase-7b-runtime-compatibility-profiles.md) Detailed command and phase guidance lives in the documentation. @@ -233,8 +248,9 @@ Detailed command and phase guidance lives in the documentation. The repository is public. OMIV remains a pre-1.0 Alpha public preview, not a production, certification, safety, or authenticity claim. -Phases 5 and 6A–6E are released in history. Phase 6F is planned, -not implemented. The signed annotated `v0.10.0` tag and GitHub pre-release exist; +Phases 5 and 6A–6E are released in history. The Phase 6F engineering scope is +complete on current main but is not released. +Phase 7 scope is not frozen. The signed annotated `v0.10.0` tag and GitHub pre-release exist; the PyPI project/version remain absent after a pre-publish failure. Public availability is not release completion. See the [roadmap](docs/roadmap.md) diff --git a/docs/README.md b/docs/README.md index 9f355c8..92f9d5a 100644 --- a/docs/README.md +++ b/docs/README.md @@ -40,6 +40,16 @@ historical command and evidence guide. - [Phase 6D: tokenizer/configuration parity](phase-6d-tokenizer-configuration-parity.md) - [Phase 6E: runtime resolution parity](phase-6e-runtime-resolution-parity.md) +## Phase 6F + +- [Phase 6F: Assurance Bundle interoperability](phase-6f-assurance-bundle-interoperability.md) + +## Phase 7 candidates + +- [Phase 7A candidate: Smart Preflight / Auto Planner](phase-7a-smart-preflight.md) +- [Muse Glimmer metadata-only acceptance example](../examples/reference-preflight/README.md) +- [Phase 7B candidate: Runtime Compatibility Profiles](phase-7b-runtime-compatibility-profiles.md) + ## Practice profiles - [Qwen model-pack and format examples](reference/technical-reference.md#core-format-adapters-and-model-packs) diff --git a/docs/architecture.md b/docs/architecture.md index 44755fa..b891200 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -97,12 +97,15 @@ emits deterministic results and limitations. Verification does not require a hos service. The [quickstart](quickstart.md) demonstrates this path with a small tracked synthetic object. -## Future Assurance Bundle boundary +## Phase 6F Assurance Bundle boundary -Phase 6F Assurance Bundle interoperability is planned and is not implemented. A -future public bundle verifier must compose existing evidence without rewriting its -canonical meaning, must expose missing components, and must remain independently -usable offline. R1C adds no Phase 6F schemas, code, fixtures, or branch changes. +Phase 6F remains Assurance Bundle interoperability. Local preflight precedes costly +work; fixed core documents retain concise verdicts, findings, and unknowns; and +offline verification covers portable directory and deterministic `.omiv` transports, +Phase 5/6A–6E schemas, semantic reconstruction, and optional Ed25519 trust-policy +binding. It composes existing evidence without rewriting canonical meaning or +replacing an earlier verifier. It is not released, and Phase 7 scope remains +unfrozen. See the [Phase 6F guide](phase-6f-assurance-bundle-interoperability.md). ## Public and future commercial operation @@ -116,6 +119,14 @@ public results. See the [public/commercial boundary](public-commercial-boundary. ## Implementation map - `src/omiv/` contains the provider-neutral core, adapters, profiles, and CLI. +- `src/omiv/assurance/` contains Phase 6F preflight, registry/projection, portable + core documents, deterministic archive, signatures, assembly, and offline verification. +- `src/omiv/smart_preflight/` contains the isolated Phase 7A candidate for bounded + local discovery and conservative generation of a normal Phase 6F request. +- `src/omiv/runtime_compatibility/` contains the isolated Phase 7B candidate for + explicitly supplied, bounded local runtime compatibility profiles and evidence. + Its Phase 7B.1 native-output profile embeds the reconstructable canonical plan, + retains raw process observations, and keeps unobservable runtime internals unknown. - `schemas/` and phase-specific directories contain portable evidence and examples. - `docs/phase-*.md` describe released engineering phases and their limits. - `tools/audit_*.py` perform bounded offline repository and preservation audits. diff --git a/docs/phase-6f-assurance-bundle-interoperability.md b/docs/phase-6f-assurance-bundle-interoperability.md new file mode 100644 index 0000000..0cee8eb --- /dev/null +++ b/docs/phase-6f-assurance-bundle-interoperability.md @@ -0,0 +1,124 @@ +# Phase 6F: Assurance Bundle interoperability + +## Status and boundary + +The Phase 6F engineering scope is complete on the current main line. It remains +Assurance Bundle interoperability; this is not a new validation phase and no release +is claimed. Phase 7 scope remains unfrozen. + +Phase 6F composes supplied Phase 5 and Phase 6A–6E canonical evidence without +changing its meaning. It does not replace an originating verifier, upgrade unknowns, +or prove authority, runtime use, numerical fidelity, or semantic fidelity that the +source evidence did not prove. Missing, unsupported, invalid, and untested states +remain explicit and fail closed. + +## Product workflow + +Preflight happens before download, network, conversion, or GPU work: + +```console +omiv assurance plan --request examples/assurance-bundle/request.json --root . --output plan.json +omiv assurance build --plan plan.json --root . --output bundle +omiv assurance pack --bundle bundle --output bundle.omiv +omiv verify bundle.omiv --report-output verification.json +``` + +The concise terminal result is for people. The plan, manifest, core documents, +evidence, signature records, and verification report retain the machine-readable +record. None of these commands downloads a model, invokes a converter, starts a +runtime, or uses a GPU. + +The request declares each evidence member, its phase, expected schema, assurance +dimension, and whether it is supporting evidence or a dimension verdict. Preflight +reports local availability, schema support, byte cost, unresolved gaps, and any +proposed costly work. A proposed operation is information only; it is never executed. + +## Portable bundle v1 + +A directory bundle and its deterministic ZIP_STORED `.omiv` transport contain: + +```text +assurance-bundle.json +subject.json +verdict.json +evidence-index.json +findings.json +unknowns.json +capabilities.json +evidence/ ... +provenance/ ... +runtime/ ... +fidelity/ ... +signatures/*.json +``` + +The manifest records canonical identity, preflight identity, profile, required +features, exact evidence members, fixed core-file hashes, and limitations. Local +source paths never enter the portable manifest. The archive uses sorted portable +paths, fixed timestamps and modes, no compression, and bounded streaming extraction, +so the same directory produces identical archive bytes. + +`capabilities.json` advertises the v1 profile, directory and ZIP_STORED transports, +Ed25519 signatures, supported Phase 5/6A–6E evidence schemas, and resource limits. +Verification rejects an unsupported profile or required feature instead of guessing. + +## Verdict and unknown semantics + +Only evidence explicitly assigned the `DIMENSION_VERDICT` role contributes a +semantic status. OMIV conservatively maps a canonical phase status to `PASS`, `WARN`, +`FAIL`, `UNKNOWN`, or `NOT_TESTED`; schema validity alone never becomes `PASS`. +Supporting evidence remains non-verdict evidence. + +Offline verification recalculates this projection from the packaged canonical +evidence and rejects a different manifest projection. `verdict.json` provides the +short dimension summary, while `unknowns.json` records unresolved claims and the +next action. Full findings and evidence remain available to automation. + +## Integrity, signatures, and trust + +Verification checks manifest identity, profile/features, the exact file set, +portable paths, regular non-symlink files, resource bounds, sizes, SHA-256 digests, +core-document canonical identities, core projections, phase schemas, and semantic +projections. Binary evidence is streamed; JSON parsing is bounded. + +An optional detached Ed25519 record signs the manifest digest: + +```console +omiv assurance sign --bundle bundle --private-key signer.pem --key-id release-key +omiv assurance pack --bundle bundle --output signed.omiv +omiv verify signed.omiv --trust-policy trust-policy.json +``` + +A cryptographically valid signature is `VALID_UNTRUSTED` without an external policy. +Signer authority requires a supplied `omiv.assurance-trust-policy.v1` policy. Invalid +signatures, or a required trusted-signature threshold that is not met, invalidate the +verification. Signing does not make the underlying claim true. + +## Exit behavior and limits + +- `0`: ready preflight or complete, policy-satisfying bundle verification; +- `1`: blocked/review-required preflight, incomplete bundle, invalid bundle, or trust + failure; +- `2`: malformed input, unsafe path, exceeded bound, or operational failure. + +V1 limits requests to 256 evidence members, JSON evidence to 64 MiB per member, +individual streamed members to 1 GiB, and total evidence to 4 GiB. Output directories +and archives must not already exist. Case-insensitive portable-path collisions, +traversal, links, compressed archive entries, duplicate entries, and undeclared files +are rejected. + +Tracked conformance requests live in `fixtures/assurance-bundle/`. One must resolve +to complete and one to incomplete without executing its declared GPU operation. + +## Non-goals and Phase 7 + +Phase 6F does not discover a model from a single artifact, collect remote evidence, +run a backend, compare logits, or decide deployment policy. Those require later +product workflows. The proposed Phase 7 sequence is planning input only: + +1. Smart Preflight / Auto Planner; +2. runtime compatibility profiles; +3. numerical and semantic fidelity profiles; +4. automated collectors and provider/runtime adapters. + +That sequence does not freeze Phase 7 scope. diff --git a/docs/phase-7a-smart-preflight.md b/docs/phase-7a-smart-preflight.md new file mode 100644 index 0000000..93963eb --- /dev/null +++ b/docs/phase-7a-smart-preflight.md @@ -0,0 +1,107 @@ +# Phase 7A candidate: Smart Preflight / Auto Planner + +## Status and boundary + +Phase 7 scope is not frozen. This document describes a minimal candidate vertical +slice under active development; it does not claim a finalized Phase 7 boundary or a +release. + +The slice discovers supplied local canonical evidence, validates it against the +existing Phase 5/6A–6E registry, classifies only final Phase 6A–6E evidence records +as possible dimension verdicts, and emits a normal `omiv.assurance-request.v1` for +the unchanged Phase 6F preflight. It does not alter an Assurance Bundle schema, +projection, verdict, unknown, signature, trust-policy, or offline-verification rule. + +## Local workflow + +```console +omiv smart-preflight plan \ + --intent examples/smart-preflight/intent.json \ + --root . \ + --output smart-plan.json \ + --assurance-request-output assurance-request.json +omiv assurance plan \ + --request assurance-request.json \ + --root . \ + --output assurance-plan.json +``` + +The intent declares a subject, one or more bounded local search paths, and optional +required assurance dimensions. The Smart Preflight plan records every validated +candidate, dimension coverage, selection decisions, local inspection costs, +findings, and an embedded Phase 6F request when a safe handoff exists. + +The planner is deterministic for unchanged local inputs. Search order is portable +path byte order. It validates canonical objects before selection and uses content +digests for portable member names. + +## Conservative selection + +Automatic `DIMENSION_VERDICT` selection is restricted to final evidence schemas: + +- `omiv.payload-integrity-evidence.v1`; +- `omiv.remote-local-reconciliation-evidence.v1`; +- `omiv.quantization-fidelity-evidence.v1`; +- `omiv.tokenizer-configuration-parity-evidence.v1`; and +- `omiv.runtime-resolution-parity-evidence.v1`. + +Other registered objects remain visible supporting candidates. One unique final +record covers a requested dimension. Byte-identical copies are deduplicated; two +distinct records for the same dimension are `AMBIGUOUS`, and neither is selected. +Missing and ambiguous dimensions never receive placeholder evidence or an inferred +PASS. The caller must narrow the search or make an explicit evidence choice. + +The generated request contains no planned costly operation. Phase 6F independently +re-opens, hashes, validates, and projects the selected evidence; Phase 7A does not +pre-authorize a bundle result. + +## Safety and limits + +Discovery is read-only, local, bounded, and does not follow symlinks. It performs no +model download, conversion, remote collection, subprocess execution, runtime start, +or GPU operation. The candidate v1 limits are 10,000 directory entries, 4,096 JSON +files, 256 MiB cumulative JSON bytes, 64 MiB per JSON member, and depth 16. Hitting a +global discovery limit blocks the partial auto plan. + +Exit behavior is: + +- `0` for a complete, unique local selection and written Phase 6F handoff; +- `1` for gaps, ambiguity, or a blocked partial plan; and +- `2` for malformed input, unsafe scope, or an operational failure. + +These limits and selection rules are candidate scope and may change before Phase 7 +is frozen. + +## Metadata-only reference acceptance + +The provider-neutral `reference-preflight` candidate extends the concise preflight +experience to reviewed remote-metadata fixtures without changing the local discovery +planner above. Its first acceptance fixture is Muse Glimmer 30B: + +```console +omiv reference-preflight plan \ + --profile fixtures/reference-preflight/muse-glimmer-30b.json \ + --reference meta-models/Muse-Glimmer-30B-GGUF \ + --root . \ + --output muse-reference-evidence.json \ + --assurance-request-output muse-assurance-request.json +``` + +The command replays pinned observations offline. Provider and runtime names, artifact +roles, revisions, declared sizes and provider-exposed identities live in the fixture; +the operational code only validates, canonicalizes and projects the generic schema. +It does not refresh metadata, download a payload, inspect GGUF bytes, invoke a runtime, +or upgrade declarations into compatibility or fidelity claims. + +The generated Phase 6F request uses an `EXTERNAL` opaque supporting member and declares +the future GPU cost. This preserves the evidence bytes in a portable Assurance Bundle +without adding the Phase 7 candidate schema to the Phase 5/6A–6E verdict registry. The +Phase 6F verdict therefore remains `UNKNOWN`, while the reference-preflight evidence +retains the more detailed qualified states such as `NOT_DOWNLOADED`, +`NOT_ESTABLISHED`, `NOT_RUN`, and `NOT_EVALUATED`. + +See the [offline acceptance example](../examples/reference-preflight/README.md) and +[normalized fixture notes](../fixtures/reference-preflight/README.md). +The example's future GPU recommendation is bounded to a deterministic PNG probe, +one 32 GB RTX 5090, USD 5, four hours, explicit disk tiers, and mandatory exact-Pod +termination; it remains unexecuted metadata rather than a runtime result. diff --git a/docs/phase-7b-runtime-compatibility-profiles.md b/docs/phase-7b-runtime-compatibility-profiles.md new file mode 100644 index 0000000..08edf55 --- /dev/null +++ b/docs/phase-7b-runtime-compatibility-profiles.md @@ -0,0 +1,174 @@ +# Phase 7B.1 candidate hardening: Runtime Compatibility Profiles + +## Status and boundary + +Phase 7 scope is not frozen. This document describes a minimal Phase 7B.1 hardening +slice under active development; it is not a Phase 7, Phase 7B, or Phase 7B.1 release, +finalized scope, certification, or general runtime-compatibility claim. + +The first profile is a local llama.cpp-compatible runner workflow. The core command +and evidence names are provider-neutral. A result applies only to one pinned artifact, +one explicitly supplied executable digest and version, one profile and command, one +CPU-only environment, one set of resource limits, and one supplied test vector. + +## Local workflow + +The tracked example is fully synthetic and offline: + +```console +omiv runtime-compat plan \ + --request examples/runtime-compatibility/request.json \ + --root . \ + --output runtime-compat-plan.json +omiv runtime-compat run \ + --plan runtime-compat-plan.json \ + --root . \ + --output runtime-compat-evidence.json +omiv runtime-compat verify --evidence runtime-compat-evidence.json +``` + +For this fixture, `plan` exits `0`; `run` and `verify` intentionally exit `1` after +writing or validating the `NOT_VERIFIED` evidence described below. + +`plan` is read-only. It never starts the executable. It reports whether the exact +executable and artifact are locally available, regular, bounded, executable where +required, and equal to their supplied SHA-256 pins. It also records the expected +version check and runtime work, and states that network, download, compilation, +conversion, and GPU work are absent. + +`run` re-hashes both inputs, executes the explicit executable's bounded `--version` +command, checks the exact version pin, re-hashes again, then executes the native +profile argument array directly. It re-hashes both inputs after execution. A change +at any boundary fails closed. OMIV does not search `PATH`, inspect a runtime +installation, select a binary, find a model, or substitute another executable. + +The successful synthetic native-output observation is intentionally not a complete +compatibility verification: + +```text +llama.cpp +LOAD UNKNOWN +TOKENIZER UNKNOWN +PREFILL UNKNOWN +DECODE UNKNOWN +OUTPUT PASS + +Runtime compatibility: NOT_VERIFIED +``` + +The JSON evidence retains canonical identity, the complete canonical plan, duplicated +plan identity, profile, artifact and executable bindings, observed version, portable +native argument arrays, the exact controlled environment semantics, limits, complete +test vector and digest, bounded base64 stdout/stderr, return codes, timeouts, +work-file observations, OMIV-derived stage results, findings, unknowns, and +limitations. `HOME` and `TMPDIR` use the canonical `{WORK_DIRECTORY}` value together +with `FRESH_EMPTY_TEMPORARY_DIRECTORY`; this records that both were exactly the fresh +per-run directory without leaking an ephemeral host path. + +## Native CLI contract and stage semantics + +Candidate profile `omiv.runtime-compatibility-profile.llama-cpp-native-output.v1` uses +llama.cpp-style `--model`, `--prompt`, `--seed`, `--temp`, `--n-predict`, +`--gpu-layers 0`, `--simple-io`, and `--no-display-prompt` arguments. It passes no +OMIV-only report, profile, stage, test-vector, or digest flags to the runtime. Native +stdout is compared byte-for-byte after strict UTF-8 decoding; native stderr is also +retained and required to be valid UTF-8 for a complete output observation. + +The five stage names have deliberately narrow meanings in this black-box profile: + +| Stage | Candidate meaning | Current derivation | +| --- | --- | --- | +| LOAD | The runtime itself opened and consumed the pinned artifact. | `UNKNOWN`: OMIV can prove the file pin and argument binding, but native stdout and exit status do not independently prove consumption. | +| TOKENIZER | The runtime itself tokenized the supplied prompt. | `UNKNOWN`: the native CLI does not expose an independent tokenizer observation. | +| PREFILL | The runtime itself executed prompt prefill. | `UNKNOWN`: the native CLI does not expose an independent prefill observation. | +| DECODE | The observed bytes resulted from model decoding. | `UNKNOWN`: a black-box executable can emit bytes without proving their origin. | +| OUTPUT | Native stdout exactly equals the supplied expected output. | `PASS` only for an exact match after a zero exit with complete bounded UTF-8 stdout/stderr; exact mismatch is `FAIL`, and incomplete observations are `UNKNOWN`. | + +A blocked invocation leaves every stage `NOT_TESTED`. Timeout, nonzero exit, stream +overflow, or non-UTF-8 native output leaves every executed stage `UNKNOWN`. JSON that +declares OMIV stage names or a top-level `PASS` is treated only as native stdout +bytes; it receives no privilege even when those bytes are the expected output. + +The current profile therefore does not emit `VERIFIED_WITHIN_PROFILE`: four internal +stages cannot be independently established. That status remains reserved for a +future explicitly versioned profile only if complete pinned end-to-end observations +support every stage. Even then it would mean a bounded compatibility probe, not +authenticity, numerical or semantic equivalence, safety, performance, production +approval, or proof that particular weights produced an output. + +## Canonical plan binding and offline verification + +Evidence embeds the complete `omiv.runtime-compatibility-plan.v1`. Offline parsing +recomputes its `plan_id` and `plan_digest`, reconstructs the profile-defined native +invocation, and verifies the plan's request-to-file bindings and availability state. +It then checks every duplicated or derived evidence field against the embedded plan: +request identity, plan identity, profile, artifact and executable paths and pins, +version pin, invocation, environment, limits, test vector, test-vector digest, and +captured process arguments and limits. Stages and unknowns are reconstructed from the +raw native capture. Recomputing only the outer evidence digest cannot make an +incoherent mutation valid. + +Canonical hashing establishes integrity and self-consistency. It does not establish +who originated the request, plan, executable, artifact, or evidence. A party able to +replace a complete coherent object and recompute every identity can create a new +self-consistent object; origin authenticity requires a separate trust mechanism. + +## Process, output, and file controls + +OMIV uses an argument array with `shell=False`, closed stdin, a fresh empty working +directory, a new process session, bounded wall time, bounded stdout and stderr, and a +minimal environment. The environment sets only `PATH=/usr/bin:/bin`, UTF-8 C locale, +an isolated `HOME` and `TMPDIR`, and empty CUDA/HIP visibility. The command pins +`--gpu-layers 0`. + +The profile expects no work files. A Linux child file-size limit bounds an individual +created file, and post-execution inspection bounds the entry count, individual size, +and cumulative captured size. Any file, directory, symlink, unsafe object, or exceeded +bound prevents verification. Captured stream bytes never exceed their declared +limits, including on timeout or overflow. + +The operator must trust the explicitly supplied executable. These controls limit and +record execution; they are not an OS security sandbox and do not make an untrusted +binary safe. Candidate execution currently requires POSIX process-group and file-size +controls; unsupported platforms fail during preflight rather than dropping a bound. + +## Exit behavior + +- `0`: `plan` is ready, or a future supported `run`/`verify` result yields + `VERIFIED_WITHIN_PROFILE`; +- `1`: preflight is blocked, or bounded evidence is valid but compatibility is not + verified because of a failed, unknown, or untested stage or another fail-closed + runtime finding; and +- `2`: a request, profile, plan, or evidence schema is unsupported or malformed; a + path is unsafe; a blocked plan is passed to `run`; or an operational error prevents + a coherent candidate record. + +The current native-output profile returns `1` from `run` and `verify` even when OUTPUT +passes, because the four internal stages remain `UNKNOWN`. Timeout, output overflow, +nonzero exit, non-UTF-8 output, version mismatch, unexpected files, and output +mismatch normally produce bounded JSON evidence and exit `1`. Unsupported control +schemas and profiles are rejected before execution with exit `2`. + +## Composition and non-goals + +Phase 7B/7B.1 evidence is deliberately not added to the Phase 6F Phase 5/6A–6E schema +registry and is not auto-selected by Phase 7A. That preserves existing Assurance +Bundle projection, unknown, signature, trust-policy, and offline-verification +semantics. A future explicitly versioned composition may treat this candidate record +as supporting evidence without changing its meaning; no such boundary is claimed now. + +This slice does not: + +- run or validate an unmodified real llama.cpp build by default; +- download, install, compile, convert, quantize, or discover a runtime or model; +- probe CUDA or other accelerators, use a GPU, or select hardware; +- establish cross-version, cross-artifact, cross-device, cross-prompt, numerical, or + semantic equivalence; +- establish throughput, production readiness, safety, authenticity, publisher + authority, or freedom from malicious runtime behavior; or +- freeze Phase 7 scope. + +The tracked executable and artifact are synthetic native-CLI fixtures, not a real +runtime or model. Direct real-runtime adapters, independently observable internal +stages, additional profiles, and stronger fidelity work require later candidate +slices and separate review. diff --git a/docs/public-commercial-boundary.md b/docs/public-commercial-boundary.md index 34d5a7b..67662e4 100644 --- a/docs/public-commercial-boundary.md +++ b/docs/public-commercial-boundary.md @@ -18,7 +18,8 @@ digests, signature verification, trust/evidence semantics, policy-result semanti the offline CLI, local verification, Model Passports, bounded public-metadata collectors, provider-neutral interfaces, public profiles, examples, synthetic fixtures, and a future public GitHub Action. Future Assurance Bundles must remain -publicly and independently verifiable; Phase 6F is not implemented today. +publicly and independently verifiable; Phase 6F implements that portable public +verification boundary on current main without claiming a hosted service. Possible future commercial services may include a hosted control plane, private evidence registry/history, continuous collection and monitoring, private connectors, diff --git a/docs/roadmap.md b/docs/roadmap.md index cc4fe7f..b40f95d 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -14,9 +14,23 @@ release gates after the repository became publicly readable. | Phase 6C | RELEASED | Quantization fidelity evidence | | Phase 6D | RELEASED | Tokenizer/configuration parity | | Phase 6E | RELEASED | Runtime resolution, deployment binding, supplied results, output provenance | -| Phase 6F | PLANNED, NOT IMPLEMENTED | Future Assurance Bundle interoperability; scope requires separate instruction | +| Phase 6F | COMPLETE; NOT RELEASED | Assurance Bundle interoperability: preflight/plan, conservative verdicts, explicit unknowns, portable directory and deterministic `.omiv`, Phase 5/6A–6E schema registry, detached signatures/trust policy, and offline fail-closed verification | | Phase 7 | FUTURE, SCOPE NOT FROZEN | No finalized scope claim | +Phase 7A Smart Preflight / Auto Planner now has a minimal candidate vertical slice +under development: bounded local discovery, conservative ambiguity handling, and a +generated Phase 6F request. It is incubation evidence rather than a frozen Phase 7 +scope or release claim. A metadata-only reference acceptance profile exercises the +same concise, evidence-qualified boundary without changing the Phase 6F verdict +registry. See the [candidate guide](phase-7a-smart-preflight.md). + +Phase 7B Runtime Compatibility Profiles now has a hardened Phase 7B.1 candidate slice +under development: explicit local executable and artifact pins, direct bounded +CPU-only native invocation, raw observations with unobservable internal stages kept +`UNKNOWN`, and evidence embedding its reconstructable canonical plan. It is +incubation evidence rather than a frozen scope, release, or general compatibility +claim. See the [candidate guide](phase-7b-runtime-compatibility-profiles.md). + ## Release Track | Milestone | Status | @@ -35,7 +49,9 @@ the signer's public key, so the PyPI project/version remain absent. See the [publication recovery record](v0.10.0-publication-recovery.md), [R1F audit](r1f-final-publication-audit.md), and [release notes](v0.10.0-release-notes.md). -Phase 6F development remains planned and unimplemented; Release Track work does not -update the existing Phase 6F branch. +The Phase 6F engineering scope is complete on the current main line. This does not +mark it released or alter any Phase 5/6A–6E boundary. Phase 7 scope is still not +frozen; candidate sequencing and the Phase 7A/7B/7B.1 slices are planning input +rather than a scope claim. Return to the [documentation index](README.md) or the [main README](../README.md). diff --git a/examples/assurance-bundle/README.md b/examples/assurance-bundle/README.md new file mode 100644 index 0000000..c2f1f30 --- /dev/null +++ b/examples/assurance-bundle/README.md @@ -0,0 +1,7 @@ +# Offline Assurance Bundle example + +From the repository root, use `request.json` with the plan, build, pack, and verify +commands in the +[Phase 6F guide](../../docs/phase-6f-assurance-bundle-interoperability.md#product-workflow). +The example packages one tracked Phase 6A comparison, performs no network or GPU +work, and is safe to run offline. diff --git a/examples/assurance-bundle/request.json b/examples/assurance-bundle/request.json new file mode 100644 index 0000000..2e20780 --- /dev/null +++ b/examples/assurance-bundle/request.json @@ -0,0 +1,18 @@ +{ + "planned_operations": [], + "request_id": "offline-phase6f-example", + "requirements": [ + { + "expected_schema": "omiv.payload-manifest-comparison.v1", + "dimension": "STRUCTURE", + "media_type": "application/json", + "member_path": "evidence/payload-comparison.json", + "phase": "PHASE_6A", + "required": true, + "source_path": "payload-integrity/comparisons/exact.json", + "verdict_role": "DIMENSION_VERDICT" + } + ], + "schema": "omiv.assurance-request.v1", + "subject": "synthetic offline Assurance Bundle interoperability example" +} diff --git a/examples/reference-preflight/README.md b/examples/reference-preflight/README.md new file mode 100644 index 0000000..aa07fd2 --- /dev/null +++ b/examples/reference-preflight/README.md @@ -0,0 +1,55 @@ +# Muse Glimmer no-payload Reference Preflight + +This acceptance example turns one model reference into a concise result by +replaying a reviewed official-metadata fixture. It performs no live network +access, model download, GGUF read, runtime invocation, conversion, or GPU work. + +```console +omiv reference-preflight plan \ + --profile fixtures/reference-preflight/muse-glimmer-30b.json \ + --reference meta-models/Muse-Glimmer-30B-GGUF \ + --root . \ + --output muse-reference-evidence.json \ + --assurance-request-output muse-assurance-request.json + +omiv reference-preflight verify --evidence muse-reference-evidence.json +omiv assurance plan \ + --request muse-assurance-request.json \ + --root . \ + --output muse-assurance-plan.json +``` + +The Assurance request deliberately stores the candidate JSON as opaque +`EXTERNAL` supporting evidence. Phase 6F therefore preserves its bytes and +future GPU cost without registering it as Phase 5/6A–6E verdict evidence. The +bundle's semantic verdict remains `UNKNOWN`; bundle `COMPLETE` describes a +complete portable container, not verified model, runtime, or fidelity claims. + +The embedded future plan pins the 17 GB main GGUF, perception projector, +DFlash drafter, llama.cpp `b10353` commit, and fixed text and image probes. The +three GGUF artifact declarations total exactly 19,788,220,960 bytes. A direct +llama.cpp test requires at least 50 GB of workspace/disk; a separate Ollama +representation in the same workspace raises the recommendation to at least +80 GB. No byte identity between the Hugging Face and Ollama representations is +assumed without matching digest evidence. + +The runtime image is the deterministic 64x64 raster PNG +`probes/red-square.png` (SHA-256 +`53bf31df09c932233812a2c7b61c89a0ecbbaee90ab63f36058099e5d008e852`). +The SVG beside it is documentation/source only and is not a runtime input. + +The future envelope authorizes one RTX 5090 with 32 GB VRAM, at most USD 5 of +total compute, and at most four hours of work, with no new work in the final +30 minutes and no additional GPU or Pod. It retains artifact hashes/sizes, +runtime and executable identity, canonical invocations, bounded process +captures and outcomes, GPU/backend observation, and probe outputs. After +evidence transfer, the exact authorized Pod must always be terminated and its +termination verified. + +Ollama remains a secondary observation. Its `muse-glimmer:30b` and +`muse-glimmer:latest` tags are mutable and must be re-resolved immediately +before execution; manifest drift aborts execution unless a new plan is +reviewed. The direct llama.cpp test is independently completable without an +Ollama download. The plan's execution status is `NOT_RUN`, all runtime stages +begin `UNKNOWN`, and no performance, fidelity, safety, or production-readiness +claim follows from a bounded probe. diff --git a/examples/reference-preflight/probes/red-square.png b/examples/reference-preflight/probes/red-square.png new file mode 100644 index 0000000..ec386cc Binary files /dev/null and b/examples/reference-preflight/probes/red-square.png differ diff --git a/examples/reference-preflight/probes/red-square.svg b/examples/reference-preflight/probes/red-square.svg new file mode 100644 index 0000000..827e122 --- /dev/null +++ b/examples/reference-preflight/probes/red-square.svg @@ -0,0 +1,4 @@ + + + + diff --git a/examples/runtime-compatibility/README.md b/examples/runtime-compatibility/README.md new file mode 100644 index 0000000..eb83c0e --- /dev/null +++ b/examples/runtime-compatibility/README.md @@ -0,0 +1,28 @@ +# Runtime compatibility candidate example + +This tracked offline example exercises the Phase 7B.1 candidate orchestration and +native-observation evidence path with a tiny synthetic artifact and executable. The +files are not a real GGUF model or llama.cpp build and perform no inference. + +```console +omiv runtime-compat plan \ + --request examples/runtime-compatibility/request.json \ + --root . \ + --output runtime-compat-plan.json +omiv runtime-compat run \ + --plan runtime-compat-plan.json \ + --root . \ + --output runtime-compat-evidence.json +omiv runtime-compat verify --evidence runtime-compat-evidence.json +``` + +The executable accepts only the profile's llama.cpp-style model, prompt, seed, +temperature, token-count, and CPU-only native arguments and emits `Paris` as plain +native stdout. It emits no OMIV report or stage claims. + +OMIV records an exact OUTPUT `PASS` but keeps LOAD, TOKENIZER, PREFILL, and DECODE +`UNKNOWN`, because this black-box fixture cannot independently establish those +internals. Consequently `run` and `verify` write/validate coherent evidence but exit +`1` with `NOT_VERIFIED`; this is the expected conservative result. Operators must +explicitly supply and trust any real executable. No binary is discovered from +`PATH`, and an unmodified real llama.cpp binary is not exercised by this example. diff --git a/examples/runtime-compatibility/request.json b/examples/runtime-compatibility/request.json new file mode 100644 index 0000000..9fa82a5 --- /dev/null +++ b/examples/runtime-compatibility/request.json @@ -0,0 +1,28 @@ +{ + "artifact_path": "examples/runtime-compatibility/synthetic-artifact.gguf.fixture", + "artifact_sha256": "90459f23651809aea3118bc1a863e9c2e054d1829f58aa6e9d7cbf3035923e3c", + "executable_path": "examples/runtime-compatibility/synthetic-llama-runner.py", + "executable_sha256": "cc52537c8b4a3bac2e5e001ecdd0293be596211f49c467d357ce06632e30afde", + "expected_runtime_version": "llama.cpp synthetic-runner 1.0", + "limits": { + "max_artifact_bytes": 1024, + "max_executable_bytes": 65536, + "max_stderr_bytes": 4096, + "max_stdout_bytes": 16384, + "max_work_file_bytes": 1024, + "max_work_files": 0, + "max_work_total_bytes": 1024, + "timeout_seconds": 5, + "version_timeout_seconds": 2 + }, + "profile_id": "omiv.runtime-compatibility-profile.llama-cpp-native-output.v1", + "request_id": "phase7b-synthetic-llama-cpp", + "schema": "omiv.runtime-compatibility-request.v1", + "test_vector": { + "expected_output": "Paris", + "max_generated_tokens": 8, + "prompt": "The capital of France is", + "seed": 0, + "vector_id": "synthetic-capital-v1" + } +} diff --git a/examples/runtime-compatibility/synthetic-artifact.gguf.fixture b/examples/runtime-compatibility/synthetic-artifact.gguf.fixture new file mode 100644 index 0000000..2c35b8a --- /dev/null +++ b/examples/runtime-compatibility/synthetic-artifact.gguf.fixture @@ -0,0 +1 @@ +OMIV SYNTHETIC ARTIFACT; NOT A MODEL OR VALID GGUF FILE. diff --git a/examples/runtime-compatibility/synthetic-llama-runner.py b/examples/runtime-compatibility/synthetic-llama-runner.py new file mode 100755 index 0000000..d5298f1 --- /dev/null +++ b/examples/runtime-compatibility/synthetic-llama-runner.py @@ -0,0 +1,36 @@ +#!/usr/bin/env python3 +"""Synthetic llama.cpp-compatible native CLI fixture; never performs inference.""" + +from __future__ import annotations + +import argparse +import sys +from pathlib import Path + +VERSION = "llama.cpp synthetic-runner 1.0" + + +def main() -> int: + if sys.argv[1:] == ["--version"]: + print(VERSION) + return 0 + parser = argparse.ArgumentParser() + parser.add_argument("--model", required=True) + parser.add_argument("--prompt", required=True) + parser.add_argument("--seed", required=True) + parser.add_argument("--temp", required=True) + parser.add_argument("--n-predict", required=True) + parser.add_argument("--gpu-layers", required=True) + parser.add_argument("--simple-io", action="store_true") + parser.add_argument("--no-display-prompt", action="store_true") + arguments = parser.parse_args() + Path(arguments.model).read_bytes() + if arguments.gpu_layers != "0" or arguments.temp != "0": + print("synthetic profile requires CPU-only deterministic arguments", file=sys.stderr) + return 4 + sys.stdout.write("Paris") + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/examples/smart-preflight/README.md b/examples/smart-preflight/README.md new file mode 100644 index 0000000..88690e2 --- /dev/null +++ b/examples/smart-preflight/README.md @@ -0,0 +1,22 @@ +# Smart Preflight candidate example + +This candidate Phase 7A example discovers one tracked local Phase 6A evidence +record and generates a Phase 6F request without downloading, converting, contacting +a remote collector, or using a GPU: + +```console +omiv smart-preflight plan \ + --intent examples/smart-preflight/intent.json \ + --root . \ + --output smart-plan.json \ + --assurance-request-output assurance-request.json +omiv assurance plan \ + --request assurance-request.json \ + --root . \ + --output assurance-plan.json +``` + +The first command selects evidence only when a requested dimension has one unique, +canonical final evidence record. Distinct competing records remain ambiguous and +require an explicit caller choice. The second command is the unchanged Phase 6F +preflight boundary. diff --git a/examples/smart-preflight/intent.json b/examples/smart-preflight/intent.json new file mode 100644 index 0000000..07d07f9 --- /dev/null +++ b/examples/smart-preflight/intent.json @@ -0,0 +1,11 @@ +{ + "intent_id": "offline-local-evidence", + "required_dimensions": [ + "STRUCTURE" + ], + "schema": "omiv.smart-preflight-intent.v1", + "search_paths": [ + "payload-integrity/evidence/authorized-complete.json" + ], + "subject": "synthetic local Smart Preflight example" +} diff --git a/fixtures/assurance-bundle/README.md b/fixtures/assurance-bundle/README.md new file mode 100644 index 0000000..0fa4b06 --- /dev/null +++ b/fixtures/assurance-bundle/README.md @@ -0,0 +1,12 @@ +# Assurance Bundle v1 conformance fixtures + +These requests exercise the portable Phase 6F contract without network, conversion, +or GPU execution. + +- `valid-request.json` resolves to `READY`, builds a `COMPLETE` bundle, and verifies + offline as `COMPLETE` against the tracked Phase 6A exact-comparison evidence. +- `incomplete-request.json` resolves to `BLOCKED`, retains the proposed GPU action in + the plan, builds an `INCOMPLETE` bundle, and verifies offline as `INCOMPLETE`. + +Implementations must not execute the declared costly operation during preflight, +assembly, packing, or verification. diff --git a/fixtures/assurance-bundle/incomplete-request.json b/fixtures/assurance-bundle/incomplete-request.json new file mode 100644 index 0000000..01f63e0 --- /dev/null +++ b/fixtures/assurance-bundle/incomplete-request.json @@ -0,0 +1,23 @@ +{ + "planned_operations": [ + { + "cost_class": "GPU", + "reason": "A runtime compatibility probe could resolve the missing claim." + } + ], + "request_id": "conformance-incomplete-v1", + "requirements": [ + { + "dimension": "RUNTIME", + "expected_schema": "omiv.runtime-resolution-parity-evidence.v1", + "media_type": "application/json", + "member_path": "runtime/parity.json", + "phase": "PHASE_6E", + "required": true, + "source_path": "fixtures/assurance-bundle/not-present.json", + "verdict_role": "DIMENSION_VERDICT" + } + ], + "schema": "omiv.assurance-request.v1", + "subject": "OMIV Assurance Bundle v1 incomplete conformance fixture" +} diff --git a/fixtures/assurance-bundle/valid-request.json b/fixtures/assurance-bundle/valid-request.json new file mode 100644 index 0000000..26ac6d4 --- /dev/null +++ b/fixtures/assurance-bundle/valid-request.json @@ -0,0 +1,18 @@ +{ + "planned_operations": [], + "request_id": "conformance-valid-v1", + "requirements": [ + { + "dimension": "STRUCTURE", + "expected_schema": "omiv.payload-manifest-comparison.v1", + "media_type": "application/json", + "member_path": "evidence/payload-comparison.json", + "phase": "PHASE_6A", + "required": true, + "source_path": "payload-integrity/comparisons/exact.json", + "verdict_role": "DIMENSION_VERDICT" + } + ], + "schema": "omiv.assurance-request.v1", + "subject": "OMIV Assurance Bundle v1 valid conformance fixture" +} diff --git a/fixtures/reference-preflight/README.md b/fixtures/reference-preflight/README.md new file mode 100644 index 0000000..9e25b1d --- /dev/null +++ b/fixtures/reference-preflight/README.md @@ -0,0 +1,24 @@ +# Reference Preflight fixtures + +`muse-glimmer-30b.json` is a normalized, bounded replay fixture built from +official public metadata observed on 2026-08-15. It contains no model bytes, +signed URLs, cookies, credentials, or transient request headers. + +The fixture preserves exact repository revisions, provider-exposed file +identities and sizes, pinned small-document identities, the llama.cpp support +and release commits, and the Ollama manifest/layer identities needed to audit +the normalization. Provider declarations remain declarations: they do not +establish local payload verification, GGUF structure, source binding, +companion cryptographic binding, runtime compatibility, or fidelity. + +Default tests replay this fixture offline. Updating it requires a separate, +bounded observation of the listed official sources and review of every changed +identity; the CLI never refreshes the fixture automatically. + +The future-plan raster input is the deterministic 64x64 +`../../examples/reference-preflight/probes/red-square.png` fixture with +SHA-256 `53bf31df09c932233812a2c7b61c89a0ecbbaee90ab63f36058099e5d008e852`. +Its three GGUF declarations total 19,788,220,960 bytes. The plan recommends at +least 50 GB for direct llama.cpp work or at least 80 GB when a separately +downloaded Ollama representation is also retained; these workspace values are +not artifact-size totals. diff --git a/fixtures/reference-preflight/muse-glimmer-30b.json b/fixtures/reference-preflight/muse-glimmer-30b.json new file mode 100644 index 0000000..9b6a9c1 --- /dev/null +++ b/fixtures/reference-preflight/muse-glimmer-30b.json @@ -0,0 +1,472 @@ +{ + "artifacts": [ + { + "artifact_id": "main-q4-k-m", + "authority_source_id": "hf-gguf-api", + "declared_size": 16756683904, + "path": "Muse-Glimmer-30B-KQuant-17GB-Q4_K_M.gguf", + "provider_identity": "4cc57c0f51040a226e5a72cc47b7613f7772950e460a665f7083de89f183f60e", + "provider_identity_kind": "HUGGING_FACE_LFS_SHA256", + "repository": "meta-models/Muse-Glimmer-30B-GGUF", + "required": true, + "revision": "43c7eadd41352a299ea8e0a36b3157978dd63596", + "role": "MAIN_MODEL" + }, + { + "artifact_id": "perception-q4-k-m", + "authority_source_id": "hf-gguf-api", + "declared_size": 1400328928, + "path": "mmproj-Muse-Glimmer-30B-Q4_K_M.gguf", + "provider_identity": "f48b452316f9b213758e8659444029b961a24a07f99a1abb2a9f88b06f7c00c6", + "provider_identity_kind": "HUGGING_FACE_LFS_SHA256", + "repository": "meta-models/Muse-Glimmer-30B-GGUF", + "required": false, + "revision": "43c7eadd41352a299ea8e0a36b3157978dd63596", + "role": "PERCEPTION_ENCODER" + }, + { + "artifact_id": "dflash-q4-k-m", + "authority_source_id": "hf-gguf-api", + "declared_size": 1631208128, + "path": "dflash-Muse-Glimmer-30B-Q4_K_M.gguf", + "provider_identity": "b2e808bf656086fe86bd0d0bd990f01d33e377537a07c02d45371517c8b264ef", + "provider_identity_kind": "HUGGING_FACE_LFS_SHA256", + "repository": "meta-models/Muse-Glimmer-30B-GGUF", + "required": false, + "revision": "43c7eadd41352a299ea8e0a36b3157978dd63596", + "role": "DRAFTER" + } + ], + "canonical_reference": "meta-models/Muse-Glimmer-30B-GGUF", + "configuration": { + "architecture": "muse-glimmer", + "architecture_authority_source_id": "hf-gguf-readme", + "bos_token_id": 200000, + "chat_template_identity": "meta-models/Muse-Glimmer-30B@a4e59da52a7bc87ae7251dd5545c0dd437c44b68:chat_template.jinja:git-blob-sha1:7507f3c9f38809152732c045df3977848f3916a6", + "config_identity": "meta-models/Muse-Glimmer-30B@a4e59da52a7bc87ae7251dd5545c0dd437c44b68:config.json:git-blob-sha1:190826dc834c13c86b8dc68e775a888a852d7c34", + "context_length": 131072, + "eos_token_ids": [ + 200001, + 200008 + ], + "input_modalities": [ + "TEXT", + "IMAGE" + ], + "output_modalities": [ + "TEXT" + ], + "pad_token_id": 200018, + "stop_tokens": [ + "<|end_of_text|>", + "<|eot|>" + ], + "tokenizer_identity": "meta-models/Muse-Glimmer-30B@a4e59da52a7bc87ae7251dd5545c0dd437c44b68:tokenizer_config.json:git-blob-sha1:fe7f0bb90fb0d0288e3a505974ae0e4baf1d3e61" + }, + "future_runtime_plan": { + "artifact_payload_bytes": 19788220960, + "artifacts": [ + { + "artifact_id": "main-q4-k-m", + "declared_size": 16756683904, + "path": "Muse-Glimmer-30B-KQuant-17GB-Q4_K_M.gguf", + "provider_identity": "4cc57c0f51040a226e5a72cc47b7613f7772950e460a665f7083de89f183f60e", + "provider_identity_kind": "HUGGING_FACE_LFS_SHA256", + "repository": "meta-models/Muse-Glimmer-30B-GGUF", + "revision": "43c7eadd41352a299ea8e0a36b3157978dd63596", + "role": "MAIN_MODEL" + }, + { + "artifact_id": "perception-q4-k-m", + "declared_size": 1400328928, + "path": "mmproj-Muse-Glimmer-30B-Q4_K_M.gguf", + "provider_identity": "f48b452316f9b213758e8659444029b961a24a07f99a1abb2a9f88b06f7c00c6", + "provider_identity_kind": "HUGGING_FACE_LFS_SHA256", + "repository": "meta-models/Muse-Glimmer-30B-GGUF", + "revision": "43c7eadd41352a299ea8e0a36b3157978dd63596", + "role": "PERCEPTION_ENCODER" + }, + { + "artifact_id": "dflash-q4-k-m", + "declared_size": 1631208128, + "path": "dflash-Muse-Glimmer-30B-Q4_K_M.gguf", + "provider_identity": "b2e808bf656086fe86bd0d0bd990f01d33e377537a07c02d45371517c8b264ef", + "provider_identity_kind": "HUGGING_FACE_LFS_SHA256", + "repository": "meta-models/Muse-Glimmer-30B-GGUF", + "revision": "43c7eadd41352a299ea8e0a36b3157978dd63596", + "role": "DRAFTER" + } + ], + "dflash_observations": [ + "DISABLED", + "ENABLED" + ], + "direct_llama_cpp_independent_of_ollama": true, + "direct_llama_cpp_workspace_bytes": 50000000000, + "evidence_to_retain": [ + "Downloaded artifact byte sizes and independently computed SHA-256 hashes", + "Runtime version, complete --version output and executable identity/digest", + "Canonical invocations and the controlled runtime environment", + "Bounded stdout and stderr, exit codes, timeout state and work-file observations", + "GPU identity and backend observation", + "Text, image and DFlash-disabled/enabled probe outputs", + "Ollama tag-to-manifest and manifest-to-layer resolution only for the separately authorized secondary observation" + ], + "image_fixture": "examples/reference-preflight/probes/red-square.png", + "image_fixture_sha256": "53bf31df09c932233812a2c7b61c89a0ecbbaee90ab63f36058099e5d008e852", + "image_probe": "Describe the number and color of the geometric shapes in this image in one sentence.", + "llama_cpp_plus_ollama_workspace_bytes": 80000000000, + "maximum_gpu_count": 1, + "maximum_pod_count": 1, + "maximum_total_authorized_compute_usd": 5, + "maximum_working_duration_seconds": 14400, + "non_claims": [ + "No source-to-GGUF cryptographic binding is established by a successful run", + "No companion-artifact cryptographic relationship is established by a successful run", + "No numerical or semantic fidelity conclusion follows from bounded output", + "No performance, fidelity, safety or production-readiness claim follows from any bounded probe", + "No result generalizes beyond the exact artifacts, runtime, GPU and probes" + ], + "ollama_mutable_tags": [ + "muse-glimmer:30b", + "muse-glimmer:latest" + ], + "ollama_observation": "Secondary runtime observation only. Re-resolve both mutable tags immediately before any future GPU execution and abort on manifest drift unless a new plan is reviewed. Do not require or download an Ollama representation for the independently completable direct llama.cpp test. Do not infer byte identity with Hugging Face artifacts without matching digest evidence.", + "ollama_role": "SECONDARY_RUNTIME_OBSERVATION", + "pod_termination_requirement": "After evidence transfer, always terminate the exact authorized Pod and verify that exact Pod is terminated.", + "preferred_gpu_class": "NVIDIA RTX 5090 (32 GB VRAM)", + "question": "Do the three pinned official GGUF artifacts load and produce bounded text, image and DFlash-off/on observations on llama.cpp b10353 at f8def7fe168bab245fbf15d3f18b26dbb1ef73c8?", + "runtime_requirements": [ + { + "authority_source_ids": [ + "hf-gguf-readme", + "llama-support", + "llama-release" + ], + "exact_commit": "f8def7fe168bab245fbf15d3f18b26dbb1ef73c8", + "exact_release": "b10353", + "requirement": "llama.cpp >= b10353; acceptance plan pins b10353 exactly", + "runtime": "llama.cpp", + "support_commit": "62bf73d25c53b8161f8a22894d4f90c4aebbd7d0" + }, + { + "authority_source_ids": [ + "ollama-manifest" + ], + "exact_commit": null, + "exact_release": null, + "requirement": "Secondary observation only: re-resolve mutable tags muse-glimmer:30b and muse-glimmer:latest immediately before execution; abort on manifest drift unless a new plan is reviewed", + "runtime": "Ollama", + "support_commit": null + } + ], + "stop_conditions": [ + "Any downloaded byte digest or size differs from its approved artifact pin", + "The runtime version or executable digest differs from the canonical plan", + "Either mutable Ollama tag drifts from the reviewed manifest plan", + "A process times out, exceeds output or file bounds, or creates an unexpected object", + "The main artifact, projector or drafter fails to load", + "Total authorized compute reaches USD 5", + "The four-hour deadline is reached or new work would begin within its final 30 minutes", + "Continuing would require an additional GPU or Pod", + "Continuing would require changing a dependency, artifact or evidence boundary" + ], + "stop_new_work_before_deadline_seconds": 1800, + "text_probe": "State the result of 2 + 2 in one sentence.", + "unknown_stages": [ + "LOAD", + "TOKENIZER", + "PREFILL", + "DECODE", + "OUTPUT" + ] + }, + "limitations": [ + "This result replays pinned provider metadata offline and performs no remote collection.", + "Provider-exposed identities and sizes are declarations, not locally verified payload bytes.", + "Canonical hashing establishes integrity and self-consistency, not origin authenticity.", + "The complete future runtime plan is a recommendation and was not executed.", + "Phase 6F stores this candidate as opaque supporting evidence and does not promote it to a dimension verdict.", + "Phase 7B.1 runtime stage semantics remain unchanged; no profile or runner may self-certify internal stages." + ], + "profile_name": "Muse Glimmer 30B official no-payload acceptance", + "provenance_gaps": [ + "The GGUF payload bytes were not downloaded or independently hashed locally.", + "GGUF structure, metadata and tensors were not read from local payloads.", + "The declared base-model relationship is not a source-to-GGUF cryptographic binding.", + "The main model, perception encoder and DFlash drafter are not cryptographically bound to each other by observed format-native evidence.", + "The Ollama model layer digest is not assumed to identify either pinned Hugging Face main-model file.", + "Runtime compatibility, numerical fidelity, semantic fidelity, performance, safety and production readiness remain unevaluated." + ], + "reference_aliases": [ + "meta-models/Muse-Glimmer-30B-GGUF", + "https://huggingface.co/meta-models/Muse-Glimmer-30B-GGUF", + "hf://meta-models/Muse-Glimmer-30B-GGUF" + ], + "registry_observations": [ + { + "authority_source_id": "ollama-manifest", + "layers": [ + { + "declared_size": 1400328928, + "digest": "sha256:f48b452316f9b213758e8659444029b961a24a07f99a1abb2a9f88b06f7c00c6", + "role": "projector" + }, + { + "declared_size": 16756681056, + "digest": "sha256:71b5c9c9abbc2a9e7c13550b6f9fe1f062650117a24490b15a310cbd7de03f8b", + "role": "model" + }, + { + "declared_size": 42, + "digest": "sha256:56380ca2ab89f1f68c283f4d50863c0bcab52ae3f1b9a88e4ab5617b176f71a3", + "role": "params" + } + ], + "manifest_digest": "de878ce33ad81d060001db1469a02eebe4d86f0ad58cfe52dc062fdcbe4464c1", + "mutable_tag": true, + "runtime": "Ollama", + "tag": "muse-glimmer:30b" + } + ], + "relationships": [ + { + "authority_source_id": "hf-gguf-api", + "object_reference": "meta-models/Muse-Glimmer-30B@a4e59da52a7bc87ae7251dd5545c0dd437c44b68", + "relationship": "QUANTIZED_FROM", + "subject_artifact_id": "main-q4-k-m" + }, + { + "authority_source_id": "hf-gguf-readme", + "object_reference": "main-q4-k-m", + "relationship": "COMPANION_OF", + "subject_artifact_id": "perception-q4-k-m" + }, + { + "authority_source_id": "hf-gguf-readme", + "object_reference": "main-q4-k-m", + "relationship": "COMPANION_OF", + "subject_artifact_id": "dflash-q4-k-m" + } + ], + "resolved_revision": "43c7eadd41352a299ea8e0a36b3157978dd63596", + "runtime_candidates": [ + { + "authority_source_ids": [ + "hf-gguf-readme", + "llama-support", + "llama-release" + ], + "exact_commit": "f8def7fe168bab245fbf15d3f18b26dbb1ef73c8", + "exact_release": "b10353", + "requirement": "llama.cpp >= b10353; acceptance plan pins b10353 exactly", + "runtime": "llama.cpp", + "support_commit": "62bf73d25c53b8161f8a22894d4f90c4aebbd7d0" + }, + { + "authority_source_ids": [ + "ollama-manifest" + ], + "exact_commit": null, + "exact_release": null, + "requirement": "Secondary observation only: re-resolve mutable tags muse-glimmer:30b and muse-glimmer:latest immediately before execution; abort on manifest drift unless a new plan is reviewed", + "runtime": "Ollama", + "support_commit": null + } + ], + "schema": "omiv.reference-preflight-profile.v1", + "sources": [ + { + "authority": "PROVIDER_API", + "limitations": [ + "Repository metadata is provider-observed; no listed payload body was retrieved." + ], + "observed_on": "2026-08-15", + "provider": "Hugging Face", + "raw_fields": { + "config_model_type": "muse_glimmer", + "gated": false, + "id": "meta-models/Muse-Glimmer-30B", + "pipeline_tag": "image-text-to-text", + "revision": "a4e59da52a7bc87ae7251dd5545c0dd437c44b68" + }, + "resolved_identity": "a4e59da52a7bc87ae7251dd5545c0dd437c44b68", + "source_id": "hf-base-api", + "url": "https://huggingface.co/api/models/meta-models/Muse-Glimmer-30B?blobs=true" + }, + { + "authority": "PINNED_PROVIDER_DOCUMENT", + "limitations": [], + "observed_on": "2026-08-15", + "provider": "Hugging Face", + "raw_fields": { + "architecture": "MuseGlimmerForConditionalGeneration", + "bos_token_id": 200000, + "context_length": 131072, + "model_type": "muse_glimmer", + "vision_model_type": "muse_glimmer_vision" + }, + "resolved_identity": "git-blob-sha1:190826dc834c13c86b8dc68e775a888a852d7c34", + "source_id": "hf-base-config", + "url": "https://huggingface.co/meta-models/Muse-Glimmer-30B/resolve/a4e59da52a7bc87ae7251dd5545c0dd437c44b68/config.json" + }, + { + "authority": "PINNED_PROVIDER_DOCUMENT", + "limitations": [], + "observed_on": "2026-08-15", + "provider": "Hugging Face", + "raw_fields": { + "bos_token_id": 200000, + "eos_token_ids": [ + 200001, + 200008 + ], + "max_length": 131072, + "pad_token_id": 200018 + }, + "resolved_identity": "git-blob-sha1:b69a50a4239f42707d54a479e0143b8ee56c8bcc", + "source_id": "hf-base-generation", + "url": "https://huggingface.co/meta-models/Muse-Glimmer-30B/resolve/a4e59da52a7bc87ae7251dd5545c0dd437c44b68/generation_config.json" + }, + { + "authority": "PINNED_PROVIDER_DOCUMENT", + "limitations": [ + "Only bounded tokenizer declarations were read; tokenizer.json was not retrieved." + ], + "observed_on": "2026-08-15", + "provider": "Hugging Face", + "raw_fields": { + "bos_token": "<|begin_of_text|>", + "eos_token": "<|end_of_text|>", + "model_max_length": 131072, + "pad_token": "<|finetune_right_pad|>" + }, + "resolved_identity": "git-blob-sha1:fe7f0bb90fb0d0288e3a505974ae0e4baf1d3e61", + "source_id": "hf-base-tokenizer", + "url": "https://huggingface.co/meta-models/Muse-Glimmer-30B/resolve/a4e59da52a7bc87ae7251dd5545c0dd437c44b68/tokenizer_config.json" + }, + { + "authority": "PROVIDER_API", + "limitations": [ + "LFS SHA-256 and sizes are provider-exposed identities, not locally verified bytes." + ], + "observed_on": "2026-08-15", + "provider": "Hugging Face", + "raw_fields": { + "base_model": "meta-models/Muse-Glimmer-30B", + "gated": false, + "id": "meta-models/Muse-Glimmer-30B-GGUF", + "pipeline_tag": "image-text-to-text", + "revision": "43c7eadd41352a299ea8e0a36b3157978dd63596" + }, + "resolved_identity": "43c7eadd41352a299ea8e0a36b3157978dd63596", + "source_id": "hf-gguf-api", + "url": "https://huggingface.co/api/models/meta-models/Muse-Glimmer-30B-GGUF?blobs=true" + }, + { + "authority": "PINNED_PROVIDER_DOCUMENT", + "limitations": [ + "Compatibility and companion relationships are official declarations, not local runtime or cryptographic binding evidence." + ], + "observed_on": "2026-08-15", + "provider": "Meta model repository", + "raw_fields": { + "architecture": "muse-glimmer", + "chat_template_size": 9992, + "context_length": 131072, + "dflash_role": "optional speculative drafter", + "minimum_llama_cpp_build": "b10353", + "perception_role": "required for image input" + }, + "resolved_identity": "git-blob-sha1:7aaea3b18cca15a537b06a8dbea40cc7436756a8", + "source_id": "hf-gguf-readme", + "url": "https://huggingface.co/meta-models/Muse-Glimmer-30B-GGUF/resolve/43c7eadd41352a299ea8e0a36b3157978dd63596/README.md" + }, + { + "authority": "PROVIDER_API", + "limitations": [ + "Collection membership is a provider declaration and is not an artifact binding." + ], + "observed_on": "2026-08-15", + "provider": "Hugging Face", + "raw_fields": { + "description": "Muse Glimmer 30B: multimodal agentic model for local deployment. BF16 weights, GGUF k-quants, ExecuTorch builds, DFlash drafter.", + "items": [ + "meta-models/Muse-Glimmer-30B", + "meta-models/Muse-Glimmer-30B-GGUF", + "meta-models/Muse-Glimmer-30B-ExecuTorch-PTE", + "meta-models/Muse-Glimmer-30B-assistant" + ], + "last_updated": "2026-08-10T11:28:08.533Z" + }, + "resolved_identity": "meta-models/muse-glimmer-6a79b5c8f66a54c334d51f31", + "source_id": "hf-collection", + "url": "https://huggingface.co/api/collections/meta-models/muse-glimmer-6a79b5c8f66a54c334d51f31" + }, + { + "authority": "UPSTREAM_VCS", + "limitations": [ + "A merged support implementation does not establish compatibility with the selected remote payload bytes." + ], + "observed_on": "2026-08-15", + "provider": "llama.cpp", + "raw_fields": { + "merged_on": "2026-08-10", + "pull_request": 26841, + "title": "model: Muse Glimmer Support" + }, + "resolved_identity": "62bf73d25c53b8161f8a22894d4f90c4aebbd7d0", + "source_id": "llama-support", + "url": "https://github.com/ggml-org/llama.cpp/commit/62bf73d25c53b8161f8a22894d4f90c4aebbd7d0" + }, + { + "authority": "UPSTREAM_RELEASE", + "limitations": [ + "Release presence and commit identity do not replace a pinned executable digest and observed version." + ], + "observed_on": "2026-08-15", + "provider": "llama.cpp", + "raw_fields": { + "release": "b10353", + "release_commit": "f8def7fe168bab245fbf15d3f18b26dbb1ef73c8" + }, + "resolved_identity": "f8def7fe168bab245fbf15d3f18b26dbb1ef73c8", + "source_id": "llama-release", + "url": "https://github.com/ggml-org/llama.cpp/releases/tag/b10353" + }, + { + "authority": "REGISTRY_MANIFEST", + "limitations": [ + "The tag is mutable; the exact manifest was observed without downloading any layer blob." + ], + "observed_on": "2026-08-15", + "provider": "Ollama", + "raw_fields": { + "manifest_digest": "de878ce33ad81d060001db1469a02eebe4d86f0ad58cfe52dc062fdcbe4464c1", + "model_digest": "sha256:71b5c9c9abbc2a9e7c13550b6f9fe1f062650117a24490b15a310cbd7de03f8b", + "projector_digest": "sha256:f48b452316f9b213758e8659444029b961a24a07f99a1abb2a9f88b06f7c00c6", + "tag": "muse-glimmer:30b" + }, + "resolved_identity": "sha256:de878ce33ad81d060001db1469a02eebe4d86f0ad58cfe52dc062fdcbe4464c1", + "source_id": "ollama-manifest", + "url": "https://ollama.com/v2/library/muse-glimmer/manifests/30b" + }, + { + "authority": "LOCAL_DOCUMENT_FETCH", + "limitations": [ + "The 205,671-byte PDF was fetched to temporary storage only; no PDF text extractor was installed, so no unobserved technical field was normalized from it." + ], + "observed_on": "2026-08-15", + "provider": "Meta Research", + "raw_fields": { + "content_length": 205671, + "content_type": "application/pdf", + "local_document_sha256": "fb08920ab31c5df3e07e2c3b32856d80d65a48dd14864f42bce8a1ac38026c91", + "pages": 7 + }, + "resolved_identity": "local-sha256:fb08920ab31c5df3e07e2c3b32856d80d65a48dd14864f42bce8a1ac38026c91", + "source_id": "meta-methodology", + "url": "https://research.meta.ai/static/muse-glimmer-methodology" + } + ] +} diff --git a/pyproject.toml b/pyproject.toml index d8d5e93..4d2f4d3 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -58,6 +58,12 @@ include = [ "/src", "/tools/audit_public_release_readiness.py", "/docs", + "/examples/assurance-bundle", + "/examples/runtime-compatibility", + "/examples/smart-preflight", + "/examples/reference-preflight", + "/fixtures/assurance-bundle", + "/fixtures/reference-preflight", "/README.md", "/CHANGELOG.md", "/CITATION.cff", @@ -74,7 +80,7 @@ include = [ [tool.pytest.ini_options] addopts = "-ra" -pythonpath = ["src"] +pythonpath = [".", "src"] markers = [ "integration: tests that require the local full checkpoint inventory", "remote_integration: opt-in tests that access a pinned remote repository", diff --git a/src/omiv/assurance/__init__.py b/src/omiv/assurance/__init__.py new file mode 100644 index 0000000..85a5624 --- /dev/null +++ b/src/omiv/assurance/__init__.py @@ -0,0 +1 @@ +"""Phase 6F Assurance Bundle interoperability.""" diff --git a/src/omiv/assurance/archive.py b/src/omiv/assurance/archive.py new file mode 100644 index 0000000..46cd98e --- /dev/null +++ b/src/omiv/assurance/archive.py @@ -0,0 +1,83 @@ +"""Deterministic, bounded ZIP_STORED transport for portable .omiv bundles.""" + +from __future__ import annotations + +import os +import shutil +import tempfile +import zipfile +from pathlib import Path + +from omiv.assurance.operations import MAX_TOTAL_BYTES +from omiv.errors import OmivInputError +from omiv.payload_integrity.paths import validate_path_set, validate_portable_path + +ARCHIVE_TIMESTAMP = (1980, 1, 1, 0, 0, 0) +MAX_ARCHIVE_ENTRIES = 320 + + +def pack_bundle(root: Path, output: Path) -> None: + if root.is_symlink() or not root.is_dir(): + raise OmivInputError("bundle root must be a regular non-symlink directory") + if output.exists() or output.is_symlink(): + raise OmivInputError(".omiv output must not already exist") + paths = sorted( + (path.relative_to(root).as_posix(), path) for path in root.rglob("*") if path.is_file() + ) + if not paths or len(paths) > MAX_ARCHIVE_ENTRIES: + raise OmivInputError("LIMIT_EXCEEDED:ARCHIVE_ENTRY_COUNT") + validate_path_set(tuple(name for name, _path in paths)) + if any(path.is_symlink() for _name, path in paths): + raise OmivInputError("symbolic links are not permitted in .omiv archives") + total = sum(path.stat().st_size for _name, path in paths) + if total > MAX_TOTAL_BYTES: + raise OmivInputError("LIMIT_EXCEEDED:ARCHIVE_TOTAL_BYTES") + output.parent.mkdir(parents=True, exist_ok=True) + descriptor, temporary_name = tempfile.mkstemp( + prefix=f".{output.name}.", suffix=".tmp", dir=output.parent + ) + os.close(descriptor) + temporary = Path(temporary_name) + try: + with zipfile.ZipFile(temporary, "w", compression=zipfile.ZIP_STORED) as archive: + archive.comment = b"" + for name, path in paths: + info = zipfile.ZipInfo(name, ARCHIVE_TIMESTAMP) + info.compress_type = zipfile.ZIP_STORED + info.create_system = 3 + info.external_attr = 0o100644 << 16 + with path.open("rb") as source, archive.open(info, "w") as destination: + shutil.copyfileobj(source, destination, length=1024 * 1024) + os.replace(temporary, output) + except Exception: + temporary.unlink(missing_ok=True) + raise + + +def extract_archive(archive_path: Path, destination: Path) -> None: + if archive_path.is_symlink() or not archive_path.is_file(): + raise OmivInputError(".omiv input must be a regular non-symlink file") + try: + with zipfile.ZipFile(archive_path, "r") as archive: + infos = archive.infolist() + if not infos or len(infos) > MAX_ARCHIVE_ENTRIES: + raise OmivInputError("LIMIT_EXCEEDED:ARCHIVE_ENTRY_COUNT") + names = tuple(info.filename for info in infos) + validate_path_set(names) + total = 0 + for info in infos: + validate_portable_path(info.filename) + if info.is_dir() or info.compress_type != zipfile.ZIP_STORED: + raise OmivInputError(".omiv requires regular ZIP_STORED entries") + mode = (info.external_attr >> 16) & 0o170000 + if mode not in {0, 0o100000}: + raise OmivInputError(".omiv archive contains a non-regular entry") + total += info.file_size + if total > MAX_TOTAL_BYTES: + raise OmivInputError("LIMIT_EXCEEDED:ARCHIVE_TOTAL_BYTES") + target = destination / Path(*info.filename.split("/")) + target.parent.mkdir(parents=True, exist_ok=True) + with archive.open(info, "r") as source, target.open("wb") as output: + shutil.copyfileobj(source, output, length=1024 * 1024) + except (OSError, zipfile.BadZipFile) as exc: + raise OmivInputError(f"invalid .omiv archive: {exc}") from exc diff --git a/src/omiv/assurance/models.py b/src/omiv/assurance/models.py new file mode 100644 index 0000000..f477c70 --- /dev/null +++ b/src/omiv/assurance/models.py @@ -0,0 +1,441 @@ +"""Strict, bounded models for the Phase 6F Assurance Bundle vertical slice.""" + +from __future__ import annotations + +from enum import StrEnum +from typing import Literal + +from pydantic import Field, JsonValue, model_validator + +from omiv.canonical import canonical_sha256 +from omiv.models import StrictModel +from omiv.payload_integrity.paths import validate_path_set, validate_portable_path + +MAX_MEMBERS = 256 + + +class EvidencePhase(StrEnum): + PHASE_5 = "PHASE_5" + PHASE_6A = "PHASE_6A" + PHASE_6B = "PHASE_6B" + PHASE_6C = "PHASE_6C" + PHASE_6D = "PHASE_6D" + PHASE_6E = "PHASE_6E" + PHASE_6F = "PHASE_6F" + EXTERNAL = "EXTERNAL" + + +class AssuranceDimension(StrEnum): + IDENTITY = "IDENTITY" + STRUCTURE = "STRUCTURE" + TRANSFORMATION = "TRANSFORMATION" + FIDELITY = "FIDELITY" + TOKENIZER_CONFIGURATION = "TOKENIZER_CONFIGURATION" + RUNTIME = "RUNTIME" + PROVENANCE = "PROVENANCE" + TRUST = "TRUST" + OTHER = "OTHER" + + +class VerdictRole(StrEnum): + SUPPORTING = "SUPPORTING" + DIMENSION_VERDICT = "DIMENSION_VERDICT" + + +class VerdictStatus(StrEnum): + PASS = "PASS" + WARN = "WARN" + FAIL = "FAIL" + UNKNOWN = "UNKNOWN" + NOT_TESTED = "NOT_TESTED" + + +class CostClass(StrEnum): + LOCAL_VALIDATION = "LOCAL_VALIDATION" + DOWNLOAD = "DOWNLOAD" + NETWORK = "NETWORK" + CONVERSION = "CONVERSION" + GPU = "GPU" + + +class Availability(StrEnum): + AVAILABLE = "AVAILABLE" + MISSING = "MISSING" + INVALID_LOCAL_OBJECT = "INVALID_LOCAL_OBJECT" + + +class SchemaSupport(StrEnum): + SUPPORTED_AND_VALID = "SUPPORTED_AND_VALID" + UNSUPPORTED = "UNSUPPORTED" + MISMATCH = "MISMATCH" + NOT_DECLARED = "NOT_DECLARED" + NOT_APPLICABLE = "NOT_APPLICABLE" + + +class PreflightStatus(StrEnum): + READY = "READY" + READY_WITH_GAPS = "READY_WITH_GAPS" + BLOCKED = "BLOCKED" + REVIEW_REQUIRED = "REVIEW_REQUIRED" + + +class BundleStatus(StrEnum): + COMPLETE = "COMPLETE" + INCOMPLETE = "INCOMPLETE" + INVALID = "INVALID" + + +class AssuranceRequirement(StrictModel): + member_path: str + source_path: str + phase: EvidencePhase + required: bool = True + media_type: Literal["application/json", "application/octet-stream"] = "application/json" + expected_schema: str | None = Field(default=None, max_length=200) + dimension: AssuranceDimension = AssuranceDimension.OTHER + verdict_role: VerdictRole = VerdictRole.SUPPORTING + + @model_validator(mode="after") + def validate_paths(self) -> AssuranceRequirement: + validate_portable_path(self.member_path) + validate_portable_path(self.source_path) + if self.member_path == "assurance-bundle.json": + raise ValueError("Assurance Bundle manifest must exclude itself") + if self.media_type != "application/json" and self.expected_schema is not None: + raise ValueError("only JSON evidence may declare an expected schema") + if self.media_type == "application/octet-stream" and self.phase != EvidencePhase.EXTERNAL: + raise ValueError("opaque binary evidence is supported only for EXTERNAL members") + return self + + +class PlannedOperation(StrictModel): + cost_class: CostClass + reason: str = Field(min_length=1, max_length=1000) + + +class AssuranceRequest(StrictModel): + schema_id: Literal["omiv.assurance-request.v1"] = Field( + default="omiv.assurance-request.v1", alias="schema" + ) + request_id: str = Field(min_length=3, max_length=128) + subject: str = Field(min_length=1, max_length=1000) + requirements: list[AssuranceRequirement] = Field(min_length=1, max_length=MAX_MEMBERS) + planned_operations: list[PlannedOperation] = Field(default_factory=list, max_length=64) + + @model_validator(mode="after") + def validate_member_paths(self) -> AssuranceRequest: + validate_path_set(tuple(item.member_path for item in self.requirements)) + return self + + +class CostSummary(StrictModel): + download: bool + network: bool + conversion: bool + gpu: bool + reasons: list[str] = Field(max_length=64) + + +class PreflightMember(StrictModel): + member_path: str + source_path: str + phase: EvidencePhase + required: bool + media_type: Literal["application/json", "application/octet-stream"] + expected_schema: str | None = Field(max_length=200) + dimension: AssuranceDimension + verdict_role: VerdictRole + availability: Availability + schema_support: SchemaSupport + size: int | None = Field(default=None, ge=0) + sha256: str | None = Field(default=None, pattern=r"^[0-9a-f]{64}$") + issues: list[str] = Field(max_length=16) + semantic_status: VerdictStatus = VerdictStatus.NOT_TESTED + semantic_summary: str = Field(default="Supporting evidence only.", max_length=1000) + + @model_validator(mode="after") + def validate_state(self) -> PreflightMember: + validate_portable_path(self.member_path) + validate_portable_path(self.source_path) + _validate_member_state(self) + return self + + +class AssurancePlan(StrictModel): + schema_id: Literal["omiv.assurance-plan.v1"] = Field( + default="omiv.assurance-plan.v1", alias="schema" + ) + plan_id: str + plan_digest: str = Field(pattern=r"^[0-9a-f]{64}$") + request_id: str + subject: str + status: PreflightStatus + members: list[PreflightMember] = Field(min_length=1, max_length=MAX_MEMBERS) + costs: CostSummary + limitations: list[str] = Field(max_length=32) + + @model_validator(mode="after") + def validate_identity(self) -> AssurancePlan: + body = self.model_dump(mode="json", by_alias=True, exclude={"plan_id", "plan_digest"}) + digest = canonical_sha256({"domain": self.schema_id, "body": body}) + if self.plan_digest != digest or self.plan_id != f"assurance_plan_{digest[:32]}": + raise ValueError("Assurance Plan canonical identity mismatch") + validate_path_set(tuple(item.member_path for item in self.members)) + return self + + +class BundleMember(StrictModel): + member_path: str + phase: EvidencePhase + required: bool + media_type: Literal["application/json", "application/octet-stream"] + expected_schema: str | None = Field(max_length=200) + dimension: AssuranceDimension + verdict_role: VerdictRole + availability: Availability + schema_support: SchemaSupport + size: int | None = Field(default=None, ge=0) + sha256: str | None = Field(default=None, pattern=r"^[0-9a-f]{64}$") + issues: list[str] = Field(max_length=16) + semantic_status: VerdictStatus + semantic_summary: str = Field(max_length=1000) + + @model_validator(mode="after") + def validate_state(self) -> BundleMember: + validate_portable_path(self.member_path) + _validate_member_state(self) + return self + + +def _validate_member_state(value: PreflightMember | BundleMember) -> None: + if value.availability == Availability.AVAILABLE: + if value.size is None or value.sha256 is None: + raise ValueError("available evidence requires size and SHA-256") + elif value.size is not None or value.sha256 is not None: + raise ValueError("unavailable evidence cannot claim size or SHA-256") + if value.media_type == "application/octet-stream": + if value.phase != EvidencePhase.EXTERNAL or value.expected_schema is not None: + raise ValueError("opaque binary evidence is supported only for EXTERNAL members") + if value.schema_support != SchemaSupport.NOT_APPLICABLE: + raise ValueError("opaque binary evidence schema support must be NOT_APPLICABLE") + elif value.schema_support == SchemaSupport.NOT_APPLICABLE: + raise ValueError("JSON evidence cannot use NOT_APPLICABLE schema support") + + +class CoreFileRecord(StrictModel): + path: str + schema_id: str + size: int = Field(ge=0) + sha256: str = Field(pattern=r"^[0-9a-f]{64}$") + + @model_validator(mode="after") + def validate_path(self) -> CoreFileRecord: + validate_portable_path(self.path) + return self + + +class AssuranceBundleManifest(StrictModel): + schema_id: Literal["omiv.assurance-bundle.v1"] = Field( + default="omiv.assurance-bundle.v1", alias="schema" + ) + bundle_id: str + bundle_digest: str = Field(pattern=r"^[0-9a-f]{64}$") + subject: str + source_plan_id: str + source_plan_digest: str = Field(pattern=r"^[0-9a-f]{64}$") + preflight_status: PreflightStatus + status: BundleStatus + members: list[BundleMember] = Field(min_length=1, max_length=MAX_MEMBERS) + core_files: list[CoreFileRecord] = Field(min_length=6, max_length=16) + bundle_profile: Literal["omiv.assurance-bundle.v1"] = "omiv.assurance-bundle.v1" + required_features: list[str] = Field(max_length=64) + costs: CostSummary + limitations: list[str] = Field(max_length=32) + + @model_validator(mode="after") + def validate_identity(self) -> AssuranceBundleManifest: + body = self.model_dump(mode="json", by_alias=True, exclude={"bundle_id", "bundle_digest"}) + digest = canonical_sha256({"domain": self.schema_id, "body": body}) + if self.bundle_digest != digest or self.bundle_id != f"assurance_bundle_{digest[:32]}": + raise ValueError("Assurance Bundle canonical identity mismatch") + validate_path_set(tuple(item.member_path for item in self.members)) + validate_path_set(tuple(item.path for item in self.core_files)) + return self + + +class AssuranceSubjectDocument(StrictModel): + schema_id: Literal["omiv.assurance-subject.v1"] = Field( + default="omiv.assurance-subject.v1", alias="schema" + ) + subject_id: str + subject_digest: str = Field(pattern=r"^[0-9a-f]{64}$") + display_name: str = Field(min_length=1, max_length=1000) + limitations: list[str] = Field(max_length=16) + + @model_validator(mode="after") + def validate_identity(self) -> AssuranceSubjectDocument: + _validate_identity(self, "subject_id", "subject_digest", "assurance_subject_") + return self + + +class EvidenceIndexDocument(StrictModel): + schema_id: Literal["omiv.assurance-evidence-index.v1"] = Field( + default="omiv.assurance-evidence-index.v1", alias="schema" + ) + index_id: str + index_digest: str = Field(pattern=r"^[0-9a-f]{64}$") + entries: list[BundleMember] = Field(min_length=1, max_length=MAX_MEMBERS) + + @model_validator(mode="after") + def validate_identity(self) -> EvidenceIndexDocument: + validate_path_set(tuple(item.member_path for item in self.entries)) + _validate_identity(self, "index_id", "index_digest", "assurance_index_") + return self + + +class DimensionVerdict(StrictModel): + dimension: AssuranceDimension + status: VerdictStatus + summary: str = Field(max_length=1000) + evidence_paths: list[str] = Field(max_length=MAX_MEMBERS) + + +class AssuranceVerdictDocument(StrictModel): + schema_id: Literal["omiv.assurance-verdict.v1"] = Field( + default="omiv.assurance-verdict.v1", alias="schema" + ) + verdict_id: str + verdict_digest: str = Field(pattern=r"^[0-9a-f]{64}$") + overall: VerdictStatus + dimensions: list[DimensionVerdict] = Field(min_length=1, max_length=32) + unresolved_claims: int = Field(ge=0) + integrity_failures: int = Field(ge=0) + summary: str = Field(max_length=2000) + limitations: list[str] = Field(max_length=32) + + @model_validator(mode="after") + def validate_identity(self) -> AssuranceVerdictDocument: + _validate_identity(self, "verdict_id", "verdict_digest", "assurance_verdict_") + return self + + +class FindingRecord(StrictModel): + code: str + severity: Literal["INFO", "WARN", "ERROR"] + member_path: str | None = None + detail: str = Field(max_length=2000) + + +class FindingsDocument(StrictModel): + schema_id: Literal["omiv.assurance-findings.v1"] = Field( + default="omiv.assurance-findings.v1", alias="schema" + ) + findings_id: str + findings_digest: str = Field(pattern=r"^[0-9a-f]{64}$") + findings: list[FindingRecord] = Field(max_length=MAX_MEMBERS * 4) + + @model_validator(mode="after") + def validate_identity(self) -> FindingsDocument: + _validate_identity(self, "findings_id", "findings_digest", "assurance_findings_") + return self + + +class UnknownRecord(StrictModel): + member_path: str + dimension: AssuranceDimension + required: bool + reason: str = Field(max_length=1000) + next_action: str = Field(max_length=1000) + + +class UnknownsDocument(StrictModel): + schema_id: Literal["omiv.assurance-unknowns.v1"] = Field( + default="omiv.assurance-unknowns.v1", alias="schema" + ) + unknowns_id: str + unknowns_digest: str = Field(pattern=r"^[0-9a-f]{64}$") + unknowns: list[UnknownRecord] = Field(max_length=MAX_MEMBERS * 2) + + @model_validator(mode="after") + def validate_identity(self) -> UnknownsDocument: + _validate_identity(self, "unknowns_id", "unknowns_digest", "assurance_unknowns_") + return self + + +class CapabilitiesDocument(StrictModel): + schema_id: Literal["omiv.assurance-capabilities.v1"] = Field( + default="omiv.assurance-capabilities.v1", alias="schema" + ) + profile: Literal["omiv.assurance-bundle.v1"] = "omiv.assurance-bundle.v1" + features: list[str] = Field(min_length=1, max_length=64) + transports: list[Literal["DIRECTORY", "ZIP_STORED"]] + signature_algorithms: list[Literal["ED25519"]] + supported_evidence_schemas: list[str] = Field(max_length=4096) + limits: dict[str, int] + + +class AssuranceSignature(StrictModel): + schema_id: Literal["omiv.assurance-signature.v1"] = Field( + default="omiv.assurance-signature.v1", alias="schema" + ) + signature_id: str + signature_digest: str = Field(pattern=r"^[0-9a-f]{64}$") + algorithm: Literal["ED25519"] = "ED25519" + key_id: str = Field(min_length=3, max_length=200) + public_key: str = Field(pattern=r"^[0-9a-f]{64}$") + public_key_sha256: str = Field(pattern=r"^[0-9a-f]{64}$") + manifest_digest: str = Field(pattern=r"^[0-9a-f]{64}$") + signature: str = Field(pattern=r"^[0-9a-f]{128}$") + limitations: list[str] = Field(max_length=16) + + @model_validator(mode="after") + def validate_identity(self) -> AssuranceSignature: + _validate_identity(self, "signature_id", "signature_digest", "assurance_signature_") + return self + + +class AssuranceTrustPolicy(StrictModel): + schema_id: Literal["omiv.assurance-trust-policy.v1"] = Field( + default="omiv.assurance-trust-policy.v1", alias="schema" + ) + policy_id: str = Field(min_length=3, max_length=200) + require_signature: bool = True + minimum_valid_signatures: int = Field(default=1, ge=0, le=32) + allowed_key_ids: list[str] = Field(default_factory=list, max_length=64) + allowed_public_key_sha256: list[str] = Field(default_factory=list, max_length=64) + + +def _validate_identity(value: StrictModel, id_field: str, digest_field: str, prefix: str) -> None: + body = value.model_dump(mode="json", by_alias=True, exclude={id_field, digest_field}) + schema_id = value.model_dump(mode="json", by_alias=True)["schema"] + digest = canonical_sha256({"domain": schema_id, "body": body}) + if getattr(value, digest_field) != digest or getattr(value, id_field) != prefix + digest[:32]: + raise ValueError(f"{schema_id} canonical identity mismatch") + + +class VerificationFinding(StrictModel): + member_path: str | None = None + code: str + detail: str + + +class AssuranceVerificationReport(StrictModel): + schema_id: Literal["omiv.assurance-verification-report.v1"] = Field( + default="omiv.assurance-verification-report.v1", + alias="schema", + ) + bundle_id: str + bundle_digest: str + status: BundleStatus + available: int = Field(ge=0) + missing: int = Field(ge=0) + unknown: int = Field(ge=0) + invalid: int = Field(ge=0) + transport: Literal["DIRECTORY", "ZIP_STORED"] = "DIRECTORY" + valid_signatures: int = Field(default=0, ge=0) + trusted_signatures: int = Field(default=0, ge=0) + signature_status: Literal["NOT_PRESENT", "VALID_UNTRUSTED", "TRUSTED", "INVALID"] = ( + "NOT_PRESENT" + ) + findings: list[VerificationFinding] = Field(max_length=MAX_MEMBERS * 4) + evidence: dict[str, JsonValue] = Field(default_factory=dict) diff --git a/src/omiv/assurance/operations.py b/src/omiv/assurance/operations.py new file mode 100644 index 0000000..33e49a4 --- /dev/null +++ b/src/omiv/assurance/operations.py @@ -0,0 +1,848 @@ +"""Offline preflight, assembly, and verification for Assurance Bundles.""" + +from __future__ import annotations + +import hashlib +import json +import os +import shutil +import tempfile +from pathlib import Path +from typing import Any + +from pydantic import BaseModel, ValidationError + +from omiv.assurance.models import ( + AssuranceBundleManifest, + AssuranceDimension, + AssurancePlan, + AssuranceRequest, + AssuranceSubjectDocument, + AssuranceVerdictDocument, + AssuranceVerificationReport, + Availability, + BundleMember, + BundleStatus, + CapabilitiesDocument, + CoreFileRecord, + CostClass, + CostSummary, + DimensionVerdict, + EvidenceIndexDocument, + EvidencePhase, + FindingRecord, + FindingsDocument, + PreflightMember, + PreflightStatus, + SchemaSupport, + UnknownRecord, + UnknownsDocument, + VerdictRole, + VerdictStatus, + VerificationFinding, +) +from omiv.assurance.projection import project_semantic_status +from omiv.assurance.registry import schema_model, supported_schemas +from omiv.canonical import canonical_sha256, load_json_value +from omiv.errors import OmivInputError +from omiv.hf.json_loader import load_bounded_json +from omiv.safe_write import atomic_write_text + +MAX_CONTROL_BYTES = 8 * 1024 * 1024 +MAX_JSON_EVIDENCE_BYTES = 64 * 1024 * 1024 +MAX_MEMBER_BYTES = 1024 * 1024 * 1024 +MAX_TOTAL_BYTES = 4 * 1024 * 1024 * 1024 +MANIFEST_NAME = "assurance-bundle.json" +CORE_FILES = ( + "subject.json", + "verdict.json", + "evidence-index.json", + "findings.json", + "unknowns.json", + "capabilities.json", +) +FEATURES = ( + "concise-verdict", + "explicit-unknowns", + "phase5-through-6e-schema-validation", + "portable-directory", + "deterministic-zip-stored", + "detached-ed25519-signatures", + "offline-verification", +) +CORE_MODELS: dict[str, type[BaseModel]] = { + "omiv.assurance-subject.v1": AssuranceSubjectDocument, + "omiv.assurance-verdict.v1": AssuranceVerdictDocument, + "omiv.assurance-evidence-index.v1": EvidenceIndexDocument, + "omiv.assurance-findings.v1": FindingsDocument, + "omiv.assurance-unknowns.v1": UnknownsDocument, + "omiv.assurance-capabilities.v1": CapabilitiesDocument, +} + + +def _pretty(model: BaseModel) -> str: + return ( + json.dumps( + model.model_dump(mode="json", by_alias=True), + ensure_ascii=False, + indent=2, + sort_keys=True, + ) + + "\n" + ) + + +def load_request(path: Path) -> AssuranceRequest: + value, _ = load_bounded_json(path, max_bytes=MAX_CONTROL_BYTES) + try: + return AssuranceRequest.model_validate(value) + except (ValidationError, ValueError) as exc: + raise OmivInputError(f"invalid Assurance Bundle request: {exc}") from exc + + +def load_plan(path: Path) -> AssurancePlan: + value, _ = load_bounded_json(path, max_bytes=MAX_CONTROL_BYTES) + try: + return AssurancePlan.model_validate(value) + except (ValidationError, ValueError) as exc: + raise OmivInputError(f"invalid Assurance Plan: {exc}") from exc + + +def load_manifest(path: Path) -> AssuranceBundleManifest: + value, _ = load_bounded_json(path, max_bytes=MAX_CONTROL_BYTES) + try: + return AssuranceBundleManifest.model_validate(value) + except (ValidationError, ValueError) as exc: + raise OmivInputError(f"invalid Assurance Bundle manifest: {exc}") from exc + + +def _safe_source(root: Path, portable_path: str) -> Path: + resolved_root = root.resolve() + candidate = (resolved_root / Path(*portable_path.split("/"))).resolve(strict=False) + if candidate != resolved_root and resolved_root not in candidate.parents: + raise OmivInputError("evidence source escapes the declared root") + return candidate + + +def _hash_file(path: Path) -> tuple[int, str]: + digest = hashlib.sha256() + size = 0 + with path.open("rb") as source: + while chunk := source.read(1024 * 1024): + size += len(chunk) + digest.update(chunk) + return size, digest.hexdigest() + + +def _schema_status( + raw: bytes, phase: EvidencePhase, expected: str | None +) -> tuple[SchemaSupport, list[str], dict[str, Any] | None]: + if expected is None: + return SchemaSupport.NOT_DECLARED, ["EVIDENCE_SCHEMA_NOT_DECLARED"], None + model = schema_model(phase, expected) + if model is None: + return SchemaSupport.UNSUPPORTED, ["EVIDENCE_SCHEMA_UNSUPPORTED_FOR_PHASE"], None + try: + value = load_json_value(raw.decode("utf-8")) + except (UnicodeError, OmivInputError): + return SchemaSupport.MISMATCH, ["EVIDENCE_JSON_INVALID"], None + if not isinstance(value, dict) or value.get("schema") != expected: + return SchemaSupport.MISMATCH, ["EVIDENCE_SCHEMA_MISMATCH"], None + try: + model.model_validate(value) + except (ValidationError, ValueError): + return SchemaSupport.MISMATCH, ["EVIDENCE_SCHEMA_VALIDATION_FAILED"], value + return SchemaSupport.SUPPORTED_AND_VALID, [], value + + +def _inspect_member(root: Path, requirement: Any) -> PreflightMember: + source = _safe_source(root, requirement.source_path) + issues: list[str] = [] + if not source.exists(): + return PreflightMember( + **requirement.model_dump(mode="json"), + availability=Availability.MISSING, + schema_support=( + SchemaSupport.NOT_APPLICABLE + if requirement.media_type != "application/json" + else SchemaSupport.NOT_DECLARED + ), + issues=[ + "REQUIRED_EVIDENCE_MISSING" if requirement.required else "OPTIONAL_EVIDENCE_MISSING" + ], + ) + if source.is_symlink() or not source.is_file(): + return PreflightMember( + **requirement.model_dump(mode="json"), + availability=Availability.INVALID_LOCAL_OBJECT, + schema_support=( + SchemaSupport.NOT_APPLICABLE + if requirement.media_type != "application/json" + else SchemaSupport.NOT_DECLARED + ), + issues=["EVIDENCE_MUST_BE_REGULAR_NON_SYMLINK_FILE"], + ) + size = source.stat().st_size + if size > MAX_MEMBER_BYTES: + return PreflightMember( + **requirement.model_dump(mode="json"), + availability=Availability.INVALID_LOCAL_OBJECT, + schema_support=( + SchemaSupport.NOT_APPLICABLE + if requirement.media_type != "application/json" + else SchemaSupport.NOT_DECLARED + ), + issues=["LIMIT_EXCEEDED:MEMBER_BYTES"], + ) + if requirement.media_type == "application/json" and size > MAX_JSON_EVIDENCE_BYTES: + return PreflightMember( + **requirement.model_dump(mode="json"), + availability=Availability.INVALID_LOCAL_OBJECT, + schema_support=SchemaSupport.NOT_DECLARED, + issues=["LIMIT_EXCEEDED:JSON_EVIDENCE_BYTES"], + ) + raw = source.read_bytes() if requirement.media_type == "application/json" else b"" + if requirement.media_type == "application/json": + support, issues, value = _schema_status(raw, requirement.phase, requirement.expected_schema) + else: + support, value = SchemaSupport.NOT_APPLICABLE, None + observed_size, observed_digest = ( + (len(raw), hashlib.sha256(raw).hexdigest()) if raw else _hash_file(source) + ) + semantic_status = VerdictStatus.NOT_TESTED + semantic_summary = "Supporting evidence only." + if requirement.verdict_role == VerdictRole.DIMENSION_VERDICT: + if support == SchemaSupport.SUPPORTED_AND_VALID and value is not None: + semantic_status, semantic_summary = project_semantic_status(value) + else: + semantic_status = VerdictStatus.UNKNOWN + semantic_summary = "Evidence was unavailable or not semantically verifiable." + return PreflightMember( + **requirement.model_dump(mode="json"), + availability=Availability.AVAILABLE, + schema_support=support, + size=observed_size, + sha256=observed_digest, + issues=issues, + semantic_status=semantic_status, + semantic_summary=semantic_summary, + ) + + +def build_preflight(request: AssuranceRequest, root: Path) -> AssurancePlan: + members = [_inspect_member(root, item) for item in request.requirements] + total = sum(item.size or 0 for item in members) + if total > MAX_TOTAL_BYTES: + raise OmivInputError("LIMIT_EXCEEDED:TOTAL_MEMBER_BYTES") + expensive = {item.cost_class for item in request.planned_operations} + costs = CostSummary( + download=CostClass.DOWNLOAD in expensive, + network=CostClass.NETWORK in expensive, + conversion=CostClass.CONVERSION in expensive, + gpu=CostClass.GPU in expensive, + reasons=[item.reason for item in request.planned_operations], + ) + required_blocker = any( + item.required + and ( + item.availability != Availability.AVAILABLE + or item.schema_support + not in {SchemaSupport.SUPPORTED_AND_VALID, SchemaSupport.NOT_APPLICABLE} + ) + for item in members + ) + has_gap = any( + item.availability != Availability.AVAILABLE + or item.schema_support in {SchemaSupport.NOT_DECLARED, SchemaSupport.UNSUPPORTED} + for item in members + ) + if required_blocker: + status = PreflightStatus.BLOCKED + elif any((costs.download, costs.network, costs.conversion, costs.gpu)): + status = PreflightStatus.REVIEW_REQUIRED + elif has_gap: + status = PreflightStatus.READY_WITH_GAPS + else: + status = PreflightStatus.READY + limitations = [ + "Preflight performs local bounded inspection only; it does not download, convert, " + "use a network, or use a GPU.", + "Bundle interoperability does not replace or upgrade Phase 5 or Phase 6A-6E " + "semantic verdicts.", + ] + body: dict[str, Any] = { + "schema": "omiv.assurance-plan.v1", + "request_id": request.request_id, + "subject": request.subject, + "status": status.value, + "members": [item.model_dump(mode="json") for item in members], + "costs": costs.model_dump(mode="json"), + "limitations": limitations, + } + digest = canonical_sha256({"domain": body["schema"], "body": body}) + return AssurancePlan.model_validate( + {**body, "plan_id": f"assurance_plan_{digest[:32]}", "plan_digest": digest} + ) + + +def write_plan(plan: AssurancePlan, output: Path) -> None: + atomic_write_text(output, _pretty(plan)) + + +def _finalized( + model: type[BaseModel], body: dict[str, Any], id_field: str, digest_field: str, prefix: str +) -> BaseModel: + digest = canonical_sha256({"domain": body["schema"], "body": body}) + return model.model_validate({**body, id_field: prefix + digest[:32], digest_field: digest}) + + +def _core_documents(subject_name: str, members: list[BundleMember]) -> dict[str, BaseModel]: + subject = _finalized( + AssuranceSubjectDocument, + { + "schema": "omiv.assurance-subject.v1", + "display_name": subject_name, + "limitations": [ + "The bundle subject label is caller supplied and does not establish identity." + ], + }, + "subject_id", + "subject_digest", + "assurance_subject_", + ) + index = _finalized( + EvidenceIndexDocument, + { + "schema": "omiv.assurance-evidence-index.v1", + "entries": [item.model_dump(mode="json") for item in members], + }, + "index_id", + "index_digest", + "assurance_index_", + ) + finding_records = [ + FindingRecord( + code=issue, + severity=( + "ERROR" + if item.required + or item.schema_support == SchemaSupport.MISMATCH + or item.availability == Availability.INVALID_LOCAL_OBJECT + else "WARN" + ), + member_path=item.member_path, + detail=item.semantic_summary, + ) + for item in members + for issue in item.issues + ] + findings = _finalized( + FindingsDocument, + { + "schema": "omiv.assurance-findings.v1", + "findings": [item.model_dump(mode="json") for item in finding_records], + }, + "findings_id", + "findings_digest", + "assurance_findings_", + ) + unknown_records = [ + UnknownRecord( + member_path=item.member_path, + dimension=item.dimension, + required=item.required, + reason=( + item.semantic_summary + if item.semantic_status == VerdictStatus.UNKNOWN + else "; ".join(item.issues) or "Evidence is not available for verification." + ), + next_action=("Supply supported canonical evidence and rebuild the Assurance Bundle."), + ) + for item in members + if item.availability != Availability.AVAILABLE + or item.schema_support + in { + SchemaSupport.NOT_DECLARED, + SchemaSupport.UNSUPPORTED, + SchemaSupport.MISMATCH, + } + or ( + item.verdict_role == VerdictRole.DIMENSION_VERDICT + and item.semantic_status in {VerdictStatus.UNKNOWN, VerdictStatus.NOT_TESTED} + ) + ] + unknowns = _finalized( + UnknownsDocument, + { + "schema": "omiv.assurance-unknowns.v1", + "unknowns": [item.model_dump(mode="json") for item in unknown_records], + }, + "unknowns_id", + "unknowns_digest", + "assurance_unknowns_", + ) + dimension_verdicts: list[DimensionVerdict] = [] + for dimension in AssuranceDimension: + selected = [ + item + for item in members + if item.dimension == dimension and item.verdict_role == VerdictRole.DIMENSION_VERDICT + ] + if not selected: + continue + statuses = {item.semantic_status for item in selected} + if VerdictStatus.FAIL in statuses: + status = VerdictStatus.FAIL + elif VerdictStatus.WARN in statuses: + status = VerdictStatus.WARN + elif VerdictStatus.UNKNOWN in statuses or VerdictStatus.NOT_TESTED in statuses: + status = VerdictStatus.UNKNOWN + elif statuses == {VerdictStatus.PASS}: + status = VerdictStatus.PASS + else: + status = VerdictStatus.UNKNOWN + dimension_verdicts.append( + DimensionVerdict( + dimension=dimension, + status=status, + summary="; ".join(sorted({item.semantic_summary for item in selected})), + evidence_paths=sorted(item.member_path for item in selected), + ) + ) + if not dimension_verdicts: + dimension_verdicts = [ + DimensionVerdict( + dimension=AssuranceDimension.OTHER, + status=VerdictStatus.NOT_TESTED, + summary="No member was designated as a dimension verdict.", + evidence_paths=[], + ) + ] + dimension_statuses = {item.status for item in dimension_verdicts} + integrity_failures = sum( + item.availability == Availability.INVALID_LOCAL_OBJECT + or item.schema_support == SchemaSupport.MISMATCH + for item in members + ) + if integrity_failures or VerdictStatus.FAIL in dimension_statuses: + overall = VerdictStatus.FAIL + elif unknown_records or VerdictStatus.WARN in dimension_statuses: + overall = VerdictStatus.WARN + elif dimension_statuses == {VerdictStatus.PASS}: + overall = VerdictStatus.PASS + else: + overall = VerdictStatus.UNKNOWN + verdict = _finalized( + AssuranceVerdictDocument, + { + "schema": "omiv.assurance-verdict.v1", + "overall": overall.value, + "dimensions": [item.model_dump(mode="json") for item in dimension_verdicts], + "unresolved_claims": len(unknown_records), + "integrity_failures": integrity_failures, + "summary": ( + "Evidence was projected conservatively; UNKNOWN and NOT_TESTED were not " + "promoted to PASS." + ), + "limitations": [ + "The concise verdict is derived from supplied canonical evidence only.", + "Phase 6F does not replace the originating phase verifier or policy.", + ], + }, + "verdict_id", + "verdict_digest", + "assurance_verdict_", + ) + capabilities = CapabilitiesDocument( + features=list(FEATURES), + transports=["DIRECTORY", "ZIP_STORED"], + signature_algorithms=["ED25519"], + supported_evidence_schemas=list(supported_schemas()), + limits={ + "maximum_members": 256, + "maximum_member_bytes": MAX_MEMBER_BYTES, + "maximum_total_bytes": MAX_TOTAL_BYTES, + }, + ) + return { + "subject.json": subject, + "verdict.json": verdict, + "evidence-index.json": index, + "findings.json": findings, + "unknowns.json": unknowns, + "capabilities.json": capabilities, + } + + +def _manifest_from_plan( + plan: AssurancePlan, members: list[BundleMember], core_files: list[CoreFileRecord] +) -> AssuranceBundleManifest: + complete = all( + item.availability == Availability.AVAILABLE + and item.schema_support in {SchemaSupport.SUPPORTED_AND_VALID, SchemaSupport.NOT_APPLICABLE} + for item in members + ) + body: dict[str, Any] = { + "schema": "omiv.assurance-bundle.v1", + "subject": plan.subject, + "source_plan_id": plan.plan_id, + "source_plan_digest": plan.plan_digest, + "preflight_status": plan.status.value, + "status": (BundleStatus.COMPLETE if complete else BundleStatus.INCOMPLETE).value, + "members": [item.model_dump(mode="json") for item in members], + "core_files": [item.model_dump(mode="json") for item in core_files], + "bundle_profile": "omiv.assurance-bundle.v1", + "required_features": list(FEATURES), + "costs": plan.costs.model_dump(mode="json"), + "limitations": plan.limitations, + } + digest = canonical_sha256({"domain": body["schema"], "body": body}) + return AssuranceBundleManifest.model_validate( + {**body, "bundle_id": f"assurance_bundle_{digest[:32]}", "bundle_digest": digest} + ) + + +def build_bundle(plan: AssurancePlan, root: Path, output: Path) -> AssuranceBundleManifest: + if output.exists() or output.is_symlink(): + raise OmivInputError("Assurance Bundle output directory must not already exist") + parent = output.parent.resolve() + parent.mkdir(parents=True, exist_ok=True) + temporary = Path(tempfile.mkdtemp(prefix=f".{output.name}.", dir=parent)) + try: + members = [ + BundleMember.model_validate(item.model_dump(mode="json", exclude={"source_path"})) + for item in plan.members + ] + for member in plan.members: + if member.availability != Availability.AVAILABLE: + continue + source = _safe_source(root, member.source_path) + if source.is_symlink() or not source.is_file(): + raise OmivInputError(f"evidence changed after preflight: {member.source_path}") + observed_size, observed_digest = _hash_file(source) + if observed_size != member.size or observed_digest != member.sha256: + raise OmivInputError(f"evidence changed after preflight: {member.source_path}") + destination = temporary / Path(*member.member_path.split("/")) + destination.parent.mkdir(parents=True, exist_ok=True) + with source.open("rb") as source_handle, destination.open("wb") as output_handle: + shutil.copyfileobj(source_handle, output_handle, length=1024 * 1024) + core_documents = _core_documents(plan.subject, members) + core_files: list[CoreFileRecord] = [] + for path, document in core_documents.items(): + raw = _pretty(document).encode("utf-8") + (temporary / path).write_bytes(raw) + core_files.append( + CoreFileRecord( + path=path, + schema_id=document.model_dump(mode="json", by_alias=True)["schema"], + size=len(raw), + sha256=hashlib.sha256(raw).hexdigest(), + ) + ) + manifest = _manifest_from_plan(plan, members, core_files) + (temporary / MANIFEST_NAME).write_text(_pretty(manifest), encoding="utf-8") + os.replace(temporary, output) + except Exception: + shutil.rmtree(temporary, ignore_errors=True) + raise + return manifest + + +def verify_bundle(root: Path) -> AssuranceVerificationReport: + if root.is_symlink() or not root.is_dir(): + raise OmivInputError("Assurance Bundle root must be a regular non-symlink directory") + manifest = load_manifest(root / MANIFEST_NAME) + if manifest.bundle_profile != "omiv.assurance-bundle.v1" or not set( + manifest.required_features + ).issubset(FEATURES): + raise OmivInputError("unsupported Assurance Bundle profile or required feature") + expected_files = { + item.member_path for item in manifest.members if item.availability == Availability.AVAILABLE + } | {item.path for item in manifest.core_files} + bundle_paths = list(root.rglob("*")) + signature_files = { + path.relative_to(root).as_posix() + for path in bundle_paths + if path.is_file() and path.parent == root / "signatures" and path.suffix == ".json" + } + actual_files = { + path.relative_to(root).as_posix() + for path in bundle_paths + if path.is_file() + and path.relative_to(root).as_posix() != MANIFEST_NAME + and path.relative_to(root).as_posix() not in signature_files + } + findings: list[VerificationFinding] = [] + core_file_set_valid = {item.path for item in manifest.core_files} == set(CORE_FILES) + if not core_file_set_valid: + findings.append( + VerificationFinding( + code="CORE_FILE_SET_MISMATCH", + detail="the v1 profile requires exactly the six canonical core documents", + ) + ) + symlinks = [path.relative_to(root).as_posix() for path in bundle_paths if path.is_symlink()] + if symlinks: + findings.append( + VerificationFinding( + code="BUNDLE_SYMLINK_PRESENT", + detail=f"observed {len(symlinks)} symbolic links", + ) + ) + if actual_files != expected_files: + findings.append( + VerificationFinding( + code="BUNDLE_FILE_SET_MISMATCH", + detail=( + f"expected {len(expected_files)} payload files, observed {len(actual_files)}" + ), + ) + ) + available = missing = unknown = 0 + invalid = int(not core_file_set_valid) + parsed_core: dict[str, BaseModel] = {} + for core in manifest.core_files: + target = root / core.path + if target.is_symlink() or not target.is_file(): + invalid += 1 + findings.append( + VerificationFinding( + member_path=core.path, + code="CORE_FILE_MISSING", + detail="required Phase 6F core document is unavailable", + ) + ) + continue + if target.stat().st_size > MAX_CONTROL_BYTES: + invalid += 1 + findings.append( + VerificationFinding( + member_path=core.path, + code="LIMIT_EXCEEDED:CORE_FILE_BYTES", + detail="core document exceeds the bounded parser limit", + ) + ) + continue + raw = target.read_bytes() + if len(raw) != core.size or hashlib.sha256(raw).hexdigest() != core.sha256: + invalid += 1 + findings.append( + VerificationFinding( + member_path=core.path, + code="CORE_FILE_INTEGRITY_MISMATCH", + detail="size or SHA-256 differs from the manifest", + ) + ) + continue + try: + value = load_json_value(raw.decode("utf-8")) + model = CORE_MODELS.get(core.schema_id) + if ( + not isinstance(value, dict) + or model is None + or value.get("schema") != core.schema_id + ): + raise ValueError("unsupported core schema") + parsed_core[core.path] = model.model_validate(value) + except (UnicodeError, OmivInputError, ValidationError, ValueError) as exc: + invalid += 1 + findings.append( + VerificationFinding( + member_path=core.path, + code="CORE_FILE_SCHEMA_INVALID", + detail=str(exc), + ) + ) + parsed_subject = parsed_core.get("subject.json") + if isinstance(parsed_subject, AssuranceSubjectDocument): + expected_core = _core_documents(parsed_subject.display_name, manifest.members) + for path, expected in expected_core.items(): + actual = parsed_core.get(path) + if actual is None or actual.model_dump( + mode="json", by_alias=True + ) != expected.model_dump(mode="json", by_alias=True): + invalid += 1 + findings.append( + VerificationFinding( + member_path=path, + code="CORE_PROJECTION_MISMATCH", + detail="core document does not reconstruct from indexed evidence", + ) + ) + observed_total = 0 + for member in manifest.members: + target = root / Path(*member.member_path.split("/")) + if member.availability != Availability.AVAILABLE: + missing += 1 + findings.append( + VerificationFinding( + member_path=member.member_path, + code="EVIDENCE_MISSING", + detail="required" if member.required else "optional", + ) + ) + if target.exists(): + invalid += 1 + continue + if target.is_symlink() or not target.is_file(): + invalid += 1 + findings.append( + VerificationFinding( + member_path=member.member_path, + code="EVIDENCE_FILE_INVALID", + detail="expected a regular non-symlink file", + ) + ) + continue + if target.stat().st_size > MAX_MEMBER_BYTES: + invalid += 1 + findings.append( + VerificationFinding( + member_path=member.member_path, + code="LIMIT_EXCEEDED:MEMBER_BYTES", + detail="evidence exceeds the portable member limit", + ) + ) + continue + observed_size, observed_digest = _hash_file(target) + observed_total += observed_size + if observed_size != member.size or observed_digest != member.sha256: + invalid += 1 + findings.append( + VerificationFinding( + member_path=member.member_path, + code="EVIDENCE_INTEGRITY_MISMATCH", + detail="size or SHA-256 differs from the manifest", + ) + ) + continue + if member.media_type == "application/json": + if observed_size > MAX_JSON_EVIDENCE_BYTES: + invalid += 1 + findings.append( + VerificationFinding( + member_path=member.member_path, + code="LIMIT_EXCEEDED:JSON_EVIDENCE_BYTES", + detail="JSON evidence exceeds the bounded parser limit", + ) + ) + continue + raw = target.read_bytes() + support, issues, value = _schema_status(raw, member.phase, member.expected_schema) + if support != SchemaSupport.SUPPORTED_AND_VALID: + unknown += int(support in {SchemaSupport.NOT_DECLARED, SchemaSupport.UNSUPPORTED}) + invalid += int(support == SchemaSupport.MISMATCH) + findings.extend( + VerificationFinding( + member_path=member.member_path, + code=issue, + detail=support.value, + ) + for issue in issues + ) + continue + if member.verdict_role == VerdictRole.DIMENSION_VERDICT: + if value is None: + invalid += 1 + findings.append( + VerificationFinding( + member_path=member.member_path, + code="SEMANTIC_PROJECTION_UNAVAILABLE", + detail="validated verdict evidence did not produce a canonical object", + ) + ) + continue + semantic_status, semantic_summary = project_semantic_status(value) + if ( + semantic_status != member.semantic_status + or semantic_summary != member.semantic_summary + ): + invalid += 1 + findings.append( + VerificationFinding( + member_path=member.member_path, + code="SEMANTIC_PROJECTION_MISMATCH", + detail=( + "manifest verdict projection does not reconstruct from " + "canonical evidence" + ), + ) + ) + continue + available += 1 + if observed_total > MAX_TOTAL_BYTES: + invalid += 1 + findings.append( + VerificationFinding( + code="LIMIT_EXCEEDED:TOTAL_MEMBER_BYTES", + detail="evidence exceeds the portable bundle total limit", + ) + ) + if invalid or symlinks or actual_files != expected_files or not core_file_set_valid: + status = BundleStatus.INVALID + elif missing or unknown or manifest.status != BundleStatus.COMPLETE: + status = BundleStatus.INCOMPLETE + else: + status = BundleStatus.COMPLETE + return AssuranceVerificationReport( + bundle_id=manifest.bundle_id, + bundle_digest=manifest.bundle_digest, + status=status, + available=available, + missing=missing, + unknown=unknown, + invalid=invalid, + findings=findings, + evidence={ + "offline": True, + "manifest_member_count": len(manifest.members), + "payload_file_count": len(actual_files), + "detached_signature_file_count": len(signature_files), + "phase_boundaries_preserved": True, + }, + ) + + +def write_verification_report(report: AssuranceVerificationReport, output: Path) -> None: + atomic_write_text(output, _pretty(report)) + + +def concise_plan_summary(plan: AssurancePlan) -> str: + missing = sum(item.availability != Availability.AVAILABLE for item in plan.members) + unknown = sum( + item.schema_support in {SchemaSupport.NOT_DECLARED, SchemaSupport.UNSUPPORTED} + for item in plan.members + ) + costly = [ + name + for name, enabled in ( + ("download", plan.costs.download), + ("network", plan.costs.network), + ("conversion", plan.costs.conversion), + ("gpu", plan.costs.gpu), + ) + if enabled + ] + return ( + f"{plan.status.value} members={len(plan.members)} missing={missing} unknown={unknown} " + f"costly={','.join(costly) if costly else 'none'}" + ) + + +def concise_bundle_summary(manifest: AssuranceBundleManifest) -> str: + missing = sum(item.availability != Availability.AVAILABLE for item in manifest.members) + unknown = sum( + item.schema_support in {SchemaSupport.NOT_DECLARED, SchemaSupport.UNSUPPORTED} + for item in manifest.members + ) + return ( + f"{manifest.status.value} bundle={manifest.bundle_id} members={len(manifest.members)} " + f"missing={missing} unknown={unknown}" + ) + + +def concise_verification_summary(report: AssuranceVerificationReport) -> str: + return ( + f"{report.status.value} bundle={report.bundle_id} available={report.available} " + f"missing={report.missing} unknown={report.unknown} invalid={report.invalid}" + ) diff --git a/src/omiv/assurance/projection.py b/src/omiv/assurance/projection.py new file mode 100644 index 0000000..8509b20 --- /dev/null +++ b/src/omiv/assurance/projection.py @@ -0,0 +1,58 @@ +"""Conservative projection of canonical evidence into concise Phase 6F verdicts.""" + +from __future__ import annotations + +from typing import Any + +from omiv.assurance.models import VerdictStatus + +_POSITIVE_EXACT = { + "PASS", + "VERIFIED", + "VALID", + "APPROVED", + "EXACT_MATCH_FOR_EXPECTATION_SCOPE", + "CONFORMS_FOR_DECLARED_SCOPE", + "PARITY_ESTABLISHED_FOR_DECLARED_SCOPE", + "SATISFACTORY_FOR_DECLARED_SCOPE", + "COMPLETE_FOR_DECLARED_LOCAL_SCOPE", + "MATCHES_COMPLETE_EXPECTATION", + "VERIFIED_WITH_LIMITATIONS", +} + + +def project_semantic_status(value: dict[str, Any]) -> tuple[VerdictStatus, str]: + """Project only explicit status fields; never infer truth from schema validity.""" + selected_name = "" + selected = "" + for name in ("overall_status", "status", "outcome", "verdict", "decision_outcome"): + candidate = value.get(name) + if isinstance(candidate, str): + selected_name, selected = name, candidate.upper() + break + if not selected: + return VerdictStatus.UNKNOWN, "No supported semantic verdict field was present." + if selected in _POSITIVE_EXACT: + return VerdictStatus.PASS, f"{selected_name}={selected}" + if any( + token in selected + for token in ("MISMATCH", "FAILED", "FAILURE", "INVALID", "REJECTED", "DENIED") + ): + return VerdictStatus.FAIL, f"{selected_name}={selected}" + if any(token in selected for token in ("PARTIAL", "WARNING", "WARN", "LIMITATION")): + return VerdictStatus.WARN, f"{selected_name}={selected}" + if any( + token in selected + for token in ( + "UNKNOWN", + "UNAVAILABLE", + "NOT_EVALUATED", + "NOT_OBSERVED", + "NOT_VERIFIED", + "INSUFFICIENT", + "INCOMPLETE", + "INDETERMINATE", + ) + ): + return VerdictStatus.UNKNOWN, f"{selected_name}={selected}" + return VerdictStatus.UNKNOWN, f"Unsupported semantic value {selected_name}={selected}." diff --git a/src/omiv/assurance/registry.py b/src/omiv/assurance/registry.py new file mode 100644 index 0000000..6d080a2 --- /dev/null +++ b/src/omiv/assurance/registry.py @@ -0,0 +1,81 @@ +"""Phase-aware schema capability registry for Assurance Bundle interoperability.""" + +from __future__ import annotations + +import inspect +from types import ModuleType +from typing import cast + +from pydantic import BaseModel + +from omiv.assurance.models import EvidencePhase +from omiv.attestations import models as attestation_models +from omiv.continuous_trust.schema import SCHEMA_MODELS as CONTINUOUS_TRUST_SCHEMAS +from omiv.custody import models as custody_models +from omiv.governance import models as governance_models +from omiv.passport import models as passport_models +from omiv.payload_integrity.schema import SCHEMA_MODELS as PAYLOAD_SCHEMAS +from omiv.quantization.schema import SCHEMA_MODELS as QUANTIZATION_SCHEMAS +from omiv.reconciliation.schema import SCHEMA_MODELS as RECONCILIATION_SCHEMAS +from omiv.runtime.schema import RUNTIME_SCHEMA_MODELS +from omiv.runtime_resolution.schema import SCHEMA_MODELS as RUNTIME_RESOLUTION_SCHEMAS +from omiv.security.schema import SECURITY_SCHEMA_MODELS +from omiv.tokenizer_parity.schema import SCHEMA_MODELS as TOKENIZER_SCHEMAS +from omiv.trust import models as trust_models + + +def _models_from_module(module: ModuleType) -> dict[str, type[BaseModel]]: + discovered: dict[str, type[BaseModel]] = {} + for _name, candidate in inspect.getmembers(module, inspect.isclass): + if not issubclass(candidate, BaseModel): + continue + field = candidate.model_fields.get("schema_id") + if field is None or field.alias != "schema" or not isinstance(field.default, str): + continue + existing = discovered.get(field.default) + if existing is not None and existing is not candidate: + raise RuntimeError(f"duplicate schema model registration: {field.default}") + discovered[field.default] = candidate + return discovered + + +def _merge(*registries: dict[str, type[BaseModel]]) -> dict[str, type[BaseModel]]: + result: dict[str, type[BaseModel]] = {} + for registry in registries: + for schema, model in registry.items(): + existing = result.get(schema) + if existing is not None and existing is not model: + raise RuntimeError(f"conflicting schema model registration: {schema}") + result[schema] = model + return result + + +PHASE_5_SCHEMAS = _merge( + _models_from_module(passport_models), + _models_from_module(custody_models), + _models_from_module(attestation_models), + _models_from_module(trust_models), + _models_from_module(governance_models), + SECURITY_SCHEMA_MODELS, + cast(dict[str, type[BaseModel]], RUNTIME_SCHEMA_MODELS), + CONTINUOUS_TRUST_SCHEMAS, +) + +PHASE_SCHEMAS: dict[EvidencePhase, dict[str, type[BaseModel]]] = { + EvidencePhase.PHASE_5: PHASE_5_SCHEMAS, + EvidencePhase.PHASE_6A: PAYLOAD_SCHEMAS, + EvidencePhase.PHASE_6B: RECONCILIATION_SCHEMAS, + EvidencePhase.PHASE_6C: QUANTIZATION_SCHEMAS, + EvidencePhase.PHASE_6D: TOKENIZER_SCHEMAS, + EvidencePhase.PHASE_6E: RUNTIME_RESOLUTION_SCHEMAS, +} + + +def supported_schemas(phase: EvidencePhase | None = None) -> tuple[str, ...]: + if phase is not None: + return tuple(sorted(PHASE_SCHEMAS.get(phase, {}))) + return tuple(sorted({schema for registry in PHASE_SCHEMAS.values() for schema in registry})) + + +def schema_model(phase: EvidencePhase, schema: str) -> type[BaseModel] | None: + return PHASE_SCHEMAS.get(phase, {}).get(schema) diff --git a/src/omiv/assurance/signatures.py b/src/omiv/assurance/signatures.py new file mode 100644 index 0000000..08dff8b --- /dev/null +++ b/src/omiv/assurance/signatures.py @@ -0,0 +1,99 @@ +"""Detached Ed25519 signatures and trust-policy binding for Assurance Bundles.""" + +from __future__ import annotations + +import hashlib +from pathlib import Path + +from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey +from pydantic import ValidationError + +from omiv.assurance.models import AssuranceSignature, AssuranceTrustPolicy +from omiv.assurance.operations import MANIFEST_NAME, _pretty, load_manifest +from omiv.canonical import canonical_sha256 +from omiv.errors import OmivInputError +from omiv.hf.json_loader import load_bounded_json +from omiv.safe_write import atomic_write_text +from omiv.trust.algorithms import public_key_from_raw, raw_public_key, sign, verify + + +def build_signature(root: Path, private_key: Ed25519PrivateKey, key_id: str) -> AssuranceSignature: + manifest = load_manifest(root / MANIFEST_NAME) + public = raw_public_key(private_key.public_key()) + body = { + "schema": "omiv.assurance-signature.v1", + "algorithm": "ED25519", + "key_id": key_id, + "public_key": public.hex(), + "public_key_sha256": hashlib.sha256(public).hexdigest(), + "manifest_digest": manifest.bundle_digest, + "signature": sign(private_key, manifest.bundle_digest.encode("ascii")).hex(), + "limitations": [ + "Signature integrity does not establish signer authority without a trust policy." + ], + } + digest = canonical_sha256({"domain": body["schema"], "body": body}) + return AssuranceSignature.model_validate( + { + **body, + "signature_id": "assurance_signature_" + digest[:32], + "signature_digest": digest, + } + ) + + +def write_signature(root: Path, signature: AssuranceSignature) -> Path: + destination = root / "signatures" / f"{signature.key_id}.json" + atomic_write_text(destination, _pretty(signature)) + return destination + + +def load_policy(path: Path) -> AssuranceTrustPolicy: + value, _raw = load_bounded_json(path, max_bytes=1024 * 1024) + try: + return AssuranceTrustPolicy.model_validate(value) + except (ValidationError, ValueError) as exc: + raise OmivInputError(f"invalid Assurance Bundle trust policy: {exc}") from exc + + +def verify_signatures( + root: Path, policy: AssuranceTrustPolicy | None = None +) -> tuple[int, int, list[str]]: + manifest = load_manifest(root / MANIFEST_NAME) + directory = root / "signatures" + paths = [] if not directory.exists() else sorted(directory.glob("*.json")) + valid = trusted = 0 + issues: list[str] = [] + for path in paths: + try: + value, _raw = load_bounded_json(path, max_bytes=1024 * 1024) + record = AssuranceSignature.model_validate(value) + if record.manifest_digest != manifest.bundle_digest: + raise ValueError("signature manifest digest mismatch") + public_raw = bytes.fromhex(record.public_key) + if hashlib.sha256(public_raw).hexdigest() != record.public_key_sha256: + raise ValueError("signature public-key digest mismatch") + if not verify( + public_key_from_raw(public_raw), + bytes.fromhex(record.signature), + manifest.bundle_digest.encode("ascii"), + ): + raise ValueError("signature verification failed") + valid += 1 + if policy is not None and ( + (not policy.allowed_key_ids or record.key_id in policy.allowed_key_ids) + and ( + not policy.allowed_public_key_sha256 + or record.public_key_sha256 in policy.allowed_public_key_sha256 + ) + ): + trusted += 1 + except (OSError, ValidationError, ValueError, OmivInputError) as exc: + issues.append(f"{path.name}:{exc}") + if ( + policy is not None + and policy.require_signature + and trusted < policy.minimum_valid_signatures + ): + issues.append("TRUST_POLICY_MINIMUM_SIGNATURES_NOT_MET") + return valid, trusted, issues diff --git a/src/omiv/cli.py b/src/omiv/cli.py index e497d27..ecc4640 100644 --- a/src/omiv/cli.py +++ b/src/omiv/cli.py @@ -1,6 +1,7 @@ """Command-line interface.""" import json +import tempfile from collections.abc import Callable from pathlib import Path from typing import Annotated, cast @@ -26,6 +27,50 @@ from omiv.article.verification import ( pretty_json as pretty_article_json, ) +from omiv.assurance.archive import extract_archive +from omiv.assurance.archive import pack_bundle as pack_assurance_bundle +from omiv.assurance.models import ( + AssuranceTrustPolicy, + BundleStatus, + PreflightStatus, +) +from omiv.assurance.models import ( + VerificationFinding as AssuranceVerificationFinding, +) +from omiv.assurance.operations import ( + build_bundle as build_assurance_bundle, +) +from omiv.assurance.operations import ( + build_preflight as build_assurance_preflight, +) +from omiv.assurance.operations import ( + concise_bundle_summary, + concise_plan_summary, + concise_verification_summary, +) +from omiv.assurance.operations import ( + load_plan as load_assurance_plan, +) +from omiv.assurance.operations import ( + load_request as load_assurance_request, +) +from omiv.assurance.operations import ( + verify_bundle as verify_assurance_bundle, +) +from omiv.assurance.operations import ( + write_plan as write_assurance_plan, +) +from omiv.assurance.operations import ( + write_verification_report as write_assurance_verification_report, +) +from omiv.assurance.signatures import ( + build_signature as build_assurance_signature, +) +from omiv.assurance.signatures import ( + load_policy as load_assurance_trust_policy, +) +from omiv.assurance.signatures import verify_signatures as verify_assurance_signatures +from omiv.assurance.signatures import write_signature as write_assurance_signature from omiv.attestations.builder import build_attestation from omiv.attestations.custody import append_attestation_to_ledger from omiv.attestations.models import ( @@ -278,6 +323,27 @@ from omiv.reconciliation.reporting import pretty_json as pretty_reconciliation_json from omiv.reconciliation.schema import load_any_reconciliation, load_reconciliation from omiv.reconciliation_profiles.huggingface import collect_huggingface_metadata +from omiv.reference_preflight.operations import ( + build_assurance_request as build_reference_assurance_request, +) +from omiv.reference_preflight.operations import ( + build_reference_preflight, +) +from omiv.reference_preflight.operations import ( + concise_evidence_summary as concise_reference_preflight_evidence, +) +from omiv.reference_preflight.operations import ( + load_evidence as load_reference_preflight_evidence, +) +from omiv.reference_preflight.operations import ( + load_profile as load_reference_preflight_profile, +) +from omiv.reference_preflight.operations import ( + write_assurance_request as write_reference_assurance_request, +) +from omiv.reference_preflight.operations import ( + write_evidence as write_reference_preflight_evidence, +) from omiv.remote.gguf_header import RemoteGGUFHeaderParser from omiv.remote.header_models import ( HEADER_REPORT_SCHEMA, @@ -364,6 +430,37 @@ from omiv.runtime.reporting import ( pretty_json as pretty_runtime_json, ) +from omiv.runtime_compatibility.models import ( + CompatibilityStatus as RuntimeCompatibilityStatus, +) +from omiv.runtime_compatibility.models import PlanStatus as RuntimeCompatibilityPlanStatus +from omiv.runtime_compatibility.operations import ( + build_plan as build_runtime_compatibility_plan, +) +from omiv.runtime_compatibility.operations import ( + concise_evidence_summary as concise_runtime_compatibility_evidence, +) +from omiv.runtime_compatibility.operations import ( + concise_plan_summary as concise_runtime_compatibility_plan, +) +from omiv.runtime_compatibility.operations import ( + execute_plan as execute_runtime_compatibility_plan, +) +from omiv.runtime_compatibility.operations import ( + load_evidence as load_runtime_compatibility_evidence, +) +from omiv.runtime_compatibility.operations import ( + load_plan as load_runtime_compatibility_plan, +) +from omiv.runtime_compatibility.operations import ( + load_request as load_runtime_compatibility_request, +) +from omiv.runtime_compatibility.operations import ( + write_evidence as write_runtime_compatibility_evidence, +) +from omiv.runtime_compatibility.operations import ( + write_plan as write_runtime_compatibility_plan, +) from omiv.runtime_resolution.artifact_index import verify_runtime_resolution_artifact_index from omiv.runtime_resolution.models import ( EvidenceStatus as RuntimeResolutionEvidenceStatus, @@ -417,6 +514,20 @@ pretty_json as pretty_security_json, ) from omiv.security.scanning import describe_local_artifact, inspect_local_artifact +from omiv.smart_preflight.models import SmartPreflightStatus +from omiv.smart_preflight.operations import build_smart_preflight +from omiv.smart_preflight.operations import ( + concise_plan_summary as concise_smart_preflight_summary, +) +from omiv.smart_preflight.operations import ( + load_intent as load_smart_preflight_intent, +) +from omiv.smart_preflight.operations import ( + write_assurance_request as write_smart_assurance_request, +) +from omiv.smart_preflight.operations import ( + write_plan as write_smart_preflight_plan, +) from omiv.tokenizer_parity.artifact_index import ( verify_tokenizer_configuration_artifact_index, ) @@ -504,6 +615,10 @@ quantization_app = typer.Typer(no_args_is_help=True) tokenizer_configuration_app = typer.Typer(no_args_is_help=True) runtime_resolution_app = typer.Typer(no_args_is_help=True) +assurance_app = typer.Typer(no_args_is_help=True) +smart_preflight_app = typer.Typer(no_args_is_help=True) +runtime_compatibility_app = typer.Typer(no_args_is_help=True) +reference_preflight_app = typer.Typer(no_args_is_help=True) def _version_callback(value: bool) -> None: @@ -536,6 +651,10 @@ def main( app.add_typer(quantization_app, name="quantization") app.add_typer(tokenizer_configuration_app, name="tokenizer-config") app.add_typer(runtime_resolution_app, name="runtime-resolution") +app.add_typer(assurance_app, name="assurance") +app.add_typer(smart_preflight_app, name="smart-preflight") +app.add_typer(runtime_compatibility_app, name="runtime-compat") +app.add_typer(reference_preflight_app, name="reference-preflight") MAX_CANONICAL_INVENTORY_BYTES = 64 * 1024 * 1024 REMOTE_REPORT_SCHEMAS = { "omiv.remote-snapshot-report.v1", @@ -4626,3 +4745,307 @@ def runtime_resolution_practice_anthropic() -> None: except (OSError, UnicodeError, ValidationError, ValueError, OmivInputError) as exc: _runtime_resolution_failure(exc) raise typer.Exit(code=1) + + +def _assurance_failure(exc: Exception) -> None: + typer.echo(f"ERROR Assurance Bundle operation failed: {exc}", err=True) + raise typer.Exit(code=2) from exc + + +def _runtime_compatibility_failure(exc: Exception) -> None: + typer.echo(f"ERROR Runtime compatibility operation failed: {exc}", err=True) + raise typer.Exit(code=2) from exc + + +def _reference_preflight_failure(exc: Exception) -> None: + typer.echo(f"ERROR Reference Preflight failed: {exc}", err=True) + raise typer.Exit(code=2) from exc + + +@reference_preflight_app.command("plan") +def reference_preflight_plan( + profile_path: Annotated[Path, typer.Option("--profile", exists=True, dir_okay=False)], + reference: Annotated[str, typer.Option("--reference")], + output: Annotated[Path, typer.Option("--output", dir_okay=False)], + assurance_request_output: Annotated[ + Path | None, typer.Option("--assurance-request-output", dir_okay=False) + ] = None, + root: Annotated[Path, typer.Option("--root", exists=True, file_okay=False)] = Path("."), +) -> None: + """Replay pinned provider metadata and emit no-payload, evidence-qualified preflight.""" + try: + profile = load_reference_preflight_profile(profile_path) + evidence = build_reference_preflight(profile, reference) + validate_output_path(output, forbidden_inputs=(profile_path,)) + request = None + if assurance_request_output is not None: + validate_output_path( + assurance_request_output, + forbidden_inputs=(profile_path, output), + ) + try: + source_path = output.resolve(strict=False).relative_to(root.resolve()).as_posix() + except ValueError as exc: + raise OmivInputError( + "reference evidence output must be inside --root for Assurance handoff" + ) from exc + request = build_reference_assurance_request(evidence, source_path) + write_reference_preflight_evidence(evidence, output) + if request is not None and assurance_request_output is not None: + write_reference_assurance_request(request, assurance_request_output) + except (OSError, UnicodeError, ValidationError, ValueError, OmivInputError) as exc: + _reference_preflight_failure(exc) + typer.echo(concise_reference_preflight_evidence(evidence)) + + +@reference_preflight_app.command("verify") +def reference_preflight_verify( + evidence_path: Annotated[Path, typer.Option("--evidence", exists=True, dir_okay=False)], +) -> None: + """Verify canonical reference evidence and its complete future plan offline.""" + try: + evidence = load_reference_preflight_evidence(evidence_path) + except (OSError, UnicodeError, ValidationError, ValueError, OmivInputError) as exc: + _reference_preflight_failure(exc) + typer.echo(concise_reference_preflight_evidence(evidence)) + + +@runtime_compatibility_app.command("plan") +def runtime_compatibility_plan( + request_path: Annotated[Path, typer.Option("--request", exists=True, dir_okay=False)], + output: Annotated[Path, typer.Option("--output", dir_okay=False)], + root: Annotated[Path, typer.Option("--root", exists=True, file_okay=False)] = Path("."), +) -> None: + """Preflight one explicit local runtime and artifact without executing either.""" + try: + request = load_runtime_compatibility_request(request_path) + plan = build_runtime_compatibility_plan(request, root) + validate_output_path( + output, + forbidden_inputs=( + request_path, + root / Path(*request.executable_path.split("/")), + root / Path(*request.artifact_path.split("/")), + ), + ) + write_runtime_compatibility_plan(plan, output) + except (OSError, UnicodeError, ValidationError, ValueError, OmivInputError) as exc: + _runtime_compatibility_failure(exc) + typer.echo(concise_runtime_compatibility_plan(plan)) + if plan.status != RuntimeCompatibilityPlanStatus.READY: + raise typer.Exit(code=1) + + +@runtime_compatibility_app.command("run") +def runtime_compatibility_run( + plan_path: Annotated[Path, typer.Option("--plan", exists=True, dir_okay=False)], + output: Annotated[Path, typer.Option("--output", dir_okay=False)], + root: Annotated[Path, typer.Option("--root", exists=True, file_okay=False)] = Path("."), +) -> None: + """Execute one preflighted native profile and write bounded raw-observation evidence.""" + try: + plan = load_runtime_compatibility_plan(plan_path) + validate_output_path( + output, + forbidden_inputs=( + plan_path, + root / Path(*plan.request.executable_path.split("/")), + root / Path(*plan.request.artifact_path.split("/")), + ), + ) + evidence = execute_runtime_compatibility_plan(plan, root) + write_runtime_compatibility_evidence(evidence, output) + except (OSError, UnicodeError, ValidationError, ValueError, OmivInputError) as exc: + _runtime_compatibility_failure(exc) + typer.echo(concise_runtime_compatibility_evidence(evidence)) + if evidence.status != RuntimeCompatibilityStatus.VERIFIED_WITHIN_PROFILE: + raise typer.Exit(code=1) + + +@runtime_compatibility_app.command("verify") +def runtime_compatibility_verify( + evidence_path: Annotated[Path, typer.Option("--evidence", exists=True, dir_okay=False)], +) -> None: + """Verify candidate evidence, its embedded plan, and fail-closed status offline.""" + try: + evidence = load_runtime_compatibility_evidence(evidence_path) + except (OSError, UnicodeError, ValidationError, ValueError, OmivInputError) as exc: + _runtime_compatibility_failure(exc) + typer.echo(concise_runtime_compatibility_evidence(evidence)) + if evidence.status != RuntimeCompatibilityStatus.VERIFIED_WITHIN_PROFILE: + raise typer.Exit(code=1) + + +@smart_preflight_app.command("plan") +def smart_preflight_plan( + intent_path: Annotated[Path, typer.Option("--intent", exists=True, dir_okay=False)], + output: Annotated[Path, typer.Option("--output", dir_okay=False)], + assurance_request_output: Annotated[ + Path | None, typer.Option("--assurance-request-output", dir_okay=False) + ] = None, + root: Annotated[Path, typer.Option("--root", exists=True, file_okay=False)] = Path("."), +) -> None: + """Discover local canonical evidence and propose a Phase 6F request without costly work.""" + try: + intent = load_smart_preflight_intent(intent_path) + plan = build_smart_preflight(intent, root) + evidence_inputs = [root / Path(*item.source_path.split("/")) for item in plan.candidates] + validate_output_path(output, forbidden_inputs=(intent_path, *evidence_inputs)) + if assurance_request_output is not None: + validate_output_path( + assurance_request_output, + forbidden_inputs=(intent_path, output, *evidence_inputs), + ) + write_smart_preflight_plan(plan, output) + if assurance_request_output is not None and plan.assurance_request is not None: + write_smart_assurance_request(plan.assurance_request, assurance_request_output) + except (OSError, UnicodeError, ValidationError, ValueError, OmivInputError) as exc: + typer.echo(f"ERROR Smart Preflight failed: {exc}", err=True) + raise typer.Exit(code=2) from exc + typer.echo(concise_smart_preflight_summary(plan)) + if plan.status != SmartPreflightStatus.READY: + raise typer.Exit(code=1) + + +@assurance_app.command("plan") +def assurance_plan( + request_path: Annotated[Path, typer.Option("--request", exists=True, dir_okay=False)], + output: Annotated[Path, typer.Option("--output", dir_okay=False)], + root: Annotated[Path, typer.Option("--root", exists=True, file_okay=False)] = Path("."), +) -> None: + """Run a local-only preflight before any download, network, conversion, or GPU work.""" + try: + request = load_assurance_request(request_path) + plan = build_assurance_preflight(request, root) + write_assurance_plan(plan, output) + except (OSError, UnicodeError, ValidationError, ValueError, OmivInputError) as exc: + _assurance_failure(exc) + typer.echo(concise_plan_summary(plan)) + if plan.status not in {PreflightStatus.READY, PreflightStatus.READY_WITH_GAPS}: + raise typer.Exit(code=1) + + +@assurance_app.command("build") +def assurance_build( + plan_path: Annotated[Path, typer.Option("--plan", exists=True, dir_okay=False)], + output: Annotated[Path, typer.Option("--output", file_okay=False)], + root: Annotated[Path, typer.Option("--root", exists=True, file_okay=False)] = Path("."), +) -> None: + """Assemble a portable bundle from preflighted local evidence; never run costly work.""" + try: + plan = load_assurance_plan(plan_path) + manifest = build_assurance_bundle(plan, root, output) + except (OSError, UnicodeError, ValidationError, ValueError, OmivInputError) as exc: + _assurance_failure(exc) + typer.echo(concise_bundle_summary(manifest)) + if manifest.status != BundleStatus.COMPLETE: + raise typer.Exit(code=1) + + +@assurance_app.command("verify") +def assurance_verify( + bundle: Annotated[Path, typer.Option("--bundle", exists=True)], + report_output: Annotated[Path | None, typer.Option("--report-output", dir_okay=False)] = None, + trust_policy: Annotated[ + Path | None, typer.Option("--trust-policy", exists=True, dir_okay=False) + ] = None, +) -> None: + """Verify an Assurance Bundle completely offline and fail closed on every gap.""" + try: + policy: AssuranceTrustPolicy | None = ( + load_assurance_trust_policy(trust_policy) if trust_policy is not None else None + ) + if bundle.is_dir(): + root = bundle + report = verify_assurance_bundle(root) + transport = "DIRECTORY" + valid, trusted, signature_issues = verify_assurance_signatures(root, policy) + else: + with tempfile.TemporaryDirectory(prefix="omiv-assurance-verify-") as temporary: + root = Path(temporary) + extract_archive(bundle, root) + report = verify_assurance_bundle(root) + transport = "ZIP_STORED" + valid, trusted, signature_issues = verify_assurance_signatures(root, policy) + signature_status = ( + "INVALID" + if signature_issues + else "TRUSTED" + if policy is not None and trusted >= policy.minimum_valid_signatures + else "VALID_UNTRUSTED" + if valid + else "NOT_PRESENT" + ) + report = report.model_copy( + update={ + "transport": transport, + "valid_signatures": valid, + "trusted_signatures": trusted, + "signature_status": signature_status, + "status": BundleStatus.INVALID if signature_issues else report.status, + "invalid": report.invalid + len(signature_issues), + "findings": [ + *report.findings, + *[ + AssuranceVerificationFinding( + code="SIGNATURE_OR_TRUST_FAILURE", detail=issue + ) + for issue in signature_issues + ], + ], + } + ) + if report_output is not None: + write_assurance_verification_report(report, report_output) + except (OSError, UnicodeError, ValidationError, ValueError, OmivInputError) as exc: + _assurance_failure(exc) + typer.echo(concise_verification_summary(report)) + if report.status != BundleStatus.COMPLETE: + raise typer.Exit(code=1) + + +@assurance_app.command("pack") +def assurance_pack( + bundle: Annotated[Path, typer.Option("--bundle", exists=True, file_okay=False)], + output: Annotated[Path, typer.Option("--output", dir_okay=False)], +) -> None: + """Create a deterministic ZIP_STORED .omiv transport from a verified directory.""" + try: + report = verify_assurance_bundle(bundle) + if report.status == BundleStatus.INVALID: + raise OmivInputError("invalid Assurance Bundle cannot be packed") + _valid, _trusted, signature_issues = verify_assurance_signatures(bundle) + if signature_issues: + raise OmivInputError("invalid Assurance Bundle signatures cannot be packed") + pack_assurance_bundle(bundle, output) + except (OSError, UnicodeError, ValidationError, ValueError, OmivInputError) as exc: + _assurance_failure(exc) + typer.echo(f"PACKED transport=ZIP_STORED output={output.name}") + + +@assurance_app.command("sign") +def assurance_sign( + bundle: Annotated[Path, typer.Option("--bundle", exists=True, file_okay=False)], + private_key: Annotated[Path, typer.Option("--private-key", exists=True, dir_okay=False)], + key_id: Annotated[str, typer.Option("--key-id")], +) -> None: + """Attach an Ed25519 manifest signature; authority still requires a trust policy.""" + try: + key = load_private_key(private_key) + signature = build_assurance_signature(bundle, key, key_id) + destination = write_assurance_signature(bundle, signature) + except (OSError, UnicodeError, ValidationError, ValueError, OmivInputError) as exc: + _assurance_failure(exc) + typer.echo(f"SIGNED key={key_id} signature={destination.name}") + + +@app.command("verify") +def verify_portable_assurance_bundle( + bundle: Annotated[Path, typer.Argument(exists=True)], + report_output: Annotated[Path | None, typer.Option("--report-output", dir_okay=False)] = None, + trust_policy: Annotated[ + Path | None, typer.Option("--trust-policy", exists=True, dir_okay=False) + ] = None, +) -> None: + """Verify a portable Assurance Bundle offline using the product-level command.""" + assurance_verify(bundle=bundle, report_output=report_output, trust_policy=trust_policy) diff --git a/src/omiv/py.typed b/src/omiv/py.typed new file mode 100644 index 0000000..e69de29 diff --git a/src/omiv/reference_preflight/__init__.py b/src/omiv/reference_preflight/__init__.py new file mode 100644 index 0000000..71d278b --- /dev/null +++ b/src/omiv/reference_preflight/__init__.py @@ -0,0 +1 @@ +"""Provider-neutral, metadata-only reference preflight candidate.""" diff --git a/src/omiv/reference_preflight/models.py b/src/omiv/reference_preflight/models.py new file mode 100644 index 0000000..493a46d --- /dev/null +++ b/src/omiv/reference_preflight/models.py @@ -0,0 +1,360 @@ +"""Strict models for provider-neutral, metadata-only reference preflight evidence.""" + +from __future__ import annotations + +from enum import StrEnum +from typing import Literal + +from pydantic import Field, JsonValue, model_validator + +from omiv.canonical import canonical_sha256 +from omiv.models import StrictModel +from omiv.payload_integrity.paths import validate_portable_path + + +class ObservationAuthority(StrEnum): + PROVIDER_API = "PROVIDER_API" + PINNED_PROVIDER_DOCUMENT = "PINNED_PROVIDER_DOCUMENT" + UPSTREAM_VCS = "UPSTREAM_VCS" + UPSTREAM_RELEASE = "UPSTREAM_RELEASE" + REGISTRY_MANIFEST = "REGISTRY_MANIFEST" + LOCAL_DOCUMENT_FETCH = "LOCAL_DOCUMENT_FETCH" + + +class ArtifactRole(StrEnum): + MAIN_MODEL = "MAIN_MODEL" + PERCEPTION_ENCODER = "PERCEPTION_ENCODER" + DRAFTER = "DRAFTER" + + +class RelationshipKind(StrEnum): + QUANTIZED_FROM = "QUANTIZED_FROM" + COMPANION_OF = "COMPANION_OF" + + +class SourceObservation(StrictModel): + source_id: str = Field(min_length=3, max_length=96, pattern=r"^[A-Za-z0-9._-]+$") + provider: str = Field(min_length=1, max_length=100) + authority: ObservationAuthority + url: str = Field(min_length=8, max_length=2000) + resolved_identity: str = Field(min_length=1, max_length=512) + observed_on: str = Field(pattern=r"^\d{4}-\d{2}-\d{2}$") + raw_fields: dict[str, JsonValue] = Field(default_factory=dict, max_length=64) + limitations: list[str] = Field(default_factory=list, max_length=16) + + +class ArtifactObservation(StrictModel): + artifact_id: str = Field(min_length=3, max_length=96, pattern=r"^[A-Za-z0-9._-]+$") + repository: str = Field(min_length=1, max_length=512) + revision: str = Field(min_length=1, max_length=512) + path: str = Field(min_length=1, max_length=1000) + role: ArtifactRole + required: bool + declared_size: int = Field(ge=0) + provider_identity_kind: str = Field(min_length=1, max_length=100) + provider_identity: str = Field(min_length=1, max_length=512) + authority_source_id: str = Field(min_length=3, max_length=96) + availability: Literal["REMOTE_ONLY"] = "REMOTE_ONLY" + payload_verification: Literal["NOT_DOWNLOADED"] = "NOT_DOWNLOADED" + + +class DeclaredRelationship(StrictModel): + subject_artifact_id: str = Field(min_length=3, max_length=96) + object_reference: str = Field(min_length=1, max_length=512) + relationship: RelationshipKind + authority_source_id: str = Field(min_length=3, max_length=96) + status: Literal["PROVIDER_DECLARED"] = "PROVIDER_DECLARED" + cryptographic_binding: Literal["NOT_ESTABLISHED"] = "NOT_ESTABLISHED" + + +class ConfigurationObservation(StrictModel): + architecture: str = Field(min_length=1, max_length=200) + architecture_authority_source_id: str = Field(min_length=3, max_length=96) + config_identity: str = Field(min_length=1, max_length=512) + tokenizer_identity: str = Field(min_length=1, max_length=512) + chat_template_identity: str = Field(min_length=1, max_length=512) + context_length: int = Field(ge=1) + bos_token_id: int = Field(ge=0) + eos_token_ids: list[int] = Field(min_length=1, max_length=16) + pad_token_id: int = Field(ge=0) + stop_tokens: list[str] = Field(min_length=1, max_length=16) + input_modalities: list[str] = Field(min_length=1, max_length=16) + output_modalities: list[str] = Field(min_length=1, max_length=16) + + @model_validator(mode="after") + def validate_unique_values(self) -> ConfigurationObservation: + if len(set(self.eos_token_ids)) != len(self.eos_token_ids): + raise ValueError("EOS token identifiers must be unique") + if len(set(self.stop_tokens)) != len(self.stop_tokens): + raise ValueError("stop tokens must be unique") + return self + + +class RuntimeRequirement(StrictModel): + runtime: str = Field(min_length=1, max_length=100) + requirement: str = Field(min_length=1, max_length=500) + exact_release: str | None = Field(default=None, max_length=200) + exact_commit: str | None = Field(default=None, pattern=r"^[0-9a-f]{40}$") + support_commit: str | None = Field(default=None, pattern=r"^[0-9a-f]{40}$") + authority_source_ids: list[str] = Field(min_length=1, max_length=16) + compatibility_status: Literal["DECLARED_NOT_PROBED"] = "DECLARED_NOT_PROBED" + + +class RegistryLayer(StrictModel): + role: str = Field(min_length=1, max_length=100) + digest: str = Field(pattern=r"^sha256:[0-9a-f]{64}$") + declared_size: int = Field(ge=0) + + +class RegistryObservation(StrictModel): + runtime: str = Field(min_length=1, max_length=100) + tag: str = Field(min_length=1, max_length=512) + mutable_tag: Literal[True] = True + manifest_digest: str = Field(pattern=r"^[0-9a-f]{64}$") + authority_source_id: str = Field(min_length=3, max_length=96) + layers: list[RegistryLayer] = Field(min_length=1, max_length=32) + locally_resolved: Literal[False] = False + runtime_probe: Literal["NOT_RUN"] = "NOT_RUN" + + +class FutureArtifactPin(StrictModel): + artifact_id: str = Field(min_length=3, max_length=96) + repository: str = Field(min_length=1, max_length=512) + revision: str = Field(min_length=1, max_length=512) + path: str = Field(min_length=1, max_length=1000) + role: ArtifactRole + provider_identity_kind: str = Field(min_length=1, max_length=100) + provider_identity: str = Field(min_length=1, max_length=512) + declared_size: int = Field(ge=0) + downloaded: Literal[False] = False + + +class FutureRuntimePlanBody(StrictModel): + question: str = Field(min_length=1, max_length=1000) + artifacts: list[FutureArtifactPin] = Field(min_length=1, max_length=16) + runtime_requirements: list[RuntimeRequirement] = Field(min_length=1, max_length=16) + preferred_gpu_class: str = Field(min_length=1, max_length=200) + maximum_gpu_count: Literal[1] + maximum_pod_count: Literal[1] + maximum_total_authorized_compute_usd: int = Field(ge=1) + maximum_working_duration_seconds: int = Field(ge=1) + stop_new_work_before_deadline_seconds: int = Field(ge=1) + artifact_payload_bytes: int = Field(ge=1) + direct_llama_cpp_workspace_bytes: int = Field(ge=1) + llama_cpp_plus_ollama_workspace_bytes: int = Field(ge=1) + text_probe: str = Field(min_length=1, max_length=4000) + image_fixture: str + image_fixture_sha256: str = Field(pattern=r"^[0-9a-f]{64}$") + image_probe: str = Field(min_length=1, max_length=4000) + dflash_observations: list[Literal["DISABLED", "ENABLED"]] = Field(min_length=2, max_length=2) + ollama_role: Literal["SECONDARY_RUNTIME_OBSERVATION"] + ollama_mutable_tags: list[str] = Field(min_length=1, max_length=16) + direct_llama_cpp_independent_of_ollama: Literal[True] + ollama_observation: str = Field(min_length=1, max_length=1000) + evidence_to_retain: list[str] = Field(min_length=1, max_length=32) + stop_conditions: list[str] = Field(min_length=1, max_length=32) + pod_termination_requirement: str = Field(min_length=1, max_length=1000) + unknown_stages: list[Literal["LOAD", "TOKENIZER", "PREFILL", "DECODE", "OUTPUT"]] = Field( + min_length=5, max_length=5 + ) + non_claims: list[str] = Field(min_length=1, max_length=32) + execution_status: Literal["NOT_RUN"] = "NOT_RUN" + + @model_validator(mode="after") + def validate_plan_scope(self) -> FutureRuntimePlanBody: + validate_portable_path(self.image_fixture) + if not self.image_fixture.lower().endswith(".png"): + raise ValueError("future runtime image fixture must be a raster PNG") + if self.dflash_observations != ["DISABLED", "ENABLED"]: + raise ValueError("future plan must observe DFlash disabled before enabled") + if self.unknown_stages != ["LOAD", "TOKENIZER", "PREFILL", "DECODE", "OUTPUT"]: + raise ValueError("future plan must retain every runtime stage as unknown") + if len({item.artifact_id for item in self.artifacts}) != len(self.artifacts): + raise ValueError("future artifact pins must be unique") + if sum(item.declared_size for item in self.artifacts) != self.artifact_payload_bytes: + raise ValueError("artifact payload total must equal the sum of planned artifact sizes") + if self.direct_llama_cpp_workspace_bytes < 50_000_000_000: + raise ValueError("direct llama.cpp workspace recommendation must be at least 50 GB") + if self.llama_cpp_plus_ollama_workspace_bytes < 80_000_000_000: + raise ValueError("combined llama.cpp and Ollama workspace must be at least 80 GB") + if ( + self.llama_cpp_plus_ollama_workspace_bytes + < self.direct_llama_cpp_workspace_bytes + ): + raise ValueError("combined workspace recommendation cannot be smaller than direct") + if self.stop_new_work_before_deadline_seconds >= self.maximum_working_duration_seconds: + raise ValueError("new-work cutoff must precede the working deadline") + if len(set(self.ollama_mutable_tags)) != len(self.ollama_mutable_tags): + raise ValueError("Ollama mutable tags must be unique") + return self + + +class FutureRuntimePlan(FutureRuntimePlanBody): + schema_id: Literal["omiv.reference-runtime-plan.v1"] = Field( + default="omiv.reference-runtime-plan.v1", alias="schema" + ) + plan_id: str + plan_digest: str = Field(pattern=r"^[0-9a-f]{64}$") + + @model_validator(mode="after") + def validate_identity(self) -> FutureRuntimePlan: + body = self.model_dump(mode="json", by_alias=True, exclude={"plan_id", "plan_digest"}) + digest = canonical_sha256({"domain": self.schema_id, "body": body}) + if self.plan_digest != digest or self.plan_id != f"reference_runtime_plan_{digest[:32]}": + raise ValueError("future runtime plan canonical identity mismatch") + return self + + +class ReferencePreflightProfile(StrictModel): + schema_id: Literal["omiv.reference-preflight-profile.v1"] = Field( + default="omiv.reference-preflight-profile.v1", alias="schema" + ) + profile_name: str = Field(min_length=3, max_length=200) + canonical_reference: str = Field(min_length=1, max_length=512) + reference_aliases: list[str] = Field(min_length=1, max_length=32) + resolved_revision: str = Field(min_length=1, max_length=512) + sources: list[SourceObservation] = Field(min_length=1, max_length=64) + artifacts: list[ArtifactObservation] = Field(min_length=1, max_length=64) + relationships: list[DeclaredRelationship] = Field(max_length=64) + configuration: ConfigurationObservation + runtime_candidates: list[RuntimeRequirement] = Field(min_length=1, max_length=16) + registry_observations: list[RegistryObservation] = Field(max_length=16) + provenance_gaps: list[str] = Field(min_length=1, max_length=32) + limitations: list[str] = Field(min_length=1, max_length=32) + future_runtime_plan: FutureRuntimePlanBody + + @model_validator(mode="after") + def validate_references(self) -> ReferencePreflightProfile: + if self.canonical_reference not in self.reference_aliases: + raise ValueError("canonical reference must be an accepted alias") + if len(set(self.reference_aliases)) != len(self.reference_aliases): + raise ValueError("reference aliases must be unique") + sources = {item.source_id for item in self.sources} + if len(sources) != len(self.sources): + raise ValueError("source identifiers must be unique") + artifacts = {item.artifact_id for item in self.artifacts} + if len(artifacts) != len(self.artifacts): + raise ValueError("artifact identifiers must be unique") + for artifact in self.artifacts: + if artifact.authority_source_id not in sources: + raise ValueError("artifact authority source is missing") + for relationship in self.relationships: + if ( + relationship.subject_artifact_id not in artifacts + or relationship.authority_source_id not in sources + ): + raise ValueError("declared relationship references missing evidence") + for runtime in self.runtime_candidates: + if not set(runtime.authority_source_ids).issubset(sources): + raise ValueError("runtime requirement authority source is missing") + for registry in self.registry_observations: + if registry.authority_source_id not in sources: + raise ValueError("registry authority source is missing") + if self.configuration.architecture_authority_source_id not in sources: + raise ValueError("architecture authority source is missing") + planned_artifacts = { + item.artifact_id: item for item in self.future_runtime_plan.artifacts + } + if set(planned_artifacts) != artifacts: + raise ValueError("future runtime plan must pin every observed artifact exactly once") + for artifact in self.artifacts: + planned = planned_artifacts[artifact.artifact_id] + observed_pin = ( + artifact.repository, + artifact.revision, + artifact.path, + artifact.role, + artifact.provider_identity_kind, + artifact.provider_identity, + artifact.declared_size, + ) + planned_pin = ( + planned.repository, + planned.revision, + planned.path, + planned.role, + planned.provider_identity_kind, + planned.provider_identity, + planned.declared_size, + ) + if planned_pin != observed_pin: + raise ValueError("future runtime artifact pin differs from observed metadata") + if self.future_runtime_plan.runtime_requirements != self.runtime_candidates: + raise ValueError("future runtime requirements differ from observed declarations") + return self + + +class ReferencePreflightEvidence(StrictModel): + schema_id: Literal["omiv.reference-preflight-evidence.v1"] = Field( + default="omiv.reference-preflight-evidence.v1", alias="schema" + ) + evidence_id: str + evidence_digest: str = Field(pattern=r"^[0-9a-f]{64}$") + candidate_notice: Literal["P7_REFERENCE_PREFLIGHT_CANDIDATE"] = ( + "P7_REFERENCE_PREFLIGHT_CANDIDATE" + ) + input_reference: str = Field(min_length=1, max_length=2000) + artifact_reference: str = Field(min_length=1, max_length=512) + resolved_identity_status: Literal["REMOTE_REVISION_PINNED"] = "REMOTE_REVISION_PINNED" + resolved_identity: str = Field(min_length=1, max_length=1200) + payload_verification: Literal["NOT_DOWNLOADED"] = "NOT_DOWNLOADED" + source_binding: Literal["NOT_ESTABLISHED"] = "NOT_ESTABLISHED" + architecture_status: Literal["DECLARED"] = "DECLARED" + architecture: str = Field(min_length=1, max_length=200) + tokenizer_configuration: Literal["REMOTE_METADATA_OBSERVED"] = "REMOTE_METADATA_OBSERVED" + companion_roles: list[ArtifactRole] = Field(max_length=16) + runtime_candidates: list[str] = Field(min_length=1, max_length=16) + runtime_requirement: Literal["DECLARED_OFFICIAL_REQUIREMENT"] = "DECLARED_OFFICIAL_REQUIREMENT" + runtime_probe: Literal["NOT_RUN"] = "NOT_RUN" + numerical_fidelity: Literal["NOT_EVALUATED"] = "NOT_EVALUATED" + semantic_fidelity: Literal["NOT_EVALUATED"] = "NOT_EVALUATED" + performance: Literal["NOT_EVALUATED"] = "NOT_EVALUATED" + safety: Literal["NOT_EVALUATED"] = "NOT_EVALUATED" + production_readiness: Literal["NOT_ESTABLISHED"] = "NOT_ESTABLISHED" + provenance_gaps: list[str] = Field(min_length=1, max_length=32) + profile_digest: str = Field(pattern=r"^[0-9a-f]{64}$") + profile: ReferencePreflightProfile + future_runtime_plan: FutureRuntimePlan + limitations: list[str] = Field(min_length=1, max_length=32) + + @model_validator(mode="after") + def validate_identity_and_projection(self) -> ReferencePreflightEvidence: + body = self.model_dump( + mode="json", by_alias=True, exclude={"evidence_id", "evidence_digest"} + ) + digest = canonical_sha256({"domain": self.schema_id, "body": body}) + if self.evidence_digest != digest or self.evidence_id != ( + f"reference_preflight_evidence_{digest[:32]}" + ): + raise ValueError("reference preflight evidence canonical identity mismatch") + expected_profile_digest = canonical_sha256( + {"reference-preflight-profile": self.profile.model_dump(mode="json", by_alias=True)} + ) + if self.profile_digest != expected_profile_digest: + raise ValueError("reference preflight profile digest mismatch") + if ( + self.artifact_reference != self.profile.canonical_reference + or self.resolved_identity + != (f"{self.profile.canonical_reference}@{self.profile.resolved_revision}") + ): + raise ValueError("reference and resolved identity projection mismatch") + expected_companions = sorted( + {item.role for item in self.profile.artifacts if item.role != ArtifactRole.MAIN_MODEL}, + key=lambda item: item.value, + ) + if self.companion_roles != expected_companions: + raise ValueError("companion-role projection mismatch") + if self.runtime_candidates != [item.runtime for item in self.profile.runtime_candidates]: + raise ValueError("runtime-candidate projection mismatch") + if self.architecture != self.profile.configuration.architecture: + raise ValueError("architecture projection mismatch") + if self.provenance_gaps != self.profile.provenance_gaps: + raise ValueError("provenance-gap projection mismatch") + if self.limitations != self.profile.limitations: + raise ValueError("limitation projection mismatch") + plan_body = self.future_runtime_plan.model_dump( + mode="json", by_alias=True, exclude={"schema_id", "plan_id", "plan_digest"} + ) + if plan_body != self.profile.future_runtime_plan.model_dump(mode="json"): + raise ValueError("future runtime plan differs from the bound profile plan") + return self diff --git a/src/omiv/reference_preflight/operations.py b/src/omiv/reference_preflight/operations.py new file mode 100644 index 0000000..bd1c5e0 --- /dev/null +++ b/src/omiv/reference_preflight/operations.py @@ -0,0 +1,200 @@ +"""Offline construction and projection for metadata-only reference preflight.""" + +from __future__ import annotations + +import json +from pathlib import Path +from typing import Any + +from pydantic import BaseModel, ValidationError + +from omiv.assurance.models import ( + AssuranceDimension, + AssuranceRequest, + AssuranceRequirement, + CostClass, + EvidencePhase, + PlannedOperation, + VerdictRole, +) +from omiv.canonical import canonical_sha256 +from omiv.errors import OmivInputError +from omiv.hf.json_loader import load_bounded_json +from omiv.reference_preflight.models import ( + ArtifactRole, + FutureRuntimePlan, + ReferencePreflightEvidence, + ReferencePreflightProfile, +) +from omiv.safe_write import atomic_write_text + +MAX_CONTROL_BYTES = 8 * 1024 * 1024 + + +def _pretty(model: BaseModel) -> str: + return ( + json.dumps( + model.model_dump(mode="json", by_alias=True), + ensure_ascii=False, + indent=2, + sort_keys=True, + ) + + "\n" + ) + + +def load_profile(path: Path) -> ReferencePreflightProfile: + value, _ = load_bounded_json(path, max_bytes=MAX_CONTROL_BYTES) + try: + return ReferencePreflightProfile.model_validate(value) + except (ValidationError, ValueError) as exc: + raise OmivInputError(f"invalid reference preflight profile: {exc}") from exc + + +def load_evidence(path: Path) -> ReferencePreflightEvidence: + value, _ = load_bounded_json(path, max_bytes=MAX_CONTROL_BYTES) + try: + return ReferencePreflightEvidence.model_validate(value) + except (ValidationError, ValueError) as exc: + raise OmivInputError(f"invalid reference preflight evidence: {exc}") from exc + + +def _normalize_reference(profile: ReferencePreflightProfile, reference: str) -> str: + candidate = reference.strip().removesuffix("/") + matches = { + alias.removesuffix("/"): profile.canonical_reference for alias in profile.reference_aliases + } + normalized = matches.get(candidate) + if normalized is None: + raise OmivInputError( + "reference is not covered by the selected offline profile; " + "no remote facts were inferred" + ) + return normalized + + +def _build_future_plan(profile: ReferencePreflightProfile) -> FutureRuntimePlan: + body: dict[str, Any] = { + "schema": "omiv.reference-runtime-plan.v1", + **profile.future_runtime_plan.model_dump(mode="json"), + } + digest = canonical_sha256({"domain": body["schema"], "body": body}) + return FutureRuntimePlan.model_validate( + { + **body, + "plan_id": f"reference_runtime_plan_{digest[:32]}", + "plan_digest": digest, + } + ) + + +def build_reference_preflight( + profile: ReferencePreflightProfile, reference: str +) -> ReferencePreflightEvidence: + canonical_reference = _normalize_reference(profile, reference) + profile_json = profile.model_dump(mode="json", by_alias=True) + profile_digest = canonical_sha256({"reference-preflight-profile": profile_json}) + future_plan = _build_future_plan(profile) + companions = sorted( + {item.role for item in profile.artifacts if item.role != ArtifactRole.MAIN_MODEL}, + key=lambda item: item.value, + ) + body: dict[str, Any] = { + "schema": "omiv.reference-preflight-evidence.v1", + "candidate_notice": "P7_REFERENCE_PREFLIGHT_CANDIDATE", + "input_reference": reference, + "artifact_reference": canonical_reference, + "resolved_identity_status": "REMOTE_REVISION_PINNED", + "resolved_identity": f"{canonical_reference}@{profile.resolved_revision}", + "payload_verification": "NOT_DOWNLOADED", + "source_binding": "NOT_ESTABLISHED", + "architecture_status": "DECLARED", + "architecture": profile.configuration.architecture, + "tokenizer_configuration": "REMOTE_METADATA_OBSERVED", + "companion_roles": [item.value for item in companions], + "runtime_candidates": [item.runtime for item in profile.runtime_candidates], + "runtime_requirement": "DECLARED_OFFICIAL_REQUIREMENT", + "runtime_probe": "NOT_RUN", + "numerical_fidelity": "NOT_EVALUATED", + "semantic_fidelity": "NOT_EVALUATED", + "performance": "NOT_EVALUATED", + "safety": "NOT_EVALUATED", + "production_readiness": "NOT_ESTABLISHED", + "provenance_gaps": profile.provenance_gaps, + "profile_digest": profile_digest, + "profile": profile_json, + "future_runtime_plan": future_plan.model_dump(mode="json", by_alias=True), + "limitations": profile.limitations, + } + digest = canonical_sha256({"domain": body["schema"], "body": body}) + return ReferencePreflightEvidence.model_validate( + { + **body, + "evidence_id": f"reference_preflight_evidence_{digest[:32]}", + "evidence_digest": digest, + } + ) + + +def build_assurance_request( + evidence: ReferencePreflightEvidence, source_path: str +) -> AssuranceRequest: + """Preserve candidate evidence as opaque supporting bytes under Phase 6F v1.""" + return AssuranceRequest( + request_id=f"reference-{evidence.evidence_digest[:24]}", + subject=evidence.resolved_identity, + requirements=[ + AssuranceRequirement( + member_path=( + f"provenance/reference-preflight-{evidence.evidence_digest[:16]}.json" + ), + source_path=source_path, + phase=EvidencePhase.EXTERNAL, + required=True, + media_type="application/octet-stream", + expected_schema=None, + dimension=AssuranceDimension.PROVENANCE, + verdict_role=VerdictRole.SUPPORTING, + ) + ], + planned_operations=[ + PlannedOperation( + cost_class=CostClass.GPU, + reason=evidence.future_runtime_plan.question, + ) + ], + ) + + +def write_evidence(evidence: ReferencePreflightEvidence, output: Path) -> None: + atomic_write_text(output, _pretty(evidence)) + + +def write_assurance_request(request: AssuranceRequest, output: Path) -> None: + atomic_write_text(output, _pretty(request)) + + +def concise_evidence_summary(evidence: ReferencePreflightEvidence) -> str: + primary_requirement = evidence.profile.runtime_candidates[0].requirement + companion_text = " + ".join(item.value for item in evidence.companion_roles) + rows = [ + ("Artifact reference", evidence.artifact_reference), + ( + "Resolved identity", + f"{evidence.resolved_identity_status}: {evidence.resolved_identity}", + ), + ("Payload verification", evidence.payload_verification), + ("Source binding", evidence.source_binding), + ("Architecture", f"{evidence.architecture_status}: {evidence.architecture}"), + ("Tokenizer/config", evidence.tokenizer_configuration), + ("Companion artifacts", f"DECLARED: {companion_text}"), + ("Runtime candidates", " / ".join(evidence.runtime_candidates)), + ("Runtime requirement", f"{evidence.runtime_requirement}: {primary_requirement}"), + ("Runtime probe", evidence.runtime_probe), + ("Provenance gaps", f"EXPLICIT: {len(evidence.provenance_gaps)}"), + ( + "Next step", + f"READY_FOR_GPU: execute {evidence.future_runtime_plan.plan_id}", + ), + ] + return "\n".join(f"{label:<21} {value}" for label, value in rows) diff --git a/src/omiv/reference_preflight/probes.py b/src/omiv/reference_preflight/probes.py new file mode 100644 index 0000000..bd15557 --- /dev/null +++ b/src/omiv/reference_preflight/probes.py @@ -0,0 +1,56 @@ +"""Deterministic raster fixtures for reference-runtime acceptance probes.""" + +from __future__ import annotations + +import struct +import zlib + +PNG_HEIGHT = 64 +PNG_WIDTH = 64 +RED_SQUARE_END = 48 +RED_SQUARE_START = 16 + + +def _png_chunk(kind: bytes, data: bytes) -> bytes: + payload = kind + data + return struct.pack(">I", len(data)) + payload + struct.pack(">I", zlib.crc32(payload)) + + +def _uncompressed_zlib_stream(data: bytes) -> bytes: + """Return a fixed zlib stream containing one uncompressed DEFLATE block.""" + if len(data) > 0xFFFF: + raise ValueError("probe payload exceeds one deterministic DEFLATE block") + length = len(data) + return ( + b"\x78\x01" + + b"\x01" + + struct.pack("I", zlib.adler32(data)) + ) + + +def build_red_square_png() -> bytes: + """Build a fixed 64x64 indexed PNG with one red square on white.""" + rows = bytearray() + for y in range(PNG_HEIGHT): + rows.append(0) # PNG filter: None + for x in range(PNG_WIDTH): + inside_square = ( + RED_SQUARE_START <= x < RED_SQUARE_END + and RED_SQUARE_START <= y < RED_SQUARE_END + ) + rows.append(1 if inside_square else 0) + + header = struct.pack(">IIBBBBB", PNG_WIDTH, PNG_HEIGHT, 8, 3, 0, 0, 0) + palette = bytes((255, 255, 255, 212, 0, 0)) + return b"".join( + ( + b"\x89PNG\r\n\x1a\n", + _png_chunk(b"IHDR", header), + _png_chunk(b"PLTE", palette), + _png_chunk(b"IDAT", _uncompressed_zlib_stream(bytes(rows))), + _png_chunk(b"IEND", b""), + ) + ) diff --git a/src/omiv/runtime_compatibility/__init__.py b/src/omiv/runtime_compatibility/__init__.py new file mode 100644 index 0000000..cb6b1a4 --- /dev/null +++ b/src/omiv/runtime_compatibility/__init__.py @@ -0,0 +1 @@ +"""Candidate Phase 7B local runtime compatibility profiles.""" diff --git a/src/omiv/runtime_compatibility/models.py b/src/omiv/runtime_compatibility/models.py new file mode 100644 index 0000000..4f76ecb --- /dev/null +++ b/src/omiv/runtime_compatibility/models.py @@ -0,0 +1,680 @@ +"""Strict models for the candidate Phase 7B runtime compatibility slice.""" + +from __future__ import annotations + +import base64 +import hashlib +from enum import StrEnum +from typing import Any, Literal + +from pydantic import Field, model_validator + +from omiv.canonical import canonical_sha256 +from omiv.models import StrictModel +from omiv.payload_integrity.paths import validate_portable_path + +PROFILE_ID = "omiv.runtime-compatibility-profile.llama-cpp-native-output.v1" +PROFILE_NAME = "llama.cpp" +STAGE_ORDER = ("LOAD", "TOKENIZER", "PREFILL", "DECODE", "OUTPUT") +PROFILE_ENVIRONMENT = ( + ("CUDA_VISIBLE_DEVICES", ""), + ("HIP_VISIBLE_DEVICES", ""), + ("HOME", "{WORK_DIRECTORY}"), + ("LANG", "C.UTF-8"), + ("LC_ALL", "C.UTF-8"), + ("PATH", "/usr/bin:/bin"), + ("TMPDIR", "{WORK_DIRECTORY}"), +) + + +class PlanStatus(StrEnum): + READY = "READY" + BLOCKED = "BLOCKED" + + +class CompatibilityStatus(StrEnum): + VERIFIED_WITHIN_PROFILE = "VERIFIED_WITHIN_PROFILE" + NOT_VERIFIED = "NOT_VERIFIED" + + +class StageName(StrEnum): + LOAD = "LOAD" + TOKENIZER = "TOKENIZER" + PREFILL = "PREFILL" + DECODE = "DECODE" + OUTPUT = "OUTPUT" + + +class StageStatus(StrEnum): + PASS = "PASS" + FAIL = "FAIL" + UNKNOWN = "UNKNOWN" + NOT_TESTED = "NOT_TESTED" + + +class FindingSeverity(StrEnum): + INFO = "INFO" + WARN = "WARN" + ERROR = "ERROR" + + +class FileAvailability(StrEnum): + AVAILABLE = "AVAILABLE" + MISSING = "MISSING" + INVALID = "INVALID" + + +class RuntimeCompatibilityLimits(StrictModel): + timeout_seconds: int = Field(default=30, ge=1, le=300) + version_timeout_seconds: int = Field(default=5, ge=1, le=30) + max_stdout_bytes: int = Field(default=1024 * 1024, ge=1024, le=16 * 1024 * 1024) + max_stderr_bytes: int = Field(default=256 * 1024, ge=1024, le=4 * 1024 * 1024) + max_executable_bytes: int = Field(default=1024 * 1024 * 1024, ge=1, le=2**31) + max_artifact_bytes: int = Field(default=16 * 1024**3, ge=1, le=1024**4) + max_work_files: int = Field(default=16, ge=0, le=256) + max_work_file_bytes: int = Field(default=1024 * 1024, ge=1, le=64 * 1024 * 1024) + max_work_total_bytes: int = Field(default=4 * 1024 * 1024, ge=1, le=256 * 1024 * 1024) + + +class RuntimeTestVector(StrictModel): + vector_id: str = Field(min_length=3, max_length=96, pattern=r"^[A-Za-z0-9._-]+$") + prompt: str = Field(min_length=1, max_length=32_768) + expected_output: str = Field(min_length=1, max_length=1024 * 1024) + seed: int = Field(default=0, ge=0, le=2**31 - 1) + max_generated_tokens: int = Field(default=16, ge=1, le=4096) + + +class RuntimeCompatibilityRequest(StrictModel): + schema_id: Literal["omiv.runtime-compatibility-request.v1"] = Field( + default="omiv.runtime-compatibility-request.v1", alias="schema" + ) + request_id: str = Field(min_length=3, max_length=128, pattern=r"^[A-Za-z0-9._-]+$") + profile_id: Literal["omiv.runtime-compatibility-profile.llama-cpp-native-output.v1"] = ( + "omiv.runtime-compatibility-profile.llama-cpp-native-output.v1" + ) + executable_path: str + executable_sha256: str = Field(pattern=r"^[0-9a-f]{64}$") + expected_runtime_version: str = Field(min_length=1, max_length=512) + artifact_path: str + artifact_sha256: str = Field(pattern=r"^[0-9a-f]{64}$") + test_vector: RuntimeTestVector + limits: RuntimeCompatibilityLimits = Field(default_factory=RuntimeCompatibilityLimits) + + @model_validator(mode="after") + def validate_request(self) -> RuntimeCompatibilityRequest: + validate_portable_path(self.executable_path) + validate_portable_path(self.artifact_path) + if self.executable_path == self.artifact_path: + raise ValueError("runtime executable and artifact paths must differ") + if len(self.test_vector.expected_output.encode("utf-8")) > self.limits.max_stdout_bytes: + raise ValueError("expected output exceeds the stdout capture limit") + return self + + +class LocalFileBinding(StrictModel): + path: str + expected_sha256: str = Field(pattern=r"^[0-9a-f]{64}$") + availability: FileAvailability + observed_sha256: str | None = Field(default=None, pattern=r"^[0-9a-f]{64}$") + size: int | None = Field(default=None, ge=0) + executable: bool | None = None + issues: list[str] = Field(max_length=16) + + @model_validator(mode="after") + def validate_binding(self) -> LocalFileBinding: + validate_portable_path(self.path) + if self.availability == FileAvailability.AVAILABLE: + if self.observed_sha256 is None or self.size is None: + raise ValueError("available file binding requires size and digest") + elif self.observed_sha256 is not None or self.size is not None: + raise ValueError("unavailable file binding cannot claim size or digest") + return self + + +class EnvironmentVariable(StrictModel): + name: str = Field(min_length=1, max_length=64, pattern=r"^[A-Z][A-Z0-9_]*$") + value: str = Field(max_length=1024) + + +class InvocationPlan(StrictModel): + version_arguments: list[str] = Field(min_length=1, max_length=16) + run_arguments: list[str] = Field(min_length=1, max_length=64) + environment: list[EnvironmentVariable] = Field(min_length=1, max_length=16) + working_directory: Literal["FRESH_EMPTY_TEMPORARY_DIRECTORY"] = ( + "FRESH_EMPTY_TEMPORARY_DIRECTORY" + ) + shell: Literal[False] = False + accelerator: Literal["CPU_ONLY"] = "CPU_ONLY" + command_digest: str = Field(pattern=r"^[0-9a-f]{64}$") + + @model_validator(mode="after") + def validate_invocation(self) -> InvocationPlan: + if len({item.name for item in self.environment}) != len(self.environment): + raise ValueError("invocation environment names must be unique") + if any( + "\x00" in argument or len(argument) > 32_768 + for argument in (*self.version_arguments, *self.run_arguments) + ): + raise ValueError("invocation arguments must be bounded and cannot contain NUL") + body = self.model_dump(mode="json", exclude={"command_digest"}) + if self.command_digest != canonical_sha256({"invocation": body}): + raise ValueError("invocation command identity mismatch") + return self + + +def build_profile_invocation(request: RuntimeCompatibilityRequest) -> InvocationPlan: + """Build the one bounded native argument array supported by this profile.""" + body: dict[str, Any] = { + "version_arguments": ["{runtime_executable}", "--version"], + "run_arguments": [ + "{runtime_executable}", + "--model", + "{artifact}", + "--prompt", + request.test_vector.prompt, + "--seed", + str(request.test_vector.seed), + "--temp", + "0", + "--n-predict", + str(request.test_vector.max_generated_tokens), + "--gpu-layers", + "0", + "--simple-io", + "--no-display-prompt", + ], + "environment": [ + EnvironmentVariable(name=name, value=value).model_dump(mode="json") + for name, value in PROFILE_ENVIRONMENT + ], + "working_directory": "FRESH_EMPTY_TEMPORARY_DIRECTORY", + "shell": False, + "accelerator": "CPU_ONLY", + } + return InvocationPlan.model_validate( + {**body, "command_digest": canonical_sha256({"invocation": body})} + ) + + +class RuntimeCostSummary(StrictModel): + network: Literal[False] = False + download: Literal[False] = False + compilation: Literal[False] = False + conversion: Literal[False] = False + runtime_execution: Literal[True] = True + gpu: Literal[False] = False + + +class RuntimeCompatibilityPlan(StrictModel): + schema_id: Literal["omiv.runtime-compatibility-plan.v1"] = Field( + default="omiv.runtime-compatibility-plan.v1", alias="schema" + ) + plan_id: str + plan_digest: str = Field(pattern=r"^[0-9a-f]{64}$") + request: RuntimeCompatibilityRequest + status: PlanStatus + profile_name: Literal["llama.cpp"] = "llama.cpp" + executable: LocalFileBinding + artifact: LocalFileBinding + invocation: InvocationPlan + available: list[str] = Field(max_length=16) + missing: list[str] = Field(max_length=16) + expected_work: list[str] = Field(max_length=16) + costs: RuntimeCostSummary + limitations: list[str] = Field(max_length=32) + + @model_validator(mode="after") + def validate_identity_and_status(self) -> RuntimeCompatibilityPlan: + body = self.model_dump(mode="json", by_alias=True, exclude={"plan_id", "plan_digest"}) + digest = canonical_sha256({"domain": self.schema_id, "body": body}) + if self.plan_digest != digest or self.plan_id != f"runtime_compat_plan_{digest[:32]}": + raise ValueError("runtime compatibility plan canonical identity mismatch") + ready = ( + self.executable.availability == FileAvailability.AVAILABLE + and self.artifact.availability == FileAvailability.AVAILABLE + and self.executable.observed_sha256 == self.executable.expected_sha256 + and self.artifact.observed_sha256 == self.artifact.expected_sha256 + and self.executable.executable is True + and not self.executable.issues + and not self.artifact.issues + ) + if (self.status == PlanStatus.READY) != ready: + raise ValueError("runtime compatibility plan status does not match local bindings") + if ( + self.executable.path != self.request.executable_path + or self.executable.expected_sha256 != self.request.executable_sha256 + or self.artifact.path != self.request.artifact_path + or self.artifact.expected_sha256 != self.request.artifact_sha256 + ): + raise ValueError("runtime compatibility plan bindings do not match its request") + if self.invocation != build_profile_invocation(self.request): + raise ValueError("runtime compatibility plan invocation does not match its profile") + expected_available = [ + label + for label, binding in ( + ("runtime executable", self.executable), + ("artifact", self.artifact), + ) + if binding.availability == FileAvailability.AVAILABLE + ] + expected_missing = [ + f"{label}: {issue}" + for label, binding in ( + ("runtime executable", self.executable), + ("artifact", self.artifact), + ) + for issue in binding.issues + ] + if self.available != expected_available or self.missing != expected_missing: + raise ValueError("runtime compatibility plan availability summary is incoherent") + return self + + +class BoundedCapture(StrictModel): + encoding: Literal["BASE64"] = "BASE64" + captured_base64: str + captured_bytes: int = Field(ge=0) + limit_bytes: int = Field(ge=1) + overflow: bool + captured_sha256: str = Field(pattern=r"^[0-9a-f]{64}$") + + @model_validator(mode="after") + def validate_capture(self) -> BoundedCapture: + try: + raw = base64.b64decode(self.captured_base64, validate=True) + except ValueError as exc: + raise ValueError("captured stream is not canonical base64") from exc + if len(raw) != self.captured_bytes or len(raw) > self.limit_bytes: + raise ValueError("captured stream length is invalid") + if hashlib.sha256(raw).hexdigest() != self.captured_sha256: + raise ValueError("captured stream digest mismatch") + return self + + +class ProcessCapture(StrictModel): + arguments: list[str] = Field(min_length=1, max_length=64) + environment: list[EnvironmentVariable] = Field(min_length=1, max_length=16) + working_directory: Literal["FRESH_EMPTY_TEMPORARY_DIRECTORY"] = ( + "FRESH_EMPTY_TEMPORARY_DIRECTORY" + ) + shell: Literal[False] = False + return_code: int | None = None + timed_out: bool + duration_ms: int = Field(ge=0) + stdout: BoundedCapture + stderr: BoundedCapture + + +class WorkFileRecord(StrictModel): + path: str + size: int = Field(ge=0) + sha256: str | None = Field(default=None, pattern=r"^[0-9a-f]{64}$") + + @model_validator(mode="after") + def validate_path(self) -> WorkFileRecord: + validate_portable_path(self.path) + return self + + +class StageResult(StrictModel): + stage: StageName + status: StageStatus + basis: str = Field(min_length=1, max_length=1000) + + +_UNOBSERVABLE_STAGE_BASIS = { + StageName.LOAD: ( + "The native CLI observation does not independently establish that the runtime " + "opened or consumed the pinned artifact." + ), + StageName.TOKENIZER: ( + "The native CLI observation does not independently expose tokenizer execution." + ), + StageName.PREFILL: ( + "The native CLI observation does not independently expose prefill execution." + ), + StageName.DECODE: ( + "The native CLI observation does not independently prove that emitted bytes were " + "produced by model decoding." + ), +} + + +def derive_native_stage_results( + capture: ProcessCapture | None, + expected_output: str, +) -> tuple[list[StageResult], list[tuple[str, str]]]: + """Derive stage results only from the bounded native process observation.""" + if capture is None: + return ( + [ + StageResult( + stage=StageName(stage), + status=StageStatus.NOT_TESTED, + basis="The native compatibility invocation was not executed.", + ) + for stage in STAGE_ORDER + ], + [], + ) + issue: tuple[str, str] | None = None + if capture.timed_out: + issue = ("NATIVE_EXECUTION_TIMEOUT", "The native compatibility execution timed out.") + elif capture.stdout.overflow: + issue = ( + "NATIVE_STDOUT_OVERFLOW", + "The native compatibility execution exceeded the stdout capture bound.", + ) + elif capture.stderr.overflow: + issue = ( + "NATIVE_STDERR_OVERFLOW", + "The native compatibility execution exceeded the stderr capture bound.", + ) + elif capture.return_code != 0: + issue = ( + "NATIVE_EXECUTION_NONZERO", + f"The native compatibility execution exited with {capture.return_code}.", + ) + try: + stdout = base64.b64decode(capture.stdout.captured_base64, validate=True).decode("utf-8") + except (UnicodeDecodeError, ValueError): + stdout = None + if issue is None: + issue = ( + "NATIVE_STDOUT_NOT_UTF8", + "The native compatibility stdout is not valid UTF-8.", + ) + try: + base64.b64decode(capture.stderr.captured_base64, validate=True).decode("utf-8") + except (UnicodeDecodeError, ValueError): + if issue is None: + issue = ( + "NATIVE_STDERR_NOT_UTF8", + "The native compatibility stderr is not valid UTF-8.", + ) + if issue is not None: + return ( + [ + StageResult( + stage=StageName(stage), + status=StageStatus.UNKNOWN, + basis=( + "The bounded native process observation was incomplete or invalid: " + f"{issue[1]}" + ), + ) + for stage in STAGE_ORDER + ], + [issue], + ) + if stdout != expected_output: + return ( + [ + *[ + StageResult( + stage=stage, + status=StageStatus.UNKNOWN, + basis=basis, + ) + for stage, basis in _UNOBSERVABLE_STAGE_BASIS.items() + ], + StageResult( + stage=StageName.OUTPUT, + status=StageStatus.FAIL, + basis="Native stdout did not exactly match the supplied expected output.", + ), + ], + [ + ( + "EXPECTED_OUTPUT_MISMATCH", + "Native stdout did not exactly match the supplied test-vector output.", + ) + ], + ) + return ( + [ + *[ + StageResult(stage=stage, status=StageStatus.UNKNOWN, basis=basis) + for stage, basis in _UNOBSERVABLE_STAGE_BASIS.items() + ], + StageResult( + stage=StageName.OUTPUT, + status=StageStatus.PASS, + basis="Native stdout exactly matched the supplied expected output bytes.", + ), + ], + [], + ) + + +class RuntimeCompatibilityFinding(StrictModel): + code: str = Field(min_length=1, max_length=200) + severity: FindingSeverity + detail: str = Field(min_length=1, max_length=1000) + + +class ExecutionFileBinding(StrictModel): + path: str + expected_sha256: str = Field(pattern=r"^[0-9a-f]{64}$") + preflight_sha256: str = Field(pattern=r"^[0-9a-f]{64}$") + execution_sha256: str | None = Field(default=None, pattern=r"^[0-9a-f]{64}$") + execution_size: int | None = Field(default=None, ge=0) + + @model_validator(mode="after") + def validate_path(self) -> ExecutionFileBinding: + validate_portable_path(self.path) + if (self.execution_sha256 is None) != (self.execution_size is None): + raise ValueError("execution file binding requires digest and size together") + return self + + +class RuntimeIdentityEvidence(ExecutionFileBinding): + expected_version: str = Field(min_length=1, max_length=512) + observed_version: str | None = Field(default=None, max_length=512) + identity_verified: bool + + @model_validator(mode="after") + def validate_identity_state(self) -> RuntimeIdentityEvidence: + verified = ( + self.execution_sha256 == self.expected_sha256 == self.preflight_sha256 + and self.observed_version == self.expected_version + ) + if self.identity_verified != verified: + raise ValueError("runtime identity verification state mismatch") + return self + + +class RuntimeCompatibilityEvidence(StrictModel): + schema_id: Literal["omiv.runtime-compatibility-evidence.v1"] = Field( + default="omiv.runtime-compatibility-evidence.v1", alias="schema" + ) + evidence_id: str + evidence_digest: str = Field(pattern=r"^[0-9a-f]{64}$") + candidate_notice: Literal["PHASE_7B_CANDIDATE_PHASE_7_UNFROZEN"] = ( + "PHASE_7B_CANDIDATE_PHASE_7_UNFROZEN" + ) + plan: RuntimeCompatibilityPlan + request_id: str + plan_id: str + plan_digest: str = Field(pattern=r"^[0-9a-f]{64}$") + profile_id: Literal["omiv.runtime-compatibility-profile.llama-cpp-native-output.v1"] + profile_name: Literal["llama.cpp"] + status: CompatibilityStatus + artifact: ExecutionFileBinding + runtime: RuntimeIdentityEvidence + invocation: InvocationPlan + test_vector: RuntimeTestVector + test_vector_digest: str = Field(pattern=r"^[0-9a-f]{64}$") + limits: RuntimeCompatibilityLimits + version_execution: ProcessCapture | None + compatibility_execution: ProcessCapture | None + work_files: list[WorkFileRecord] = Field(max_length=256) + stages: list[StageResult] = Field(min_length=5, max_length=5) + findings: list[RuntimeCompatibilityFinding] = Field(max_length=256) + unknowns: list[str] = Field(max_length=32) + limitations: list[str] = Field(max_length=32) + + @model_validator(mode="after") + def validate_evidence(self) -> RuntimeCompatibilityEvidence: + body = self.model_dump( + mode="json", by_alias=True, exclude={"evidence_id", "evidence_digest"} + ) + digest = canonical_sha256({"domain": self.schema_id, "body": body}) + if ( + self.evidence_digest != digest + or self.evidence_id != f"runtime_compat_evidence_{digest[:32]}" + ): + raise ValueError("runtime compatibility evidence canonical identity mismatch") + if self.plan.status != PlanStatus.READY: + raise ValueError("runtime compatibility evidence must embed an executable plan") + if self.plan_id != self.plan.plan_id or self.plan_digest != self.plan.plan_digest: + raise ValueError("runtime compatibility evidence plan identity mismatch") + if self.request_id != self.plan.request.request_id: + raise ValueError("runtime compatibility evidence request identity mismatch") + if ( + self.profile_id != self.plan.request.profile_id + or self.profile_name != self.plan.profile_name + ): + raise ValueError("runtime compatibility evidence profile mismatch") + if self.invocation != self.plan.invocation: + raise ValueError("runtime compatibility evidence invocation mismatch") + if self.test_vector != self.plan.request.test_vector: + raise ValueError("runtime compatibility evidence test vector mismatch") + if self.limits != self.plan.request.limits: + raise ValueError("runtime compatibility evidence limits mismatch") + if ( + self.artifact.path != self.plan.artifact.path + or self.artifact.expected_sha256 != self.plan.artifact.expected_sha256 + or self.artifact.preflight_sha256 != self.plan.artifact.observed_sha256 + ): + raise ValueError("runtime compatibility evidence artifact binding mismatch") + if ( + self.runtime.path != self.plan.executable.path + or self.runtime.expected_sha256 != self.plan.executable.expected_sha256 + or self.runtime.preflight_sha256 != self.plan.executable.observed_sha256 + or self.runtime.expected_version != self.plan.request.expected_runtime_version + ): + raise ValueError("runtime compatibility evidence executable binding mismatch") + if self.test_vector_digest != canonical_sha256( + {"runtime-test-vector": self.test_vector.model_dump(mode="json")} + ): + raise ValueError("runtime test vector digest mismatch") + if tuple(item.stage.value for item in self.stages) != STAGE_ORDER: + raise ValueError("runtime evidence must contain every stage exactly once in order") + + def capture_matches(capture: ProcessCapture, arguments: list[str]) -> bool: + return ( + capture.arguments == arguments + and capture.environment == self.invocation.environment + and capture.working_directory == self.invocation.working_directory + and capture.shell == self.invocation.shell + and capture.stdout.limit_bytes == self.limits.max_stdout_bytes + and capture.stderr.limit_bytes == self.limits.max_stderr_bytes + ) + + version_capture_valid = False + if self.version_execution is not None: + try: + observed_version = base64.b64decode( + self.version_execution.stdout.captured_base64, validate=True + ).decode("utf-8").strip() + base64.b64decode( + self.version_execution.stderr.captured_base64, validate=True + ).decode("utf-8") + except (UnicodeDecodeError, ValueError): + observed_version = "" + version_capture_valid = ( + capture_matches(self.version_execution, self.invocation.version_arguments) + and self.version_execution.return_code == 0 + and not self.version_execution.timed_out + and not self.version_execution.stdout.overflow + and not self.version_execution.stderr.overflow + and observed_version == self.runtime.expected_version + and observed_version == self.runtime.observed_version + ) + if self.version_execution is not None and not capture_matches( + self.version_execution, self.invocation.version_arguments + ): + raise ValueError("runtime version capture does not match the canonical plan") + if self.runtime.identity_verified != version_capture_valid: + raise ValueError("runtime identity does not match its captured version observation") + if self.compatibility_execution is not None and not capture_matches( + self.compatibility_execution, self.invocation.run_arguments + ): + raise ValueError("native runtime capture does not match the canonical plan") + if self.compatibility_execution is not None and not version_capture_valid: + raise ValueError("native execution lacks a valid pinned version observation") + derived_stages, derived_issues = derive_native_stage_results( + self.compatibility_execution, self.test_vector.expected_output + ) + if self.stages != derived_stages: + raise ValueError("runtime stages do not match the raw native observation") + expected_unknowns = [ + f"{item.stage.value} was {item.status.value}; compatibility is not established." + for item in derived_stages + if item.status in {StageStatus.UNKNOWN, StageStatus.NOT_TESTED} + ] + if self.unknowns != expected_unknowns: + raise ValueError("runtime unknowns do not match the derived stage results") + finding_by_code = {item.code: item for item in self.findings} + for code, detail in derived_issues: + finding = finding_by_code.get(code) + if ( + finding is None + or finding.severity != FindingSeverity.ERROR + or finding.detail != detail + ): + raise ValueError("native observation finding is missing or incoherent") + if self.compatibility_execution is not None and not derived_issues: + finding = finding_by_code.get("INTERNAL_STAGES_UNOBSERVABLE") + if finding is None or finding.severity != FindingSeverity.WARN: + raise ValueError("unobservable native stages must remain explicit") + artifact_binding_valid = ( + self.artifact.execution_sha256 + == self.artifact.expected_sha256 + == self.artifact.preflight_sha256 + and self.artifact.execution_size == self.plan.artifact.size + ) + if not artifact_binding_valid and not any( + item.code.startswith("ARTIFACT_CHANGED_") and item.severity == FindingSeverity.ERROR + for item in self.findings + ): + raise ValueError("changed artifact binding lacks a fail-closed finding") + runtime_file_binding_valid = ( + self.runtime.execution_sha256 + == self.runtime.expected_sha256 + == self.runtime.preflight_sha256 + and self.runtime.execution_size == self.plan.executable.size + ) + if not runtime_file_binding_valid and not any( + item.code.startswith("RUNTIME_CHANGED_") and item.severity == FindingSeverity.ERROR + for item in self.findings + ): + raise ValueError("changed executable binding lacks a fail-closed finding") + native_capture_valid = ( + self.compatibility_execution is not None + and capture_matches(self.compatibility_execution, self.invocation.run_arguments) + and self.compatibility_execution.return_code == 0 + and not self.compatibility_execution.timed_out + and not self.compatibility_execution.stdout.overflow + and not self.compatibility_execution.stderr.overflow + ) + verified = ( + version_capture_valid + and native_capture_valid + and self.runtime.identity_verified + and artifact_binding_valid + and runtime_file_binding_valid + and self.compatibility_execution is not None + and self.compatibility_execution.return_code == 0 + and not self.compatibility_execution.timed_out + and not self.compatibility_execution.stdout.overflow + and not self.compatibility_execution.stderr.overflow + and all(item.status == StageStatus.PASS for item in self.stages) + and not any(item.severity == FindingSeverity.ERROR for item in self.findings) + and not self.unknowns + and not self.work_files + ) + if (self.status == CompatibilityStatus.VERIFIED_WITHIN_PROFILE) != verified: + raise ValueError("runtime compatibility status does not match bounded evidence") + return self diff --git a/src/omiv/runtime_compatibility/operations.py b/src/omiv/runtime_compatibility/operations.py new file mode 100644 index 0000000..d09c2f9 --- /dev/null +++ b/src/omiv/runtime_compatibility/operations.py @@ -0,0 +1,874 @@ +"""Bounded local orchestration for candidate runtime compatibility profiles.""" + +from __future__ import annotations + +import base64 +import hashlib +import json +import os +import selectors +import signal +import stat +import subprocess +import tempfile +import time +from contextlib import suppress +from pathlib import Path +from typing import IO, Any + +from pydantic import BaseModel, ValidationError + +from omiv.canonical import canonical_sha256 +from omiv.errors import OmivInputError +from omiv.hf.json_loader import load_bounded_json +from omiv.payload_integrity.paths import validate_portable_path +from omiv.runtime_compatibility.models import ( + PROFILE_ENVIRONMENT, + PROFILE_NAME, + BoundedCapture, + CompatibilityStatus, + EnvironmentVariable, + ExecutionFileBinding, + FileAvailability, + FindingSeverity, + LocalFileBinding, + PlanStatus, + ProcessCapture, + RuntimeCompatibilityEvidence, + RuntimeCompatibilityFinding, + RuntimeCompatibilityPlan, + RuntimeCompatibilityRequest, + RuntimeCostSummary, + RuntimeIdentityEvidence, + StageResult, + StageStatus, + WorkFileRecord, + build_profile_invocation, + derive_native_stage_results, +) +from omiv.safe_write import atomic_write_text + +MAX_CONTROL_BYTES = 64 * 1024 * 1024 +_READ_CHUNK = 64 * 1024 + + +def _pretty(model: BaseModel) -> str: + return ( + json.dumps( + model.model_dump(mode="json", by_alias=True), + ensure_ascii=False, + indent=2, + sort_keys=True, + ) + + "\n" + ) + + +def load_request(path: Path) -> RuntimeCompatibilityRequest: + value, _ = load_bounded_json(path, max_bytes=MAX_CONTROL_BYTES) + try: + return RuntimeCompatibilityRequest.model_validate(value) + except (ValidationError, ValueError) as exc: + raise OmivInputError(f"invalid runtime compatibility request: {exc}") from exc + + +def load_plan(path: Path) -> RuntimeCompatibilityPlan: + value, _ = load_bounded_json(path, max_bytes=MAX_CONTROL_BYTES) + try: + return RuntimeCompatibilityPlan.model_validate(value) + except (ValidationError, ValueError) as exc: + raise OmivInputError(f"invalid runtime compatibility plan: {exc}") from exc + + +def load_evidence(path: Path) -> RuntimeCompatibilityEvidence: + value, _ = load_bounded_json(path, max_bytes=MAX_CONTROL_BYTES) + try: + return RuntimeCompatibilityEvidence.model_validate(value) + except (ValidationError, ValueError) as exc: + raise OmivInputError(f"invalid runtime compatibility evidence: {exc}") from exc + + +def write_plan(plan: RuntimeCompatibilityPlan, output: Path) -> None: + atomic_write_text(output, _pretty(plan)) + + +def write_evidence(evidence: RuntimeCompatibilityEvidence, output: Path) -> None: + atomic_write_text(output, _pretty(evidence)) + + +def _resolve_local_path(root: Path, portable_path: str) -> Path: + validate_portable_path(portable_path) + target = root / Path(*portable_path.split("/")) + cursor = root + for component in portable_path.split("/"): + cursor /= component + if cursor.is_symlink(): + raise OmivInputError(f"local input path contains a symlink: {portable_path}") + resolved = target.resolve(strict=False) + if resolved != root and root not in resolved.parents: + raise OmivInputError(f"local input path escapes root: {portable_path}") + return target + + +def _hash_regular_file(path: Path, maximum_bytes: int) -> tuple[int, str]: + before = path.stat(follow_symlinks=False) + if not stat.S_ISREG(before.st_mode): + raise OmivInputError(f"local input is not a regular file: {path.name}") + if before.st_size > maximum_bytes: + raise OmivInputError( + f"local input exceeds its byte limit: {path.name} ({before.st_size} > {maximum_bytes})" + ) + digest = hashlib.sha256() + observed = 0 + with path.open("rb") as handle: + while True: + chunk = handle.read(_READ_CHUNK) + if not chunk: + break + observed += len(chunk) + if observed > maximum_bytes: + raise OmivInputError(f"local input grew beyond its byte limit: {path.name}") + digest.update(chunk) + after = path.stat(follow_symlinks=False) + if ( + before.st_dev, + before.st_ino, + before.st_size, + before.st_mtime_ns, + ) != ( + after.st_dev, + after.st_ino, + after.st_size, + after.st_mtime_ns, + ) or observed != after.st_size: + raise OmivInputError(f"local input changed while it was hashed: {path.name}") + return observed, digest.hexdigest() + + +def _inspect_binding( + root: Path, + portable_path: str, + expected_sha256: str, + maximum_bytes: int, + *, + require_executable: bool, +) -> LocalFileBinding: + try: + path = _resolve_local_path(root, portable_path) + except (OSError, ValueError, OmivInputError) as exc: + return LocalFileBinding( + path=portable_path, + expected_sha256=expected_sha256, + availability=FileAvailability.INVALID, + executable=False if require_executable else None, + issues=[str(exc)], + ) + if not path.exists(): + return LocalFileBinding( + path=portable_path, + expected_sha256=expected_sha256, + availability=FileAvailability.MISSING, + executable=False if require_executable else None, + issues=["Explicitly supplied local file is missing."], + ) + try: + size, digest = _hash_regular_file(path, maximum_bytes) + except (OSError, OmivInputError) as exc: + return LocalFileBinding( + path=portable_path, + expected_sha256=expected_sha256, + availability=FileAvailability.INVALID, + executable=False if require_executable else None, + issues=[str(exc)], + ) + executable = os.access(path, os.X_OK) if require_executable else None + issues: list[str] = [] + if digest != expected_sha256: + issues.append("Observed SHA-256 does not match the explicitly supplied digest.") + if require_executable and not executable: + issues.append("Explicitly supplied runtime file is not executable.") + return LocalFileBinding( + path=portable_path, + expected_sha256=expected_sha256, + availability=FileAvailability.AVAILABLE, + observed_sha256=digest, + size=size, + executable=executable, + issues=issues, + ) + + +def build_plan( + request: RuntimeCompatibilityRequest, root: Path +) -> RuntimeCompatibilityPlan: + if os.name != "posix": + raise OmivInputError( + "candidate runtime compatibility execution requires POSIX process controls" + ) + if root.is_symlink() or not root.is_dir(): + raise OmivInputError("runtime compatibility root must be a regular local directory") + resolved_root = root.resolve() + executable = _inspect_binding( + resolved_root, + request.executable_path, + request.executable_sha256, + request.limits.max_executable_bytes, + require_executable=True, + ) + artifact = _inspect_binding( + resolved_root, + request.artifact_path, + request.artifact_sha256, + request.limits.max_artifact_bytes, + require_executable=False, + ) + ready = not executable.issues and not artifact.issues + available = [ + label + for label, binding in (("runtime executable", executable), ("artifact", artifact)) + if binding.availability == FileAvailability.AVAILABLE + ] + missing = [ + f"{label}: {issue}" + for label, binding in (("runtime executable", executable), ("artifact", artifact)) + for issue in binding.issues + ] + body: dict[str, Any] = { + "schema": "omiv.runtime-compatibility-plan.v1", + "request": request.model_dump(mode="json", by_alias=True), + "status": PlanStatus.READY.value if ready else PlanStatus.BLOCKED.value, + "profile_name": PROFILE_NAME, + "executable": executable.model_dump(mode="json"), + "artifact": artifact.model_dump(mode="json"), + "invocation": build_profile_invocation(request).model_dump(mode="json"), + "available": available, + "missing": missing, + "expected_work": [ + "Re-hash the executable and artifact before and after execution.", + "Run the supplied executable version command under bounded local controls.", + "Run one CPU-only supplied test vector through the native CLI and capture its " + "raw output.", + ], + "costs": RuntimeCostSummary().model_dump(mode="json"), + "limitations": [ + "Phase 7B is a candidate capability; Phase 7 scope remains unfrozen.", + "The profile applies only to the exact artifact, executable, version, invocation, " + "CPU-only environment, limits, and test vector recorded here.", + "The explicitly supplied executable must be trusted by the operator; this " + "orchestrator is not a security sandbox.", + "No result is generalized to another artifact, runtime, device, accelerator, " + "prompt, or environment.", + "The native CLI does not independently expose model loading, tokenization, " + "prefill, or decode internals; those stages remain UNKNOWN.", + ], + } + digest = canonical_sha256({"domain": body["schema"], "body": body}) + return RuntimeCompatibilityPlan.model_validate( + {**body, "plan_id": f"runtime_compat_plan_{digest[:32]}", "plan_digest": digest} + ) + + +def _bounded_capture(raw: bytes, limit: int, overflow: bool) -> BoundedCapture: + return BoundedCapture( + captured_base64=base64.b64encode(raw).decode("ascii"), + captured_bytes=len(raw), + limit_bytes=limit, + overflow=overflow, + captured_sha256=hashlib.sha256(raw).hexdigest(), + ) + + +def _terminate(process: subprocess.Popen[bytes]) -> None: + with suppress(ProcessLookupError): + os.killpg(process.pid, signal.SIGKILL) + try: + process.wait(timeout=2) + except subprocess.TimeoutExpired as exc: + raise OmivInputError("runtime process could not be terminated within its bound") from exc + + +def _child_file_limit(maximum_bytes: int) -> None: + import resource + + resource.setrlimit(resource.RLIMIT_FSIZE, (maximum_bytes, maximum_bytes)) + + +def _run_bounded( + actual_arguments: list[str], + evidence_arguments: list[str], + *, + cwd: Path, + environment: dict[str, str], + evidence_environment: list[EnvironmentVariable], + timeout_seconds: int, + max_stdout_bytes: int, + max_stderr_bytes: int, + max_work_file_bytes: int, +) -> ProcessCapture: + started = time.monotonic() + process = subprocess.Popen( + actual_arguments, + shell=False, + cwd=cwd, + env=environment, + stdin=subprocess.DEVNULL, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE, + start_new_session=True, + preexec_fn=lambda: _child_file_limit(max_work_file_bytes), + ) + if process.stdout is None or process.stderr is None: + _terminate(process) + raise OmivInputError("runtime process pipes were not available") + stdout_descriptor = process.stdout.fileno() + stderr_descriptor = process.stderr.fileno() + streams: dict[int, tuple[IO[bytes], bytearray, int]] = { + stdout_descriptor: (process.stdout, bytearray(), max_stdout_bytes), + stderr_descriptor: (process.stderr, bytearray(), max_stderr_bytes), + } + overflow = {process.stdout.fileno(): False, process.stderr.fileno(): False} + selector = selectors.DefaultSelector() + for descriptor, (stream, _buffer, _limit) in streams.items(): + selector.register(stream, selectors.EVENT_READ, descriptor) + deadline = started + timeout_seconds + timed_out = False + killed = False + try: + while selector.get_map(): + remaining = deadline - time.monotonic() + if remaining <= 0: + timed_out = True + _terminate(process) + killed = True + break + events = selector.select(timeout=min(0.05, remaining)) + if not events and process.poll() is not None: + events = selector.select(timeout=0) + for key, _mask in events: + descriptor = int(key.data) + stream, buffer, limit = streams[descriptor] + chunk = os.read(descriptor, _READ_CHUNK) + if not chunk: + selector.unregister(stream) + stream.close() + continue + room = limit - len(buffer) + if room > 0: + buffer.extend(chunk[:room]) + if len(chunk) > room: + overflow[descriptor] = True + _terminate(process) + killed = True + break + if killed: + break + finally: + selector.close() + for stream, _buffer, _limit in streams.values(): + if not stream.closed: + stream.close() + if not killed: + try: + process.wait(timeout=max(0.1, deadline - time.monotonic())) + except subprocess.TimeoutExpired: + timed_out = True + _terminate(process) + duration_ms = max(0, int((time.monotonic() - started) * 1000)) + stdout_buffer = bytes(streams[stdout_descriptor][1]) + stderr_buffer = bytes(streams[stderr_descriptor][1]) + return ProcessCapture( + arguments=evidence_arguments, + environment=evidence_environment, + return_code=process.returncode, + timed_out=timed_out, + duration_ms=duration_ms, + stdout=_bounded_capture(stdout_buffer, max_stdout_bytes, overflow[stdout_descriptor]), + stderr=_bounded_capture(stderr_buffer, max_stderr_bytes, overflow[stderr_descriptor]), + ) + + +def _capture_bytes(capture: BoundedCapture) -> bytes: + return base64.b64decode(capture.captured_base64, validate=True) + + +def _execution_binding( + root: Path, + planned: LocalFileBinding, + maximum_bytes: int, +) -> tuple[ExecutionFileBinding, str | None]: + if planned.observed_sha256 is None: + raise OmivInputError("ready plan lacks a preflight file digest") + try: + path = _resolve_local_path(root, planned.path) + size, digest = _hash_regular_file(path, maximum_bytes) + except (OSError, ValueError, OmivInputError) as exc: + return ( + ExecutionFileBinding( + path=planned.path, + expected_sha256=planned.expected_sha256, + preflight_sha256=planned.observed_sha256, + ), + str(exc), + ) + binding = ExecutionFileBinding( + path=planned.path, + expected_sha256=planned.expected_sha256, + preflight_sha256=planned.observed_sha256, + execution_sha256=digest, + execution_size=size, + ) + if digest != planned.expected_sha256 or digest != planned.observed_sha256: + return binding, f"{planned.path} changed after preflight or no longer matches its pin" + return binding, None + + +def _finding( + findings: list[RuntimeCompatibilityFinding], + code: str, + severity: FindingSeverity, + detail: str, +) -> None: + if len(findings) < 256: + findings.append( + RuntimeCompatibilityFinding(code=code, severity=severity, detail=detail[:1000]) + ) + + +def _inspect_work_directory( + root: Path, + plan: RuntimeCompatibilityPlan, + findings: list[RuntimeCompatibilityFinding], +) -> list[WorkFileRecord]: + records: list[WorkFileRecord] = [] + stack = [root] + observed_entries = 0 + observed_total = 0 + while stack: + directory = stack.pop() + try: + entries = sorted(os.scandir(directory), key=lambda item: item.name.encode("utf-8")) + except OSError as exc: + _finding(findings, "WORK_DIRECTORY_READ_FAILED", FindingSeverity.ERROR, str(exc)) + break + for entry in entries: + observed_entries += 1 + if observed_entries > plan.request.limits.max_work_files: + _finding( + findings, + "WORK_FILE_COUNT_EXCEEDED", + FindingSeverity.ERROR, + "The runtime working directory exceeded its entry bound.", + ) + return records + path = Path(entry.path) + relative = path.relative_to(root).as_posix() + try: + validate_portable_path(relative) + except ValueError as exc: + _finding(findings, "UNSAFE_WORK_FILE", FindingSeverity.ERROR, str(exc)) + continue + if entry.is_symlink(): + _finding( + findings, + "UNEXPECTED_WORK_SYMLINK", + FindingSeverity.ERROR, + f"The runtime created a symlink: {relative}", + ) + elif entry.is_dir(follow_symlinks=False): + stack.append(path) + _finding( + findings, + "UNEXPECTED_WORK_DIRECTORY", + FindingSeverity.ERROR, + f"The runtime created a directory: {relative}", + ) + elif entry.is_file(follow_symlinks=False): + size = entry.stat(follow_symlinks=False).st_size + observed_total += size + digest: str | None = None + if size > plan.request.limits.max_work_file_bytes: + _finding( + findings, + "WORK_FILE_SIZE_EXCEEDED", + FindingSeverity.ERROR, + f"The runtime-created file exceeded its bound: {relative}", + ) + elif observed_total > plan.request.limits.max_work_total_bytes: + _finding( + findings, + "WORK_FILE_TOTAL_EXCEEDED", + FindingSeverity.ERROR, + "Runtime-created files exceeded the cumulative byte bound.", + ) + else: + _observed, digest = _hash_regular_file( + path, plan.request.limits.max_work_file_bytes + ) + records.append(WorkFileRecord(path=relative, size=size, sha256=digest)) + _finding( + findings, + "UNEXPECTED_WORK_FILE", + FindingSeverity.ERROR, + f"The no-output-files profile observed a runtime-created file: {relative}", + ) + else: + _finding( + findings, + "UNEXPECTED_WORK_OBJECT", + FindingSeverity.ERROR, + f"The runtime created a non-regular object: {relative}", + ) + return records + + +def _parse_version(capture: ProcessCapture) -> tuple[str | None, str | None]: + if capture.timed_out: + return None, "Runtime version command timed out." + if capture.stdout.overflow or capture.stderr.overflow: + return None, "Runtime version command exceeded a captured-output bound." + if capture.return_code != 0: + return None, f"Runtime version command exited with {capture.return_code}." + try: + version = _capture_bytes(capture.stdout).decode("utf-8").strip() + _capture_bytes(capture.stderr).decode("utf-8") + except UnicodeDecodeError: + return None, "Runtime version stdout or stderr is not valid UTF-8." + if not version: + return None, "Runtime version output is empty." + if len(version) > 512: + return None, "Runtime version output is too long to identify unambiguously." + return version, None + + +def _runtime_environment(work_directory: Path) -> dict[str, str]: + return { + name: str(work_directory) if value == "{WORK_DIRECTORY}" else value + for name, value in PROFILE_ENVIRONMENT + } + + +def _build_evidence( + plan: RuntimeCompatibilityPlan, + *, + artifact: ExecutionFileBinding, + runtime: RuntimeIdentityEvidence, + version_execution: ProcessCapture | None, + compatibility_execution: ProcessCapture | None, + work_files: list[WorkFileRecord], + stages: list[StageResult], + findings: list[RuntimeCompatibilityFinding], +) -> RuntimeCompatibilityEvidence: + unknowns = [ + f"{item.stage.value} was {item.status.value}; compatibility is not established." + for item in stages + if item.status in {StageStatus.UNKNOWN, StageStatus.NOT_TESTED} + ] + verified = ( + version_execution is not None + and version_execution.return_code == 0 + and not version_execution.timed_out + and not version_execution.stdout.overflow + and not version_execution.stderr.overflow + and runtime.identity_verified + and artifact.execution_sha256 + == artifact.expected_sha256 + == artifact.preflight_sha256 + and compatibility_execution is not None + and compatibility_execution.return_code == 0 + and not compatibility_execution.timed_out + and not compatibility_execution.stdout.overflow + and not compatibility_execution.stderr.overflow + and all(item.status == StageStatus.PASS for item in stages) + and not any(item.severity == FindingSeverity.ERROR for item in findings) + and not unknowns + and not work_files + ) + vector_digest = canonical_sha256( + {"runtime-test-vector": plan.request.test_vector.model_dump(mode="json")} + ) + body: dict[str, Any] = { + "schema": "omiv.runtime-compatibility-evidence.v1", + "candidate_notice": "PHASE_7B_CANDIDATE_PHASE_7_UNFROZEN", + "plan": plan.model_dump(mode="json", by_alias=True), + "request_id": plan.request.request_id, + "plan_id": plan.plan_id, + "plan_digest": plan.plan_digest, + "profile_id": plan.request.profile_id, + "profile_name": PROFILE_NAME, + "status": ( + CompatibilityStatus.VERIFIED_WITHIN_PROFILE.value + if verified + else CompatibilityStatus.NOT_VERIFIED.value + ), + "artifact": artifact.model_dump(mode="json"), + "runtime": runtime.model_dump(mode="json"), + "invocation": plan.invocation.model_dump(mode="json"), + "test_vector": plan.request.test_vector.model_dump(mode="json"), + "test_vector_digest": vector_digest, + "limits": plan.request.limits.model_dump(mode="json"), + "version_execution": ( + version_execution.model_dump(mode="json") if version_execution is not None else None + ), + "compatibility_execution": ( + compatibility_execution.model_dump(mode="json") + if compatibility_execution is not None + else None + ), + "work_files": [item.model_dump(mode="json") for item in work_files], + "stages": [item.model_dump(mode="json") for item in stages], + "findings": [item.model_dump(mode="json") for item in findings], + "unknowns": unknowns, + "limitations": [ + "Phase 7B is a candidate capability; Phase 7 scope remains unfrozen.", + "VERIFIED_WITHIN_PROFILE is reserved for a profile with complete independently " + "observable stages; this native-output profile leaves internal stages UNKNOWN.", + "Raw native stdout and stderr are runtime observations, but they do not prove " + "model loading, tokenization, prefill, or decode internals; those stages remain " + "UNKNOWN for this profile.", + "Canonical hashes establish integrity and internal consistency, not origin " + "authenticity, runtime honesty, or weight-attributable inference.", + "The bounded probe does not establish numerical or semantic equivalence, " + "production readiness, performance, safety, or approval.", + "No conclusion is generalized to another artifact, runtime, device, accelerator, " + "prompt, or environment.", + ], + } + digest = canonical_sha256({"domain": body["schema"], "body": body}) + return RuntimeCompatibilityEvidence.model_validate( + { + **body, + "evidence_id": f"runtime_compat_evidence_{digest[:32]}", + "evidence_digest": digest, + } + ) + + +def execute_plan(plan: RuntimeCompatibilityPlan, root: Path) -> RuntimeCompatibilityEvidence: + if plan.status != PlanStatus.READY: + raise OmivInputError("blocked runtime compatibility plan cannot be executed") + if root.is_symlink() or not root.is_dir(): + raise OmivInputError("runtime compatibility root must be a regular local directory") + resolved_root = root.resolve() + findings: list[RuntimeCompatibilityFinding] = [] + artifact, artifact_issue = _execution_binding( + resolved_root, plan.artifact, plan.request.limits.max_artifact_bytes + ) + runtime_file, runtime_issue = _execution_binding( + resolved_root, plan.executable, plan.request.limits.max_executable_bytes + ) + if artifact_issue is not None: + _finding( + findings, + "ARTIFACT_CHANGED_AFTER_PREFLIGHT", + FindingSeverity.ERROR, + artifact_issue, + ) + if runtime_issue is not None: + _finding(findings, "RUNTIME_CHANGED_AFTER_PREFLIGHT", FindingSeverity.ERROR, runtime_issue) + runtime = RuntimeIdentityEvidence( + **runtime_file.model_dump(mode="python"), + expected_version=plan.request.expected_runtime_version, + observed_version=None, + identity_verified=False, + ) + stages, _stage_issues = derive_native_stage_results( + None, plan.request.test_vector.expected_output + ) + if artifact_issue is not None or runtime_issue is not None: + return _build_evidence( + plan, + artifact=artifact, + runtime=runtime, + version_execution=None, + compatibility_execution=None, + work_files=[], + stages=stages, + findings=findings, + ) + + executable_path = _resolve_local_path(resolved_root, plan.executable.path).resolve() + artifact_path = _resolve_local_path(resolved_root, plan.artifact.path).resolve() + version_execution: ProcessCapture | None = None + compatibility_execution: ProcessCapture | None = None + work_files: list[WorkFileRecord] = [] + output_parent = resolved_root + with tempfile.TemporaryDirectory(prefix=".omiv-runtime-compat-", dir=output_parent) as name: + work_directory = Path(name) + environment = _runtime_environment(work_directory) + try: + version_execution = _run_bounded( + [str(executable_path), "--version"], + plan.invocation.version_arguments, + cwd=work_directory, + environment=environment, + evidence_environment=plan.invocation.environment, + timeout_seconds=plan.request.limits.version_timeout_seconds, + max_stdout_bytes=plan.request.limits.max_stdout_bytes, + max_stderr_bytes=plan.request.limits.max_stderr_bytes, + max_work_file_bytes=plan.request.limits.max_work_file_bytes, + ) + except (OSError, OmivInputError) as exc: + _finding(findings, "VERSION_EXECUTION_FAILED", FindingSeverity.ERROR, str(exc)) + observed_version: str | None = None + version_issue: str | None = "Runtime version command did not start." + if version_execution is not None: + observed_version, version_issue = _parse_version(version_execution) + if version_issue is not None: + _finding(findings, "RUNTIME_IDENTITY_AMBIGUOUS", FindingSeverity.ERROR, version_issue) + + after_version_artifact, artifact_after_issue = _execution_binding( + resolved_root, plan.artifact, plan.request.limits.max_artifact_bytes + ) + after_version_runtime, runtime_after_issue = _execution_binding( + resolved_root, plan.executable, plan.request.limits.max_executable_bytes + ) + artifact = after_version_artifact + runtime_file = after_version_runtime + if artifact_after_issue is not None: + _finding( + findings, + "ARTIFACT_CHANGED_DURING_VERSION_CHECK", + FindingSeverity.ERROR, + artifact_after_issue, + ) + if runtime_after_issue is not None: + _finding( + findings, + "RUNTIME_CHANGED_DURING_VERSION_CHECK", + FindingSeverity.ERROR, + runtime_after_issue, + ) + runtime = RuntimeIdentityEvidence( + **runtime_file.model_dump(mode="python"), + expected_version=plan.request.expected_runtime_version, + observed_version=observed_version, + identity_verified=( + observed_version == plan.request.expected_runtime_version + and runtime_after_issue is None + ), + ) + if ( + observed_version is not None + and observed_version != plan.request.expected_runtime_version + ): + _finding( + findings, + "RUNTIME_VERSION_MISMATCH", + FindingSeverity.ERROR, + "Observed runtime version does not match the explicitly supplied version pin.", + ) + + work_files = _inspect_work_directory(work_directory, plan, findings) + if ( + runtime.identity_verified + and artifact_after_issue is None + and runtime_after_issue is None + and not any(item.severity == FindingSeverity.ERROR for item in findings) + ): + actual_arguments = [ + str(executable_path) if item == "{runtime_executable}" else + str(artifact_path) if item == "{artifact}" else item + for item in plan.invocation.run_arguments + ] + try: + compatibility_execution = _run_bounded( + actual_arguments, + plan.invocation.run_arguments, + cwd=work_directory, + environment=environment, + evidence_environment=plan.invocation.environment, + timeout_seconds=plan.request.limits.timeout_seconds, + max_stdout_bytes=plan.request.limits.max_stdout_bytes, + max_stderr_bytes=plan.request.limits.max_stderr_bytes, + max_work_file_bytes=plan.request.limits.max_work_file_bytes, + ) + except (OSError, OmivInputError) as exc: + _finding( + findings, + "COMPATIBILITY_EXECUTION_FAILED", + FindingSeverity.ERROR, + str(exc), + ) + stages, stage_issues = derive_native_stage_results( + compatibility_execution, plan.request.test_vector.expected_output + ) + for code, detail in stage_issues: + _finding(findings, code, FindingSeverity.ERROR, detail) + if compatibility_execution is not None and not stage_issues: + _finding( + findings, + "INTERNAL_STAGES_UNOBSERVABLE", + FindingSeverity.WARN, + "Native output cannot independently establish LOAD, TOKENIZER, PREFILL, " + "or DECODE internals.", + ) + work_files = _inspect_work_directory(work_directory, plan, findings) + + final_artifact, final_artifact_issue = _execution_binding( + resolved_root, plan.artifact, plan.request.limits.max_artifact_bytes + ) + final_runtime_file, final_runtime_issue = _execution_binding( + resolved_root, plan.executable, plan.request.limits.max_executable_bytes + ) + artifact = final_artifact + if final_artifact_issue is not None: + _finding( + findings, + "ARTIFACT_CHANGED_DURING_EXECUTION", + FindingSeverity.ERROR, + final_artifact_issue, + ) + if final_runtime_issue is not None: + _finding( + findings, + "RUNTIME_CHANGED_DURING_EXECUTION", + FindingSeverity.ERROR, + final_runtime_issue, + ) + runtime = RuntimeIdentityEvidence( + **final_runtime_file.model_dump(mode="python"), + expected_version=plan.request.expected_runtime_version, + observed_version=runtime.observed_version, + identity_verified=( + runtime.observed_version == plan.request.expected_runtime_version + and final_runtime_issue is None + ), + ) + return _build_evidence( + plan, + artifact=artifact, + runtime=runtime, + version_execution=version_execution, + compatibility_execution=compatibility_execution, + work_files=work_files, + stages=stages, + findings=findings, + ) + + +def concise_plan_summary(plan: RuntimeCompatibilityPlan) -> str: + executable_present = plan.executable.availability == FileAvailability.AVAILABLE + artifact_present = plan.artifact.availability == FileAvailability.AVAILABLE + executable_pinned = ( + executable_present + and plan.executable.observed_sha256 == plan.executable.expected_sha256 + ) + artifact_pinned = ( + artifact_present and plan.artifact.observed_sha256 == plan.artifact.expected_sha256 + ) + return ( + f"{plan.status.value} profile={plan.profile_name} " + f"executable_present={'yes' if executable_present else 'no'} " + f"executable={'yes' if plan.executable.executable is True else 'no'} " + f"executable_pinned={'yes' if executable_pinned else 'no'} " + f"artifact_present={'yes' if artifact_present else 'no'} " + f"artifact_pinned={'yes' if artifact_pinned else 'no'} " + "network=no download=no compilation=no conversion=no runtime=yes gpu=no" + ) + + +def concise_evidence_summary(evidence: RuntimeCompatibilityEvidence) -> str: + rows: list[str] = [evidence.profile_name] + rows.extend(f"{item.stage.value:<10} {item.status.value}" for item in evidence.stages) + rows.extend(("", f"Runtime compatibility: {evidence.status.value}")) + return "\n".join(rows) diff --git a/src/omiv/smart_preflight/__init__.py b/src/omiv/smart_preflight/__init__.py new file mode 100644 index 0000000..f16dd08 --- /dev/null +++ b/src/omiv/smart_preflight/__init__.py @@ -0,0 +1 @@ +"""Phase 7A candidate Smart Preflight / Auto Planner.""" diff --git a/src/omiv/smart_preflight/models.py b/src/omiv/smart_preflight/models.py new file mode 100644 index 0000000..dd53f0c --- /dev/null +++ b/src/omiv/smart_preflight/models.py @@ -0,0 +1,177 @@ +"""Strict models for the candidate Phase 7A local auto planner.""" + +from __future__ import annotations + +from enum import StrEnum +from typing import Literal + +from pydantic import Field, model_validator + +from omiv.assurance.models import AssuranceDimension, AssuranceRequest, EvidencePhase, VerdictRole +from omiv.canonical import canonical_sha256 +from omiv.models import StrictModel +from omiv.payload_integrity.paths import validate_path_set, validate_portable_path + +MAX_SEARCH_PATHS = 32 +MAX_CANDIDATES = 512 +MAX_FINDINGS = 512 + + +class SmartPreflightStatus(StrEnum): + READY = "READY" + READY_WITH_GAPS = "READY_WITH_GAPS" + BLOCKED = "BLOCKED" + + +class CoverageStatus(StrEnum): + COVERED = "COVERED" + MISSING = "MISSING" + AMBIGUOUS = "AMBIGUOUS" + + +class FindingSeverity(StrEnum): + INFO = "INFO" + WARN = "WARN" + ERROR = "ERROR" + + +class SmartPreflightIntent(StrictModel): + schema_id: Literal["omiv.smart-preflight-intent.v1"] = Field( + default="omiv.smart-preflight-intent.v1", alias="schema" + ) + intent_id: str = Field(min_length=3, max_length=96, pattern=r"^[A-Za-z0-9._-]+$") + subject: str = Field(min_length=1, max_length=1000) + search_paths: list[str] = Field(default_factory=lambda: ["."], max_length=MAX_SEARCH_PATHS) + required_dimensions: list[AssuranceDimension] = Field(default_factory=list, max_length=32) + + @model_validator(mode="after") + def validate_search_scope(self) -> SmartPreflightIntent: + if not self.search_paths: + raise ValueError("Smart Preflight requires at least one search path") + if len(set(self.required_dimensions)) != len(self.required_dimensions): + raise ValueError("required dimensions must be unique") + if "." in self.search_paths: + if self.search_paths != ["."]: + raise ValueError("root search path cannot be combined with narrower paths") + else: + validate_path_set(tuple(self.search_paths)) + return self + + +class DiscoveryCandidate(StrictModel): + source_path: str + schema_id: str = Field(max_length=200) + phase: EvidencePhase + dimension: AssuranceDimension + proposed_verdict_role: VerdictRole + member_path: str + size: int = Field(ge=0) + sha256: str = Field(pattern=r"^[0-9a-f]{64}$") + selected: bool + selection_reason: str = Field(min_length=1, max_length=1000) + + @model_validator(mode="after") + def validate_paths(self) -> DiscoveryCandidate: + validate_portable_path(self.source_path) + validate_portable_path(self.member_path) + return self + + +class DimensionCoverage(StrictModel): + dimension: AssuranceDimension + status: CoverageStatus + candidate_paths: list[str] = Field(max_length=MAX_CANDIDATES) + selected_path: str | None = None + + @model_validator(mode="after") + def validate_selection(self) -> DimensionCoverage: + for path in self.candidate_paths: + validate_portable_path(path) + if self.status == CoverageStatus.COVERED: + if self.selected_path is None or self.selected_path not in self.candidate_paths: + raise ValueError("covered dimension requires a selected candidate") + elif self.selected_path is not None: + raise ValueError("uncovered dimension cannot claim a selected candidate") + return self + + +class DiscoveryFinding(StrictModel): + code: str = Field(min_length=1, max_length=200) + severity: FindingSeverity + detail: str = Field(min_length=1, max_length=1000) + source_path: str | None = None + + @model_validator(mode="after") + def validate_source_path(self) -> DiscoveryFinding: + if self.source_path is not None: + validate_portable_path(self.source_path) + return self + + +class SmartCostSummary(StrictModel): + inspected_entries: int = Field(ge=0) + inspected_json_files: int = Field(ge=0) + inspected_json_bytes: int = Field(ge=0) + download: Literal[False] = False + network: Literal[False] = False + conversion: Literal[False] = False + remote_collector: Literal[False] = False + gpu: Literal[False] = False + + +class SmartPreflightPlan(StrictModel): + schema_id: Literal["omiv.smart-preflight-plan.v1"] = Field( + default="omiv.smart-preflight-plan.v1", alias="schema" + ) + plan_id: str + plan_digest: str = Field(pattern=r"^[0-9a-f]{64}$") + intent: SmartPreflightIntent + status: SmartPreflightStatus + candidates: list[DiscoveryCandidate] = Field(max_length=MAX_CANDIDATES) + coverage: list[DimensionCoverage] = Field(max_length=32) + assurance_request: AssuranceRequest | None + findings: list[DiscoveryFinding] = Field(max_length=MAX_FINDINGS) + costs: SmartCostSummary + limitations: list[str] = Field(max_length=32) + + @model_validator(mode="after") + def validate_identity_and_handoff(self) -> SmartPreflightPlan: + body = self.model_dump(mode="json", by_alias=True, exclude={"plan_id", "plan_digest"}) + digest = canonical_sha256({"domain": self.schema_id, "body": body}) + if self.plan_digest != digest or self.plan_id != f"smart_plan_{digest[:32]}": + raise ValueError("Smart Preflight Plan canonical identity mismatch") + selected = [item for item in self.candidates if item.selected] + if (self.assurance_request is None) != (not selected): + raise ValueError("Assurance request must exactly reflect selected candidates") + if self.assurance_request is not None: + expected = { + ( + item.source_path, + item.member_path, + item.schema_id, + item.phase, + item.dimension, + VerdictRole.DIMENSION_VERDICT, + ) + for item in selected + } + actual = { + ( + item.source_path, + item.member_path, + item.expected_schema, + item.phase, + item.dimension, + item.verdict_role, + ) + for item in self.assurance_request.requirements + } + if actual != expected: + raise ValueError("Assurance request does not match selected candidates") + if ( + self.assurance_request.subject != self.intent.subject + or self.assurance_request.request_id != f"smart-{self.intent.intent_id}" + or self.assurance_request.planned_operations + ): + raise ValueError("Assurance request does not preserve the local intent boundary") + return self diff --git a/src/omiv/smart_preflight/operations.py b/src/omiv/smart_preflight/operations.py new file mode 100644 index 0000000..90baac1 --- /dev/null +++ b/src/omiv/smart_preflight/operations.py @@ -0,0 +1,541 @@ +"""Bounded local discovery and conservative Phase 6F request generation.""" + +from __future__ import annotations + +import hashlib +import json +from dataclasses import dataclass +from pathlib import Path +from typing import Any + +from pydantic import BaseModel, ValidationError + +from omiv.assurance.models import ( + AssuranceDimension, + AssuranceRequest, + AssuranceRequirement, + EvidencePhase, + VerdictRole, +) +from omiv.assurance.registry import PHASE_SCHEMAS +from omiv.canonical import canonical_sha256 +from omiv.errors import OmivInputError +from omiv.hf.json_loader import load_bounded_json +from omiv.payload_integrity.paths import validate_portable_path +from omiv.safe_write import atomic_write_text +from omiv.smart_preflight.models import ( + MAX_CANDIDATES, + MAX_FINDINGS, + CoverageStatus, + DimensionCoverage, + DiscoveryCandidate, + DiscoveryFinding, + FindingSeverity, + SmartCostSummary, + SmartPreflightIntent, + SmartPreflightPlan, + SmartPreflightStatus, +) + +MAX_CONTROL_BYTES = 8 * 1024 * 1024 +MAX_DISCOVERY_ENTRIES = 10_000 +MAX_DISCOVERY_JSON_FILES = 4_096 +MAX_DISCOVERY_JSON_BYTES = 256 * 1024 * 1024 +MAX_DISCOVERY_MEMBER_BYTES = 64 * 1024 * 1024 +MAX_DISCOVERY_DEPTH = 16 + +_DIMENSION_BY_PHASE = { + EvidencePhase.PHASE_5: AssuranceDimension.OTHER, + EvidencePhase.PHASE_6A: AssuranceDimension.STRUCTURE, + EvidencePhase.PHASE_6B: AssuranceDimension.STRUCTURE, + EvidencePhase.PHASE_6C: AssuranceDimension.FIDELITY, + EvidencePhase.PHASE_6D: AssuranceDimension.TOKENIZER_CONFIGURATION, + EvidencePhase.PHASE_6E: AssuranceDimension.RUNTIME, +} + +# Only final, phase-native evidence records are eligible for automatic verdict use. +# Everything else remains visible as supporting evidence and is not auto-selected. +_VERDICT_SCHEMAS = { + "omiv.payload-integrity-evidence.v1", + "omiv.remote-local-reconciliation-evidence.v1", + "omiv.quantization-fidelity-evidence.v1", + "omiv.tokenizer-configuration-parity-evidence.v1", + "omiv.runtime-resolution-parity-evidence.v1", +} + + +@dataclass +class _Counters: + entries: int = 0 + json_files: int = 0 + json_bytes: int = 0 + limit_exceeded: bool = False + + +@dataclass(frozen=True) +class _Observed: + source_path: str + schema_id: str + phase: EvidencePhase + dimension: AssuranceDimension + verdict_role: VerdictRole + size: int + sha256: str + + +def _pretty(model: BaseModel) -> str: + return ( + json.dumps( + model.model_dump(mode="json", by_alias=True), + ensure_ascii=False, + indent=2, + sort_keys=True, + ) + + "\n" + ) + + +def load_intent(path: Path) -> SmartPreflightIntent: + value, _ = load_bounded_json(path, max_bytes=MAX_CONTROL_BYTES) + try: + return SmartPreflightIntent.model_validate(value) + except (ValidationError, ValueError) as exc: + raise OmivInputError(f"invalid Smart Preflight intent: {exc}") from exc + + +def write_plan(plan: SmartPreflightPlan, output: Path) -> None: + atomic_write_text(output, _pretty(plan)) + + +def write_assurance_request(request: AssuranceRequest, output: Path) -> None: + atomic_write_text(output, _pretty(request)) + + +def _registry_by_schema() -> dict[str, tuple[EvidencePhase, type[BaseModel]]]: + result: dict[str, tuple[EvidencePhase, type[BaseModel]]] = {} + for phase, registry in PHASE_SCHEMAS.items(): + for schema_id, model in registry.items(): + existing = result.get(schema_id) + if existing is not None and existing != (phase, model): + raise RuntimeError(f"schema is ambiguous across phases: {schema_id}") + result[schema_id] = (phase, model) + return result + + +def _finding( + findings: list[DiscoveryFinding], + code: str, + severity: FindingSeverity, + detail: str, + source_path: str | None = None, +) -> None: + if len(findings) < MAX_FINDINGS: + findings.append( + DiscoveryFinding( + code=code, + severity=severity, + detail=detail[:1000], + source_path=source_path, + ) + ) + + +def _portable_relative(root: Path, path: Path) -> str | None: + try: + relative = path.relative_to(root).as_posix() + validate_portable_path(relative) + except (ValueError, OSError): + return None + return relative + + +def _inspect_json( + root: Path, + path: Path, + registry: dict[str, tuple[EvidencePhase, type[BaseModel]]], + counters: _Counters, + findings: list[DiscoveryFinding], +) -> _Observed | None: + source_path = _portable_relative(root, path) + if source_path is None: + _finding( + findings, + "UNSAFE_DISCOVERY_PATH", + FindingSeverity.ERROR, + "The local path is not portable and was not inspected.", + ) + return None + try: + size = path.stat().st_size + except OSError as exc: + _finding(findings, "LOCAL_STAT_FAILED", FindingSeverity.ERROR, str(exc), source_path) + return None + counters.json_files += 1 + counters.json_bytes += size + if counters.json_files > MAX_DISCOVERY_JSON_FILES: + counters.limit_exceeded = True + _finding( + findings, + "LIMIT_EXCEEDED:DISCOVERY_JSON_FILES", + FindingSeverity.ERROR, + "Smart Preflight stopped after reaching its JSON file limit.", + source_path, + ) + return None + if counters.json_bytes > MAX_DISCOVERY_JSON_BYTES: + counters.limit_exceeded = True + _finding( + findings, + "LIMIT_EXCEEDED:DISCOVERY_JSON_BYTES", + FindingSeverity.ERROR, + "Smart Preflight stopped after reaching its cumulative JSON byte limit.", + source_path, + ) + return None + if size > MAX_DISCOVERY_MEMBER_BYTES: + _finding( + findings, + "LIMIT_EXCEEDED:DISCOVERY_MEMBER_BYTES", + FindingSeverity.WARN, + "The JSON object exceeds the bounded local discovery member limit.", + source_path, + ) + return None + try: + value, raw = load_bounded_json(path, max_bytes=MAX_DISCOVERY_MEMBER_BYTES) + except OmivInputError as exc: + _finding(findings, "DISCOVERY_JSON_INVALID", FindingSeverity.WARN, str(exc), source_path) + return None + if not isinstance(value, dict): + return None + schema_id = value.get("schema") + if not isinstance(schema_id, str): + return None + registered = registry.get(schema_id) + if registered is None: + if schema_id.startswith("omiv."): + _finding( + findings, + "DISCOVERY_SCHEMA_UNSUPPORTED", + FindingSeverity.INFO, + f"No Phase 5/6A-6E Assurance registry entry exists for {schema_id}.", + source_path, + ) + return None + phase, model = registered + try: + model.model_validate(value) + except (ValidationError, ValueError) as exc: + _finding( + findings, + "DISCOVERY_SCHEMA_VALIDATION_FAILED", + FindingSeverity.WARN, + f"{schema_id} failed canonical validation: {exc}", + source_path, + ) + return None + role = ( + VerdictRole.DIMENSION_VERDICT + if schema_id in _VERDICT_SCHEMAS + else VerdictRole.SUPPORTING + ) + return _Observed( + source_path=source_path, + schema_id=schema_id, + phase=phase, + dimension=_DIMENSION_BY_PHASE[phase], + verdict_role=role, + size=len(raw), + sha256=hashlib.sha256(raw).hexdigest(), + ) + + +def _walk( + root: Path, + target: Path, + registry: dict[str, tuple[EvidencePhase, type[BaseModel]]], + counters: _Counters, + findings: list[DiscoveryFinding], + observed: list[_Observed], + depth: int = 0, +) -> None: + if counters.limit_exceeded: + return + relative = _portable_relative(root, target) + display_path = relative or target.name + if target.is_symlink(): + _finding( + findings, + "DISCOVERY_SYMLINK_SKIPPED", + FindingSeverity.WARN, + "Smart Preflight does not follow local symlinks.", + relative, + ) + return + if target.is_file(): + counters.entries += 1 + if target.suffix == ".json": + item = _inspect_json(root, target, registry, counters, findings) + if item is not None: + observed.append(item) + return + if not target.is_dir(): + _finding( + findings, + "DISCOVERY_OBJECT_SKIPPED", + FindingSeverity.WARN, + "Only regular files and directories are inspected.", + relative, + ) + return + if depth > MAX_DISCOVERY_DEPTH: + counters.limit_exceeded = True + _finding( + findings, + "LIMIT_EXCEEDED:DISCOVERY_DEPTH", + FindingSeverity.ERROR, + f"Discovery depth exceeded at {display_path}.", + relative, + ) + return + try: + children = sorted(target.iterdir(), key=lambda item: item.name.encode("utf-8")) + except OSError as exc: + _finding(findings, "LOCAL_DIRECTORY_READ_FAILED", FindingSeverity.ERROR, str(exc), relative) + return + for child in children: + counters.entries += 1 + if counters.entries > MAX_DISCOVERY_ENTRIES: + counters.limit_exceeded = True + _finding( + findings, + "LIMIT_EXCEEDED:DISCOVERY_ENTRIES", + FindingSeverity.ERROR, + "Smart Preflight stopped after reaching its directory-entry limit.", + _portable_relative(root, child), + ) + return + if child.is_symlink(): + _finding( + findings, + "DISCOVERY_SYMLINK_SKIPPED", + FindingSeverity.WARN, + "Smart Preflight does not follow local symlinks.", + _portable_relative(root, child), + ) + continue + if child.is_dir(): + _walk(root, child, registry, counters, findings, observed, depth + 1) + elif child.is_file() and child.suffix == ".json": + item = _inspect_json(root, child, registry, counters, findings) + if item is not None: + observed.append(item) + if counters.limit_exceeded: + return + + +def _resolve_search_path(root: Path, portable_path: str) -> Path: + target = root if portable_path == "." else root / Path(*portable_path.split("/")) + cursor = root + if portable_path != ".": + for component in portable_path.split("/"): + cursor /= component + if cursor.is_symlink(): + raise OmivInputError( + f"Smart Preflight search path contains a symlink: {portable_path}" + ) + resolved = target.resolve(strict=False) + if resolved != root and root not in resolved.parents: + raise OmivInputError(f"Smart Preflight search path escapes root: {portable_path}") + if not target.exists(): + raise OmivInputError(f"Smart Preflight search path does not exist: {portable_path}") + return target + + +def _member_path(item: _Observed) -> str: + return f"evidence/{item.phase.value.lower()}/{item.sha256}.json" + + +def build_smart_preflight(intent: SmartPreflightIntent, root: Path) -> SmartPreflightPlan: + if root.is_symlink() or not root.is_dir(): + raise OmivInputError("Smart Preflight root must be a regular local directory") + resolved_root = root.resolve() + registry = _registry_by_schema() + counters = _Counters() + findings: list[DiscoveryFinding] = [] + observed: list[_Observed] = [] + for search_path in intent.search_paths: + _walk( + resolved_root, + _resolve_search_path(resolved_root, search_path), + registry, + counters, + findings, + observed, + ) + if counters.limit_exceeded: + break + + observed.sort(key=lambda item: item.source_path.encode("utf-8")) + if len(observed) > MAX_CANDIDATES: + counters.limit_exceeded = True + observed = observed[:MAX_CANDIDATES] + _finding( + findings, + "LIMIT_EXCEEDED:DISCOVERY_CANDIDATES", + FindingSeverity.ERROR, + "Smart Preflight stopped after reaching its validated-candidate limit.", + ) + verdict_candidates = [ + item for item in observed if item.verdict_role == VerdictRole.DIMENSION_VERDICT + ] + desired = ( + list(intent.required_dimensions) + if intent.required_dimensions + else sorted({item.dimension for item in verdict_candidates}, key=lambda item: item.value) + ) + if not desired: + _finding( + findings, + "NO_AUTO_VERDICT_EVIDENCE", + FindingSeverity.WARN, + "No final Phase 6A-6E evidence was available for automatic selection.", + ) + selected_sources: set[str] = set() + coverage: list[DimensionCoverage] = [] + for dimension in desired: + dimension_items = [item for item in verdict_candidates if item.dimension == dimension] + by_digest: dict[str, list[_Observed]] = {} + for item in dimension_items: + by_digest.setdefault(item.sha256, []).append(item) + paths = [item.source_path for item in dimension_items] + if not dimension_items: + coverage_status = CoverageStatus.MISSING + selected_path = None + _finding( + findings, + "REQUIRED_DIMENSION_MISSING", + FindingSeverity.WARN, + f"No auto-verdict evidence was discovered for {dimension.value}.", + ) + elif len(by_digest) > 1: + coverage_status = CoverageStatus.AMBIGUOUS + selected_path = None + _finding( + findings, + "VERDICT_CANDIDATE_AMBIGUOUS", + FindingSeverity.WARN, + f"Multiple distinct verdict records were discovered for {dimension.value}; " + "none was selected.", + ) + else: + coverage_status = CoverageStatus.COVERED + selected_path = dimension_items[0].source_path + selected_sources.add(selected_path) + if len(dimension_items) > 1: + _finding( + findings, + "DUPLICATE_VERDICT_BYTES_DEDUPLICATED", + FindingSeverity.INFO, + f"Identical {dimension.value} verdict bytes were deduplicated " + "deterministically.", + selected_path, + ) + coverage.append( + DimensionCoverage( + dimension=dimension, + status=coverage_status, + candidate_paths=paths, + selected_path=selected_path, + ) + ) + + candidates = [ + DiscoveryCandidate( + source_path=item.source_path, + schema_id=item.schema_id, + phase=item.phase, + dimension=item.dimension, + proposed_verdict_role=item.verdict_role, + member_path=_member_path(item), + size=item.size, + sha256=item.sha256, + selected=item.source_path in selected_sources, + selection_reason=( + "Unique final evidence record for the requested dimension." + if item.source_path in selected_sources + else ( + "Supporting, duplicate, out-of-scope, or ambiguous evidence was not " + "auto-selected." + ) + ), + ) + for item in observed + ] + selected = [item for item in candidates if item.selected] + assurance_request: AssuranceRequest | None = None + if selected: + assurance_request = AssuranceRequest( + request_id=f"smart-{intent.intent_id}", + subject=intent.subject, + requirements=[ + AssuranceRequirement( + member_path=item.member_path, + source_path=item.source_path, + phase=item.phase, + required=True, + expected_schema=item.schema_id, + dimension=item.dimension, + verdict_role=VerdictRole.DIMENSION_VERDICT, + ) + for item in selected + ], + planned_operations=[], + ) + gaps = any(item.status != CoverageStatus.COVERED for item in coverage) + if counters.limit_exceeded or not selected: + plan_status = SmartPreflightStatus.BLOCKED + elif gaps: + plan_status = SmartPreflightStatus.READY_WITH_GAPS + else: + plan_status = SmartPreflightStatus.READY + limitations = [ + "Phase 7A is a candidate slice; Phase 7 scope is not frozen.", + "Discovery is bounded, local, and read-only; it does not download, convert, " + "contact a remote collector, or use a GPU.", + "Only unique final Phase 6A-6E evidence records are auto-selected as dimension verdicts.", + "The generated request remains subject to the unchanged Phase 6F Assurance " + "preflight and verification semantics.", + ] + body: dict[str, Any] = { + "schema": "omiv.smart-preflight-plan.v1", + "intent": intent.model_dump(mode="json", by_alias=True), + "status": plan_status.value, + "candidates": [item.model_dump(mode="json") for item in candidates], + "coverage": [item.model_dump(mode="json") for item in coverage], + "assurance_request": ( + assurance_request.model_dump(mode="json", by_alias=True) + if assurance_request is not None + else None + ), + "findings": [item.model_dump(mode="json") for item in findings], + "costs": SmartCostSummary( + inspected_entries=counters.entries, + inspected_json_files=counters.json_files, + inspected_json_bytes=counters.json_bytes, + ).model_dump(mode="json"), + "limitations": limitations, + } + digest = canonical_sha256({"domain": body["schema"], "body": body}) + return SmartPreflightPlan.model_validate( + {**body, "plan_id": f"smart_plan_{digest[:32]}", "plan_digest": digest} + ) + + +def concise_plan_summary(plan: SmartPreflightPlan) -> str: + selected = sum(item.selected for item in plan.candidates) + gaps = sum(item.status != CoverageStatus.COVERED for item in plan.coverage) + return ( + f"{plan.status.value} candidates={len(plan.candidates)} selected={selected} " + f"gaps={gaps} costly=none" + ) diff --git a/tests/test_assurance_bundle.py b/tests/test_assurance_bundle.py new file mode 100644 index 0000000..ad221c4 --- /dev/null +++ b/tests/test_assurance_bundle.py @@ -0,0 +1,438 @@ +"""Phase 6F Assurance Bundle vertical-slice tests.""" + +from __future__ import annotations + +import hashlib +import json +from pathlib import Path + +import pytest +from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey +from typer.testing import CliRunner + +from omiv.assurance.archive import pack_bundle +from omiv.assurance.models import ( + AssuranceBundleManifest, + AssuranceTrustPolicy, + BundleStatus, + PreflightStatus, +) +from omiv.assurance.operations import ( + build_bundle, + build_preflight, + load_request, + verify_bundle, +) +from omiv.assurance.registry import supported_schemas +from omiv.assurance.signatures import build_signature, verify_signatures, write_signature +from omiv.canonical import canonical_sha256 +from omiv.cli import app +from omiv.errors import OmivInputError + +ROOT = Path(__file__).parents[1] +KNOWN_SOURCE = "payload-integrity/comparisons/exact.json" +KNOWN_SCHEMA = "omiv.payload-manifest-comparison.v1" +runner = CliRunner() + + +def _request( + tmp_path: Path, requirements: list[dict], operations: list[dict] | None = None +) -> Path: + path = tmp_path / "request.json" + path.write_text( + json.dumps( + { + "schema": "omiv.assurance-request.v1", + "request_id": "unsloth-case-followup", + "subject": "synthetic Phase 6F interoperability test", + "requirements": requirements, + "planned_operations": operations or [], + } + ), + encoding="utf-8", + ) + return path + + +def _known(member_path: str = "evidence/payload-comparison.json") -> dict: + return { + "member_path": member_path, + "source_path": KNOWN_SOURCE, + "phase": "PHASE_6A", + "required": True, + "media_type": "application/json", + "expected_schema": KNOWN_SCHEMA, + "dimension": "STRUCTURE", + "verdict_role": "DIMENSION_VERDICT", + } + + +def test_tracked_offline_example_preflights_ready() -> None: + request = load_request(ROOT / "examples/assurance-bundle/request.json") + plan = build_preflight(request, ROOT) + assert plan.status == PreflightStatus.READY + assert not any((plan.costs.download, plan.costs.network, plan.costs.conversion, plan.costs.gpu)) + + +def test_tracked_conformance_requests_preserve_complete_and_incomplete_states() -> None: + valid = build_preflight( + load_request(ROOT / "fixtures/assurance-bundle/valid-request.json"), ROOT + ) + incomplete = build_preflight( + load_request(ROOT / "fixtures/assurance-bundle/incomplete-request.json"), ROOT + ) + assert valid.status == PreflightStatus.READY + assert incomplete.status == PreflightStatus.BLOCKED + assert incomplete.costs.gpu is True + + +def test_complete_plan_build_and_offline_verify(tmp_path: Path) -> None: + request = load_request(_request(tmp_path, [_known()])) + plan = build_preflight(request, ROOT) + assert plan.status == PreflightStatus.READY + assert plan.costs.network is False + assert plan.members[0].schema_support.value == "SUPPORTED_AND_VALID" + + bundle = tmp_path / "bundle" + manifest = build_bundle(plan, ROOT, bundle) + assert manifest.status == BundleStatus.COMPLETE + manifest_json = json.loads((bundle / "assurance-bundle.json").read_text()) + assert not any("source_path" in item for item in manifest_json["members"]) + assert {item["path"] for item in manifest_json["core_files"]} == { + "subject.json", + "verdict.json", + "evidence-index.json", + "findings.json", + "unknowns.json", + "capabilities.json", + } + verdict = json.loads((bundle / "verdict.json").read_text()) + assert verdict["overall"] == "PASS" + assert verdict["dimensions"][0]["dimension"] == "STRUCTURE" + + report = verify_bundle(bundle) + assert report.status == BundleStatus.COMPLETE + assert report.available == 1 + assert report.evidence["offline"] is True + + +def test_missing_unknown_and_costly_work_are_explicit_and_fail_closed(tmp_path: Path) -> None: + unknown = { + "member_path": "evidence/future.json", + "source_path": KNOWN_SOURCE, + "phase": "EXTERNAL", + "required": False, + "media_type": "application/json", + "expected_schema": "external.future-evidence.v1", + } + missing = { + "member_path": "evidence/missing.json", + "source_path": "not-present/missing.json", + "phase": "PHASE_6E", + "required": True, + "media_type": "application/json", + "expected_schema": "omiv.runtime-resolution-parity-evidence.v1", + } + request = load_request( + _request( + tmp_path, + [unknown, missing], + [ + {"cost_class": "NETWORK", "reason": "would collect missing evidence"}, + {"cost_class": "GPU", "reason": "would run compatibility probes"}, + ], + ) + ) + plan = build_preflight(request, ROOT) + assert plan.status == PreflightStatus.BLOCKED + assert plan.costs.network and plan.costs.gpu + assert {issue for item in plan.members for issue in item.issues} >= { + "EVIDENCE_SCHEMA_UNSUPPORTED_FOR_PHASE", + "REQUIRED_EVIDENCE_MISSING", + } + + bundle = tmp_path / "incomplete" + manifest = build_bundle(plan, ROOT, bundle) + assert manifest.status == BundleStatus.INCOMPLETE + report = verify_bundle(bundle) + assert report.status == BundleStatus.INCOMPLETE + assert report.missing == 1 + assert report.unknown == 1 + + +def test_tampering_and_untracked_files_are_invalid(tmp_path: Path) -> None: + plan = build_preflight(load_request(_request(tmp_path, [_known()])), ROOT) + bundle = tmp_path / "bundle" + build_bundle(plan, ROOT, bundle) + evidence = bundle / "evidence/payload-comparison.json" + evidence.write_text("{}", encoding="utf-8") + (bundle / "untracked.txt").write_text("unexpected", encoding="utf-8") + + report = verify_bundle(bundle) + assert report.status == BundleStatus.INVALID + assert {item.code for item in report.findings} >= { + "BUNDLE_FILE_SET_MISMATCH", + "EVIDENCE_INTEGRITY_MISMATCH", + } + + +def test_core_projection_tampering_is_invalid(tmp_path: Path) -> None: + plan = build_preflight(load_request(_request(tmp_path, [_known()])), ROOT) + bundle = tmp_path / "bundle" + build_bundle(plan, ROOT, bundle) + verdict_path = bundle / "verdict.json" + verdict = json.loads(verdict_path.read_text()) + verdict["overall"] = "FAIL" + verdict_path.write_text(json.dumps(verdict), encoding="utf-8") + assert verify_bundle(bundle).status == BundleStatus.INVALID + + +def test_v1_rejects_noncanonical_core_file_set(tmp_path: Path) -> None: + plan = build_preflight(load_request(_request(tmp_path, [_known()])), ROOT) + bundle = tmp_path / "bundle" + build_bundle(plan, ROOT, bundle) + manifest_path = bundle / "assurance-bundle.json" + raw = json.loads(manifest_path.read_text()) + raw["core_files"][0]["path"] = "alternate-subject.json" + (bundle / "subject.json").replace(bundle / "alternate-subject.json") + body = {key: value for key, value in raw.items() if key not in {"bundle_id", "bundle_digest"}} + digest = canonical_sha256({"domain": raw["schema"], "body": body}) + raw["bundle_id"] = f"assurance_bundle_{digest[:32]}" + raw["bundle_digest"] = digest + manifest_path.write_text(json.dumps(raw), encoding="utf-8") + + report = verify_bundle(bundle) + assert report.status == BundleStatus.INVALID + assert "CORE_FILE_SET_MISMATCH" in {item.code for item in report.findings} + + +def test_unsupported_required_feature_fails_closed(tmp_path: Path) -> None: + plan = build_preflight(load_request(_request(tmp_path, [_known()])), ROOT) + bundle = tmp_path / "bundle" + build_bundle(plan, ROOT, bundle) + manifest_path = bundle / "assurance-bundle.json" + raw = json.loads(manifest_path.read_text()) + raw["required_features"].append("future-feature") + body = {key: value for key, value in raw.items() if key not in {"bundle_id", "bundle_digest"}} + digest = canonical_sha256({"domain": raw["schema"], "body": body}) + raw["bundle_id"] = f"assurance_bundle_{digest[:32]}" + raw["bundle_digest"] = digest + manifest_path.write_text(json.dumps(raw), encoding="utf-8") + + with pytest.raises(OmivInputError, match="unsupported Assurance Bundle profile"): + verify_bundle(bundle) + + +def test_manifest_semantic_projection_must_reconstruct_from_evidence(tmp_path: Path) -> None: + plan = build_preflight(load_request(_request(tmp_path, [_known()])), ROOT) + bundle = tmp_path / "bundle" + build_bundle(plan, ROOT, bundle) + manifest_path = bundle / "assurance-bundle.json" + raw = json.loads(manifest_path.read_text()) + raw["members"][0]["semantic_status"] = "FAIL" + raw["members"][0]["semantic_summary"] = "forged projection" + body = {key: value for key, value in raw.items() if key not in {"bundle_id", "bundle_digest"}} + digest = canonical_sha256({"domain": raw["schema"], "body": body}) + raw["bundle_id"] = f"assurance_bundle_{digest[:32]}" + raw["bundle_digest"] = digest + AssuranceBundleManifest.model_validate(raw) + manifest_path.write_text(json.dumps(raw), encoding="utf-8") + + report = verify_bundle(bundle) + assert report.status == BundleStatus.INVALID + assert "SEMANTIC_PROJECTION_MISMATCH" in {item.code for item in report.findings} + + +def test_phase5_schema_registry_is_interoperable(tmp_path: Path) -> None: + assert "omiv.model-passport.v1" in supported_schemas() + requirement = { + "member_path": "provenance/passport.json", + "source_path": "passports/unsloth_Kimi-K3-GGUF_UD-IQ1_M.passport.json", + "phase": "PHASE_5", + "required": True, + "media_type": "application/json", + "expected_schema": "omiv.model-passport.v1", + "dimension": "PROVENANCE", + "verdict_role": "SUPPORTING", + } + plan = build_preflight(load_request(_request(tmp_path, [requirement])), ROOT) + assert plan.status == PreflightStatus.READY + assert plan.members[0].schema_support.value == "SUPPORTED_AND_VALID" + + +def test_phase7b_candidate_evidence_is_not_registered_in_phase6f() -> None: + assert "omiv.runtime-compatibility-evidence.v1" not in supported_schemas() + + +def test_deterministic_archive_and_offline_cli_verification(tmp_path: Path) -> None: + plan = build_preflight(load_request(_request(tmp_path, [_known()])), ROOT) + bundle = tmp_path / "bundle" + build_bundle(plan, ROOT, bundle) + first, second = tmp_path / "first.omiv", tmp_path / "second.omiv" + pack_bundle(bundle, first) + pack_bundle(bundle, second) + assert first.read_bytes() == second.read_bytes() + result = runner.invoke(app, ["assurance", "verify", "--bundle", str(first)]) + assert result.exit_code == 0, result.output + assert "COMPLETE" in result.stdout + product_result = runner.invoke(app, ["verify", str(first)]) + assert product_result.exit_code == 0, product_result.output + assert "COMPLETE" in product_result.stdout + + +def test_signature_integrity_and_external_trust_policy(tmp_path: Path) -> None: + plan = build_preflight(load_request(_request(tmp_path, [_known()])), ROOT) + bundle = tmp_path / "bundle" + build_bundle(plan, ROOT, bundle) + private = Ed25519PrivateKey.generate() + signature = build_signature(bundle, private, "test-key") + write_signature(bundle, signature) + policy = AssuranceTrustPolicy( + policy_id="assurance-policy.test.v1", + allowed_key_ids=["test-key"], + allowed_public_key_sha256=[hashlib.sha256(bytes.fromhex(signature.public_key)).hexdigest()], + ) + valid, trusted, issues = verify_signatures(bundle, policy) + assert (valid, trusted, issues) == (1, 1, []) + rejected = policy.model_copy(update={"allowed_key_ids": ["different-key"]}) + assert verify_signatures(bundle, rejected)[2] == ["TRUST_POLICY_MINIMUM_SIGNATURES_NOT_MET"] + + +def test_opaque_binary_members_are_streamed( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + source_root = tmp_path / "source" + source_root.mkdir() + artifact = source_root / "payload.bin" + artifact.write_bytes(b"streamed" * 200_000) + request = load_request( + _request( + tmp_path, + [ + { + "member_path": "provenance/payload.bin", + "source_path": "payload.bin", + "phase": "EXTERNAL", + "required": True, + "media_type": "application/octet-stream", + "expected_schema": None, + "dimension": "PROVENANCE", + "verdict_role": "SUPPORTING", + } + ], + ) + ) + original = Path.read_bytes + + def guarded_read(path: Path) -> bytes: + if path == artifact: + raise AssertionError("binary evidence must be streamed") + return original(path) + + monkeypatch.setattr(Path, "read_bytes", guarded_read) + plan = build_preflight(request, source_root) + bundle = tmp_path / "streamed-bundle" + build_bundle(plan, source_root, bundle) + assert verify_bundle(bundle).status == BundleStatus.COMPLETE + + +def test_build_detects_change_after_preflight(tmp_path: Path) -> None: + source_root = tmp_path / "source" + source_root.mkdir() + source = source_root / "evidence.json" + source.write_bytes((ROOT / KNOWN_SOURCE).read_bytes()) + request = load_request( + _request( + tmp_path, + [ + { + **_known(), + "source_path": "evidence.json", + } + ], + ) + ) + plan = build_preflight(request, source_root) + source.write_text("{}", encoding="utf-8") + with pytest.raises(OmivInputError, match="changed after preflight"): + build_bundle(plan, source_root, tmp_path / "bundle") + + +def test_request_rejects_traversal_and_portable_collisions(tmp_path: Path) -> None: + with pytest.raises(OmivInputError): + load_request(_request(tmp_path, [{**_known(), "source_path": "../secret.json"}])) + with pytest.raises(OmivInputError): + load_request(_request(tmp_path, [_known("Evidence/a.json"), _known("evidence/A.json")])) + with pytest.raises(OmivInputError, match="opaque binary"): + load_request( + _request( + tmp_path, + [ + { + **_known(), + "media_type": "application/octet-stream", + "expected_schema": None, + } + ], + ) + ) + + +def test_bundle_symlink_fails_closed_when_platform_supports_it(tmp_path: Path) -> None: + plan = build_preflight(load_request(_request(tmp_path, [_known()])), ROOT) + bundle = tmp_path / "bundle" + build_bundle(plan, ROOT, bundle) + link = bundle / "broken-link" + try: + link.symlink_to(bundle / "absent") + except OSError: + pytest.skip("symlinks unavailable") + report = verify_bundle(bundle) + assert report.status == BundleStatus.INVALID + assert "BUNDLE_SYMLINK_PRESENT" in {item.code for item in report.findings} + + +def test_assurance_slice_has_no_network_process_or_gpu_execution_surface() -> None: + source = (ROOT / "src/omiv/assurance/operations.py").read_text(encoding="utf-8") + for forbidden in ("subprocess", "socket", "urllib", "requests", "httpx", "cuda"): + assert f"import {forbidden}" not in source + assert "collect_" not in source + assert "run_conversion" not in source + + +def test_cli_is_concise_but_retains_machine_reports(tmp_path: Path) -> None: + request = _request(tmp_path, [_known()]) + plan = tmp_path / "plan.json" + planned = runner.invoke( + app, + [ + "assurance", + "plan", + "--request", + str(request), + "--root", + str(ROOT), + "--output", + str(plan), + ], + ) + assert planned.exit_code == 0 + assert planned.stdout.startswith("READY members=1 missing=0 unknown=0 costly=none") + assert json.loads(plan.read_text())["schema"] == "omiv.assurance-plan.v1" + + bundle = tmp_path / "bundle" + built = runner.invoke( + app, + ["assurance", "build", "--plan", str(plan), "--root", str(ROOT), "--output", str(bundle)], + ) + assert built.exit_code == 0 + assert built.stdout.startswith("COMPLETE bundle=assurance_bundle_") + + report = tmp_path / "verification.json" + verified = runner.invoke( + app, + ["assurance", "verify", "--bundle", str(bundle), "--report-output", str(report)], + ) + assert verified.exit_code == 0 + assert verified.stdout.startswith("COMPLETE bundle=assurance_bundle_") + assert json.loads(report.read_text())["evidence"]["offline"] is True diff --git a/tests/test_conversion_capture.py b/tests/test_conversion_capture.py index b2dc76f..2d5789f 100644 --- a/tests/test_conversion_capture.py +++ b/tests/test_conversion_capture.py @@ -3,8 +3,10 @@ import hashlib import importlib.metadata import json +import os import platform import subprocess +import sys from pathlib import Path from typing import Any @@ -21,6 +23,12 @@ from omiv.provenance.models import InventoryEvidence, ObservedArtifact +@pytest.fixture(autouse=True) +def _current_python_on_path(monkeypatch: pytest.MonkeyPatch) -> None: + python_directory = str(Path(sys.executable).parent) + monkeypatch.setenv("PATH", python_directory + os.pathsep + os.environ.get("PATH", "")) + + def _git(repository: Path, *arguments: str) -> str: result = subprocess.run( ["git", "-C", str(repository), *arguments], @@ -122,7 +130,7 @@ def _spec( "repository_dir": str(repository), "expected_revision": revision, "require_clean_worktree": True, - "executable": "python", + "executable": Path(sys.executable).name, "entrypoint": "convert.py", }, "invocation": {"arguments": arguments}, diff --git a/tests/test_github_publication_controls.py b/tests/test_github_publication_controls.py index e88bc6c..c8897f4 100644 --- a/tests/test_github_publication_controls.py +++ b/tests/test_github_publication_controls.py @@ -386,12 +386,12 @@ def test_release_states_make_no_publication_claim() -> None: assert _policy()["repository"]["required_pre_public_visibility"] == "PRIVATE" -def test_roadmap_truth_preserves_r1f_and_phase6f_boundaries() -> None: +def test_roadmap_truth_preserves_r1f_phase6f_and_phase7_boundaries() -> None: roadmap = (ROOT / "docs/roadmap.md").read_text(encoding="utf-8") assert "R1D offline walkthrough | COMPLETE" in roadmap assert "R1E GitHub metadata/security | COMPLETE" in roadmap assert "R1F final publication audit | COMPLETE; public controls verified" in roadmap - assert "Phase 6F | PLANNED, NOT IMPLEMENTED" in roadmap + assert "Phase 6F | COMPLETE; NOT RELEASED" in roadmap assert "Phase 7 | FUTURE, SCOPE NOT FROZEN" in roadmap @@ -591,7 +591,10 @@ def test_security_route_to_public_issue_fails_closed(tmp_path: Path) -> None: "R1F final publication audit | COMPLETE; public controls verified", "R1F final publication audit | COMPLETE", ), - ("Phase 6F | PLANNED, NOT IMPLEMENTED", "Phase 6F | COMPLETE"), + ( + "Phase 6F | COMPLETE; NOT RELEASED", + "Phase 6F | RELEASED", + ), ), ) def test_roadmap_overclaim_mutations_fail_closed(tmp_path: Path, old: str, new: str) -> None: diff --git a/tests/test_offline_evidence_walkthrough.py b/tests/test_offline_evidence_walkthrough.py index 18b782d..08397f8 100644 --- a/tests/test_offline_evidence_walkthrough.py +++ b/tests/test_offline_evidence_walkthrough.py @@ -378,16 +378,13 @@ def test_markdown_links_and_local_fragments_resolve() -> None: assert unquote(target.fragment).lower() in anchors -def test_phase6f_and_public_commercial_boundary_remain_unchanged() -> None: +def test_phase6f_and_public_commercial_boundary_remain_narrow() -> None: roadmap = (ROOT / "docs/roadmap.md").read_text(encoding="utf-8") - assert "| Phase 6F | PLANNED, NOT IMPLEMENTED |" in roadmap + assert "| Phase 6F | COMPLETE; NOT RELEASED |" in roadmap assert "| Phase 7 | FUTURE, SCOPE NOT FROZEN |" in roadmap - result = subprocess.run( - ["git", "diff", "--quiet", BASELINE, "--", "docs/public-commercial-boundary.md"], - cwd=ROOT, - check=False, - ) - assert result.returncode == 0 + boundary = (ROOT / "docs/public-commercial-boundary.md").read_text(encoding="utf-8") + assert "portable public\nverification boundary" in boundary + assert "without claiming a hosted service" in boundary def test_audit_is_deterministic_and_passes() -> None: diff --git a/tests/test_public_launch_ux.py b/tests/test_public_launch_ux.py index 9a9fe1b..b2fe067 100644 --- a/tests/test_public_launch_ux.py +++ b/tests/test_public_launch_ux.py @@ -174,7 +174,7 @@ def test_quickstart_limitations_and_roadmap_state_are_independent() -> None: "Phase 6C | RELEASED", "Phase 6D | RELEASED", "Phase 6E | RELEASED", - "Phase 6F | PLANNED, NOT IMPLEMENTED", + "Phase 6F | COMPLETE; NOT RELEASED", "Phase 7 | FUTURE, SCOPE NOT FROZEN", "R1C launch UX | COMPLETE", ): diff --git a/tests/test_r1f_publication_readiness.py b/tests/test_r1f_publication_readiness.py index 07fba40..2ddd552 100644 --- a/tests/test_r1f_publication_readiness.py +++ b/tests/test_r1f_publication_readiness.py @@ -439,7 +439,7 @@ def test_roadmap_preserves_release_and_engineering_boundaries() -> None: ): assert f"{release} | COMPLETE" in roadmap assert "R1F final publication audit | COMPLETE; public controls verified" in roadmap - assert "Phase 6F | PLANNED, NOT IMPLEMENTED" in roadmap + assert "Phase 6F | COMPLETE; NOT RELEASED" in roadmap assert "Phase 7 | FUTURE, SCOPE NOT FROZEN" in roadmap diff --git a/tests/test_reference_preflight.py b/tests/test_reference_preflight.py new file mode 100644 index 0000000..e4b5011 --- /dev/null +++ b/tests/test_reference_preflight.py @@ -0,0 +1,447 @@ +"""Muse Glimmer acceptance tests for provider-neutral Reference Preflight.""" + +from __future__ import annotations + +import base64 +import hashlib +import json +import struct +import zlib +from pathlib import Path + +import pytest +from typer.testing import CliRunner + +from omiv.assurance.archive import pack_bundle +from omiv.assurance.models import ( + BundleStatus, + PreflightStatus, + SchemaSupport, + VerdictRole, +) +from omiv.assurance.operations import build_bundle, build_preflight, verify_bundle +from omiv.cli import app +from omiv.reference_preflight.models import ( + ArtifactRole, + ReferencePreflightEvidence, + ReferencePreflightProfile, +) +from omiv.reference_preflight.operations import ( + build_assurance_request, + build_reference_preflight, + concise_evidence_summary, + load_profile, + write_evidence, +) +from omiv.reference_preflight.probes import ( + PNG_HEIGHT, + PNG_WIDTH, + RED_SQUARE_END, + RED_SQUARE_START, + build_red_square_png, +) +from omiv.runtime_compatibility.models import ( + BoundedCapture, + EnvironmentVariable, + ProcessCapture, + StageStatus, + derive_native_stage_results, +) + +ROOT = Path(__file__).parents[1] +PROFILE_PATH = ROOT / "fixtures/reference-preflight/muse-glimmer-30b.json" +IMAGE_FIXTURE_PATH = ROOT / "examples/reference-preflight/probes/red-square.png" +runner = CliRunner() + + +def _profile() -> ReferencePreflightProfile: + return load_profile(PROFILE_PATH) + + +def _evidence(reference: str = "meta-models/Muse-Glimmer-30B-GGUF") -> ReferencePreflightEvidence: + return build_reference_preflight(_profile(), reference) + + +def test_muse_offline_replay_separates_every_evidence_boundary() -> None: + first = _evidence() + second = _evidence() + + assert first == second + assert first.resolved_identity == ( + "meta-models/Muse-Glimmer-30B-GGUF@43c7eadd41352a299ea8e0a36b3157978dd63596" + ) + assert first.resolved_identity_status == "REMOTE_REVISION_PINNED" + assert first.payload_verification == "NOT_DOWNLOADED" + assert first.source_binding == "NOT_ESTABLISHED" + assert first.architecture_status == "DECLARED" + assert first.architecture == "muse-glimmer" + assert first.tokenizer_configuration == "REMOTE_METADATA_OBSERVED" + assert first.runtime_probe == "NOT_RUN" + assert first.numerical_fidelity == "NOT_EVALUATED" + assert first.semantic_fidelity == "NOT_EVALUATED" + assert first.performance == "NOT_EVALUATED" + assert first.safety == "NOT_EVALUATED" + assert first.production_readiness == "NOT_ESTABLISHED" + + artifacts = {item.role: item for item in first.profile.artifacts} + assert set(artifacts) == { + ArtifactRole.MAIN_MODEL, + ArtifactRole.PERCEPTION_ENCODER, + ArtifactRole.DRAFTER, + } + assert all(item.payload_verification == "NOT_DOWNLOADED" for item in artifacts.values()) + assert artifacts[ArtifactRole.MAIN_MODEL].provider_identity == ( + "4cc57c0f51040a226e5a72cc47b7613f7772950e460a665f7083de89f183f60e" + ) + assert all( + item.cryptographic_binding == "NOT_ESTABLISHED" for item in first.profile.relationships + ) + + llama, ollama = first.profile.runtime_candidates + assert llama.exact_release == "b10353" + assert llama.exact_commit == "f8def7fe168bab245fbf15d3f18b26dbb1ef73c8" + assert llama.support_commit == "62bf73d25c53b8161f8a22894d4f90c4aebbd7d0" + assert ollama.compatibility_status == "DECLARED_NOT_PROBED" + registry = first.profile.registry_observations[0] + assert registry.mutable_tag is True + assert registry.manifest_digest == ( + "de878ce33ad81d060001db1469a02eebe4d86f0ad58cfe52dc062fdcbe4464c1" + ) + assert registry.runtime_probe == "NOT_RUN" + assert {layer.role for layer in registry.layers} == {"model", "projector", "params"} + + +def test_concise_verdict_is_a_deterministic_twelve_line_projection() -> None: + evidence = _evidence() + first = concise_evidence_summary(evidence) + second = concise_evidence_summary(_evidence()) + + assert first == second + assert len(first.splitlines()) == 12 + assert first.splitlines() == [ + "Artifact reference meta-models/Muse-Glimmer-30B-GGUF", + "Resolved identity REMOTE_REVISION_PINNED: " + "meta-models/Muse-Glimmer-30B-GGUF@" + "43c7eadd41352a299ea8e0a36b3157978dd63596", + "Payload verification NOT_DOWNLOADED", + "Source binding NOT_ESTABLISHED", + "Architecture DECLARED: muse-glimmer", + "Tokenizer/config REMOTE_METADATA_OBSERVED", + "Companion artifacts DECLARED: DRAFTER + PERCEPTION_ENCODER", + "Runtime candidates llama.cpp / Ollama", + "Runtime requirement DECLARED_OFFICIAL_REQUIREMENT: llama.cpp >= b10353; " + "acceptance plan pins b10353 exactly", + "Runtime probe NOT_RUN", + "Provenance gaps EXPLICIT: 6", + f"Next step READY_FOR_GPU: execute {evidence.future_runtime_plan.plan_id}", + ] + + +@pytest.mark.parametrize( + "reference", + [ + "meta-models/Muse-Glimmer-30B-GGUF", + "https://huggingface.co/meta-models/Muse-Glimmer-30B-GGUF/", + "hf://meta-models/Muse-Glimmer-30B-GGUF", + ], +) +def test_reference_aliases_normalize_without_becoming_resolved_identity(reference: str) -> None: + evidence = _evidence(reference) + + assert evidence.input_reference == reference + assert evidence.artifact_reference == "meta-models/Muse-Glimmer-30B-GGUF" + assert evidence.resolved_identity.endswith("@43c7eadd41352a299ea8e0a36b3157978dd63596") + + +def test_unknown_reference_fails_without_inference() -> None: + with pytest.raises(ValueError, match="not covered"): + _evidence("somewhere-else/Muse-Glimmer-30B-GGUF") + + +def test_cli_replay_and_machine_evidence_are_byte_deterministic(tmp_path: Path) -> None: + first = tmp_path / "first.json" + second = tmp_path / "second.json" + request = tmp_path / "request.json" + arguments = [ + "reference-preflight", + "plan", + "--profile", + str(PROFILE_PATH), + "--reference", + "meta-models/Muse-Glimmer-30B-GGUF", + "--root", + str(tmp_path), + ] + first_result = runner.invoke( + app, + [*arguments, "--output", str(first), "--assurance-request-output", str(request)], + ) + second_result = runner.invoke(app, [*arguments, "--output", str(second)]) + + assert first_result.exit_code == 0, first_result.output + assert second_result.exit_code == 0, second_result.output + assert first_result.stdout == second_result.stdout + assert first.read_bytes() == second.read_bytes() + assert ( + json.loads(request.read_text(encoding="utf-8"))["planned_operations"][0]["cost_class"] + == "GPU" + ) + + verified = runner.invoke(app, ["reference-preflight", "verify", "--evidence", str(first)]) + assert verified.exit_code == 0, verified.output + assert verified.stdout == first_result.stdout + + +def test_phase6f_bundle_preserves_candidate_as_supporting_evidence(tmp_path: Path) -> None: + evidence = _evidence() + evidence_path = tmp_path / "muse-reference-evidence.json" + write_evidence(evidence, evidence_path) + request = build_assurance_request(evidence, evidence_path.name) + + plan = build_preflight(request, tmp_path) + assert request.request_id == "reference-588e595d7c9c9c71c40a3ca8" + assert plan.plan_id == "assurance_plan_dac49586cd0838c76bd3af9baf7c2d76" + assert plan.plan_digest == ( + "dac49586cd0838c76bd3af9baf7c2d76e49b5ba9186955ceb3157038c96ea998" + ) + assert plan.status == PreflightStatus.REVIEW_REQUIRED + assert plan.costs.gpu is True + assert plan.members[0].schema_support == SchemaSupport.NOT_APPLICABLE + assert plan.members[0].verdict_role == VerdictRole.SUPPORTING + assert plan.members[0].semantic_status == "NOT_TESTED" + + bundle = tmp_path / "bundle" + manifest = build_bundle(plan, tmp_path, bundle) + report = verify_bundle(bundle) + verdict = json.loads((bundle / "verdict.json").read_text(encoding="utf-8")) + bundled = next((bundle / "provenance").glob("reference-preflight-*.json")) + + assert manifest.status == BundleStatus.COMPLETE + assert manifest.bundle_id == "assurance_bundle_21d17ccfae39ef3d219c83d6926bdf3a" + assert manifest.bundle_digest == ( + "21d17ccfae39ef3d219c83d6926bdf3a9db8d05bffa078bc432a2d0647d9a4e1" + ) + assert report.status == BundleStatus.COMPLETE + assert report.evidence["phase_boundaries_preserved"] is True + assert verdict["overall"] == "UNKNOWN" + assert verdict["verdict_id"] == "assurance_verdict_16ebe37bc2cac6b22daae8cca0c7261f" + assert verdict["verdict_digest"] == ( + "16ebe37bc2cac6b22daae8cca0c7261f78a2a7a0b14a9bacfc8fff2f87609d17" + ) + assert verdict["dimensions"][0]["status"] == "NOT_TESTED" + assert bundled.read_bytes() == evidence_path.read_bytes() + assert evidence.payload_verification == "NOT_DOWNLOADED" + assert evidence.runtime_probe == "NOT_RUN" + + first_archive = tmp_path / "first.omiv" + second_archive = tmp_path / "second.omiv" + pack_bundle(bundle, first_archive) + pack_bundle(bundle, second_archive) + assert first_archive.read_bytes() == second_archive.read_bytes() + + +def test_complete_future_plan_is_canonical_and_unexecuted() -> None: + evidence = _evidence() + plan = evidence.future_runtime_plan + + assert plan.execution_status == "NOT_RUN" + assert plan.unknown_stages == ["LOAD", "TOKENIZER", "PREFILL", "DECODE", "OUTPUT"] + assert plan.dflash_observations == ["DISABLED", "ENABLED"] + assert plan.preferred_gpu_class == "NVIDIA RTX 5090 (32 GB VRAM)" + assert plan.maximum_gpu_count == 1 + assert plan.maximum_pod_count == 1 + assert plan.maximum_total_authorized_compute_usd == 5 + assert plan.maximum_working_duration_seconds == 4 * 60 * 60 + assert plan.stop_new_work_before_deadline_seconds == 30 * 60 + assert plan.artifact_payload_bytes == 19_788_220_960 + assert plan.direct_llama_cpp_workspace_bytes == 50_000_000_000 + assert plan.llama_cpp_plus_ollama_workspace_bytes == 80_000_000_000 + assert plan.image_fixture == "examples/reference-preflight/probes/red-square.png" + assert plan.image_fixture_sha256 == ( + "53bf31df09c932233812a2c7b61c89a0ecbbaee90ab63f36058099e5d008e852" + ) + assert plan.ollama_role == "SECONDARY_RUNTIME_OBSERVATION" + assert plan.ollama_mutable_tags == ["muse-glimmer:30b", "muse-glimmer:latest"] + assert plan.direct_llama_cpp_independent_of_ollama is True + assert "immediately before any future GPU execution" in plan.ollama_observation + assert "abort on manifest drift unless a new plan is reviewed" in plan.ollama_observation + assert "exact authorized Pod" in plan.pod_termination_requirement + assert "verify that exact Pod is terminated" in plan.pod_termination_requirement + assert any("GPU identity and backend observation" in item for item in plan.evidence_to_retain) + assert any("exit codes, timeout state" in item for item in plan.evidence_to_retain) + assert any("additional GPU or Pod" in item for item in plan.stop_conditions) + assert any( + "No performance, fidelity, safety or production-readiness claim" in item + for item in plan.non_claims + ) + assert {item.role for item in plan.artifacts} == { + ArtifactRole.MAIN_MODEL, + ArtifactRole.PERCEPTION_ENCODER, + ArtifactRole.DRAFTER, + } + assert all(item.downloaded is False for item in plan.artifacts) + assert sum(item.declared_size for item in plan.artifacts) == plan.artifact_payload_bytes + assert plan.plan_id == "reference_runtime_plan_97626690850636d5dd0276f9844742c6" + assert plan.plan_digest == ( + "97626690850636d5dd0276f9844742c6eb164a090bebed03b70a0246271f3a48" + ) + assert evidence.profile_digest == ( + "ea5f5c792e27e8bba080fbd2a406e4629db3acb06e5d588f03ff3804e81047c3" + ) + assert evidence.evidence_id == ( + "reference_preflight_evidence_588e595d7c9c9c71c40a3ca88822195f" + ) + assert evidence.evidence_digest == ( + "588e595d7c9c9c71c40a3ca88822195f94c7dfcb18d0e55ede45af5332760c77" + ) + + raw = evidence.model_dump(mode="json", by_alias=True) + raw["future_runtime_plan"]["question"] = "coherently replaced only outside the plan" + with pytest.raises(ValueError, match="future runtime plan canonical identity mismatch"): + ReferencePreflightEvidence.model_validate(raw) + + +def test_runtime_image_fixture_is_a_fixed_png_with_one_red_square() -> None: + fixture = IMAGE_FIXTURE_PATH.read_bytes() + + assert fixture == build_red_square_png() + assert len(fixture) == 4246 + assert hashlib.sha256(fixture).hexdigest() == ( + "53bf31df09c932233812a2c7b61c89a0ecbbaee90ab63f36058099e5d008e852" + ) + assert fixture.startswith(b"\x89PNG\r\n\x1a\n") + + chunks: dict[bytes, bytes] = {} + offset = 8 + while offset < len(fixture): + length = struct.unpack(">I", fixture[offset : offset + 4])[0] + kind = fixture[offset + 4 : offset + 8] + data = fixture[offset + 8 : offset + 8 + length] + chunks[kind] = data + offset += length + 12 + + width, height, bit_depth, color_type, compression, filtering, interlace = struct.unpack( + ">IIBBBBB", chunks[b"IHDR"] + ) + assert (width, height) == (PNG_WIDTH, PNG_HEIGHT) == (64, 64) + assert (bit_depth, color_type, compression, filtering, interlace) == (8, 3, 0, 0, 0) + assert chunks[b"PLTE"] == bytes((255, 255, 255, 212, 0, 0)) + + scanlines = zlib.decompress(chunks[b"IDAT"]) + assert len(scanlines) == PNG_HEIGHT * (PNG_WIDTH + 1) + for y in range(PNG_HEIGHT): + row = scanlines[y * (PNG_WIDTH + 1) : (y + 1) * (PNG_WIDTH + 1)] + assert row[0] == 0 + for x, palette_index in enumerate(row[1:]): + expected = int( + RED_SQUARE_START <= x < RED_SQUARE_END + and RED_SQUARE_START <= y < RED_SQUARE_END + ) + assert palette_index == expected + + +def test_future_plan_cannot_drift_from_observed_pins_or_bound_profile() -> None: + profile_raw = _profile().model_dump(mode="json", by_alias=True) + profile_raw["future_runtime_plan"]["artifacts"][0]["provider_identity"] = "0" * 64 + with pytest.raises(ValueError, match="artifact pin differs"): + ReferencePreflightProfile.model_validate(profile_raw) + + evidence = _evidence() + evidence_raw = evidence.model_dump(mode="json", by_alias=True) + plan = evidence_raw["future_runtime_plan"] + plan["question"] = "A different, coherently canonicalized future question" + plan_body = {key: value for key, value in plan.items() if key not in {"plan_id", "plan_digest"}} + digest = hashlib.sha256( + json.dumps( + {"body": plan_body, "domain": plan["schema"]}, + ensure_ascii=False, + separators=(",", ":"), + sort_keys=True, + ).encode("utf-8") + ).hexdigest() + plan["plan_digest"] = digest + plan["plan_id"] = f"reference_runtime_plan_{digest[:32]}" + evidence_body = { + key: value + for key, value in evidence_raw.items() + if key not in {"evidence_id", "evidence_digest"} + } + evidence_digest = hashlib.sha256( + json.dumps( + {"body": evidence_body, "domain": evidence_raw["schema"]}, + ensure_ascii=False, + separators=(",", ":"), + sort_keys=True, + ).encode("utf-8") + ).hexdigest() + evidence_raw["evidence_digest"] = evidence_digest + evidence_raw["evidence_id"] = f"reference_preflight_evidence_{evidence_digest[:32]}" + with pytest.raises(ValueError, match="differs from the bound profile plan"): + ReferencePreflightEvidence.model_validate(evidence_raw) + + +def _capture(raw: bytes) -> BoundedCapture: + return BoundedCapture( + captured_base64=base64.b64encode(raw).decode("ascii"), + captured_bytes=len(raw), + limit_bytes=4096, + overflow=False, + captured_sha256=hashlib.sha256(raw).hexdigest(), + ) + + +def test_phase7b1_self_authored_pass_declaration_remains_conservative() -> None: + declared = json.dumps( + { + "schema": "omiv.runtime-compatibility-runner-report.v1", + "status": "PASS", + "stages": [ + {"stage": stage, "status": "PASS"} + for stage in ("LOAD", "TOKENIZER", "PREFILL", "DECODE", "OUTPUT") + ], + }, + sort_keys=True, + ) + capture = ProcessCapture( + arguments=["{runtime_executable}"], + environment=[EnvironmentVariable(name="PATH", value="/usr/bin:/bin")], + return_code=0, + timed_out=False, + duration_ms=1, + stdout=_capture(declared.encode("utf-8")), + stderr=_capture(b""), + ) + + stages, issues = derive_native_stage_results(capture, declared) + + assert issues == [] + assert [item.status for item in stages] == [ + StageStatus.UNKNOWN, + StageStatus.UNKNOWN, + StageStatus.UNKNOWN, + StageStatus.UNKNOWN, + StageStatus.PASS, + ] + + +def test_generic_engine_has_no_muse_branch_and_accepts_another_profile() -> None: + profile = _profile() + raw = profile.model_dump(mode="json", by_alias=True) + raw["profile_name"] = "Provider-neutral synthetic reference" + raw["canonical_reference"] = "provider.example/model" + raw["reference_aliases"] = ["provider.example/model"] + raw["resolved_revision"] = "revision-001" + raw["configuration"]["architecture"] = "example-architecture" + synthetic = ReferencePreflightProfile.model_validate(raw) + + evidence = build_reference_preflight(synthetic, "provider.example/model") + source = ( + (ROOT / "src/omiv/reference_preflight/operations.py").read_text(encoding="utf-8").lower() + ) + + assert evidence.artifact_reference == "provider.example/model" + assert evidence.resolved_identity == "provider.example/model@revision-001" + assert evidence.architecture == "example-architecture" + assert "muse" not in source + for forbidden in ("subprocess", "socket", "urllib", "requests", "httpx"): + assert f"import {forbidden}" not in source diff --git a/tests/test_runtime_compatibility.py b/tests/test_runtime_compatibility.py new file mode 100644 index 0000000..2d74cf9 --- /dev/null +++ b/tests/test_runtime_compatibility.py @@ -0,0 +1,586 @@ +"""Candidate Phase 7B local runtime compatibility profile tests.""" + +from __future__ import annotations + +import hashlib +import json +import os +from pathlib import Path + +import pytest +from typer.testing import CliRunner + +from omiv.canonical import canonical_sha256 +from omiv.cli import app +from omiv.errors import OmivInputError +from omiv.runtime_compatibility.models import ( + PROFILE_ID, + CompatibilityStatus, + PlanStatus, + RuntimeCompatibilityLimits, + RuntimeCompatibilityRequest, + RuntimeTestVector, + StageStatus, +) +from omiv.runtime_compatibility.operations import ( + build_plan, + execute_plan, + load_evidence, + load_request, +) + +ROOT = Path(__file__).parents[1] +runner = CliRunner() +VERSION = "llama.cpp synthetic-runner 1.0" + + +def _digest(path: Path) -> str: + return hashlib.sha256(path.read_bytes()).hexdigest() + + +def _rehash_evidence(value: dict[str, object]) -> None: + body = { + key: item + for key, item in value.items() + if key not in {"evidence_id", "evidence_digest"} + } + digest = canonical_sha256({"domain": value["schema"], "body": body}) + value["evidence_id"] = f"runtime_compat_evidence_{digest[:32]}" + value["evidence_digest"] = digest + + +def _script(tmp_path: Path, run_body: str) -> Path: + path = tmp_path / "explicit-runner.py" + path.write_text( + "#!/usr/bin/env python3\n" + "import json\n" + "import sys\n" + "import time\n" + "from pathlib import Path\n" + f"VERSION = {VERSION!r}\n" + "if sys.argv[1:] == ['--version']:\n" + " print(VERSION)\n" + " raise SystemExit(0)\n" + f"{run_body}\n", + encoding="utf-8", + ) + path.chmod(0o755) + return path + + +def _request(tmp_path: Path, executable: Path, artifact: Path) -> RuntimeCompatibilityRequest: + return RuntimeCompatibilityRequest( + request_id="phase7b-test", + executable_path=executable.relative_to(tmp_path).as_posix(), + executable_sha256=_digest(executable), + expected_runtime_version=VERSION, + artifact_path=artifact.relative_to(tmp_path).as_posix(), + artifact_sha256=_digest(artifact), + test_vector=RuntimeTestVector( + vector_id="vector-v1", + prompt="The capital of France is", + expected_output="Paris", + max_generated_tokens=8, + ), + limits=RuntimeCompatibilityLimits( + timeout_seconds=1, + version_timeout_seconds=1, + max_stdout_bytes=1024, + max_stderr_bytes=1024, + max_executable_bytes=64 * 1024, + max_artifact_bytes=1024, + max_work_files=4, + max_work_file_bytes=1024, + max_work_total_bytes=2048, + ), + ) + + +def _fixture( + tmp_path: Path, run_body_factory: object | None = None +) -> tuple[RuntimeCompatibilityRequest, Path, Path]: + artifact = tmp_path / "synthetic.gguf" + artifact.write_bytes(b"synthetic artifact, not a model\n") + body = "sys.stdout.write('Paris')" + if isinstance(run_body_factory, str): + body = run_body_factory + executable = _script(tmp_path, body) + return _request(tmp_path, executable, artifact), executable, artifact + + +def test_tracked_native_output_probe_retains_raw_observation_and_unknown_internals() -> None: + request = load_request(ROOT / "examples/runtime-compatibility/request.json") + plan = build_plan(request, ROOT) + assert plan.status == PlanStatus.READY + assert plan.invocation.shell is False + assert plan.invocation.accelerator == "CPU_ONLY" + assert plan.costs.network is False + assert plan.costs.download is False + assert plan.costs.compilation is False + assert plan.costs.conversion is False + assert plan.costs.runtime_execution is True + assert plan.costs.gpu is False + + evidence = execute_plan(plan, ROOT) + + assert evidence.status == CompatibilityStatus.NOT_VERIFIED + assert [item.status for item in evidence.stages] == [ + StageStatus.UNKNOWN, + StageStatus.UNKNOWN, + StageStatus.UNKNOWN, + StageStatus.UNKNOWN, + StageStatus.PASS, + ] + assert evidence.runtime.execution_sha256 == request.executable_sha256 + assert evidence.runtime.observed_version == request.expected_runtime_version + assert evidence.artifact.execution_sha256 == request.artifact_sha256 + assert evidence.compatibility_execution is not None + assert evidence.compatibility_execution.stdout.captured_bytes <= ( + request.limits.max_stdout_bytes + ) + assert len(evidence.unknowns) == 4 + assert evidence.plan.plan_id == plan.plan_id + assert evidence.plan.plan_digest == plan.plan_digest + assert evidence.compatibility_execution.arguments == plan.invocation.run_arguments + assert evidence.compatibility_execution.environment == plan.invocation.environment + assert not any(argument.startswith("--omiv-") for argument in plan.invocation.run_arguments) + + +def test_cli_vertical_slice_writes_evidence_and_concise_stage_summary(tmp_path: Path) -> None: + plan_path = tmp_path / "plan.json" + evidence_path = tmp_path / "evidence.json" + planned = runner.invoke( + app, + [ + "runtime-compat", + "plan", + "--request", + str(ROOT / "examples/runtime-compatibility/request.json"), + "--root", + str(ROOT), + "--output", + str(plan_path), + ], + ) + assert planned.exit_code == 0, planned.output + assert "executable_present=yes executable=yes executable_pinned=yes" in planned.stdout + assert "artifact_present=yes artifact_pinned=yes" in planned.stdout + assert "network=no download=no compilation=no conversion=no runtime=yes gpu=no" in ( + planned.stdout + ) + + executed = runner.invoke( + app, + [ + "runtime-compat", + "run", + "--plan", + str(plan_path), + "--root", + str(ROOT), + "--output", + str(evidence_path), + ], + ) + assert executed.exit_code == 1, executed.output + assert "llama.cpp\nLOAD UNKNOWN\nTOKENIZER UNKNOWN" in executed.stdout + assert "OUTPUT PASS" in executed.stdout + assert "Runtime compatibility: NOT_VERIFIED" in executed.stdout + assert load_evidence(evidence_path).candidate_notice == ( + "PHASE_7B_CANDIDATE_PHASE_7_UNFROZEN" + ) + + verified = runner.invoke( + app, + ["runtime-compat", "verify", "--evidence", str(evidence_path)], + ) + assert verified.exit_code == 1, verified.output + + +def test_plan_is_read_only_and_never_executes_the_supplied_binary( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + request, _executable, _artifact = _fixture(tmp_path) + + def forbidden(*_args: object, **_kwargs: object) -> None: + raise AssertionError("preflight must not start a process") + + monkeypatch.setattr("subprocess.Popen", forbidden) + before = sorted(path.name for path in tmp_path.iterdir()) + plan = build_plan(request, tmp_path) + assert plan.status == PlanStatus.READY + assert sorted(path.name for path in tmp_path.iterdir()) == before + + +def test_missing_explicit_executable_blocks_without_path_discovery( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + path_directory = tmp_path / "path-bin" + path_directory.mkdir() + path_candidate = _script(path_directory, "sys.stdout.write('should not run')") + path_candidate.rename(path_directory / "missing-runner") + monkeypatch.setenv("PATH", str(path_directory)) + artifact = tmp_path / "synthetic.gguf" + artifact.write_bytes(b"synthetic\n") + request = RuntimeCompatibilityRequest( + request_id="missing-runtime", + executable_path="missing-runner", + executable_sha256="0" * 64, + expected_runtime_version=VERSION, + artifact_path=artifact.name, + artifact_sha256=_digest(artifact), + test_vector=RuntimeTestVector( + vector_id="vector-v1", prompt="prompt", expected_output="output" + ), + ) + plan = build_plan(request, tmp_path) + assert plan.status == PlanStatus.BLOCKED + assert plan.executable.observed_sha256 is None + assert any("missing" in item.lower() for item in plan.missing) + + +def test_changed_artifact_after_preflight_is_not_executed(tmp_path: Path) -> None: + request, _executable, artifact = _fixture(tmp_path) + plan = build_plan(request, tmp_path) + artifact.write_bytes(b"changed after preflight\n") + + evidence = execute_plan(plan, tmp_path) + + assert evidence.status == CompatibilityStatus.NOT_VERIFIED + assert evidence.compatibility_execution is None + assert all(item.status == StageStatus.NOT_TESTED for item in evidence.stages) + assert "ARTIFACT_CHANGED_AFTER_PREFLIGHT" in {item.code for item in evidence.findings} + + +def test_changed_runtime_after_preflight_is_not_executed(tmp_path: Path) -> None: + request, executable, _artifact = _fixture(tmp_path) + plan = build_plan(request, tmp_path) + executable.write_text("#!/bin/sh\nexit 99\n", encoding="utf-8") + executable.chmod(0o755) + + evidence = execute_plan(plan, tmp_path) + + assert evidence.status == CompatibilityStatus.NOT_VERIFIED + assert evidence.version_execution is None + assert evidence.compatibility_execution is None + assert all(item.status == StageStatus.NOT_TESTED for item in evidence.stages) + assert "RUNTIME_CHANGED_AFTER_PREFLIGHT" in {item.code for item in evidence.findings} + + +@pytest.mark.parametrize("binding", ["runtime", "artifact"]) +def test_mismatched_explicit_pin_blocks_preflight(tmp_path: Path, binding: str) -> None: + request, _executable, _artifact = _fixture(tmp_path) + request = request.model_copy( + update={ + "executable_sha256" if binding == "runtime" else "artifact_sha256": "0" * 64 + } + ) + + plan = build_plan(request, tmp_path) + + assert plan.status == PlanStatus.BLOCKED + selected = plan.executable if binding == "runtime" else plan.artifact + assert selected.observed_sha256 != selected.expected_sha256 + assert selected.issues == [ + "Observed SHA-256 does not match the explicitly supplied digest." + ] + + +@pytest.mark.parametrize( + ("body", "expected_code"), + [ + ("sys.stdout.buffer.write(b'\\xff')", "NATIVE_STDOUT_NOT_UTF8"), + ( + "sys.stdout.write('Paris'); sys.stderr.buffer.write(b'\\xff')", + "NATIVE_STDERR_NOT_UTF8", + ), + ("sys.stdout.write('x' * 2048)", "NATIVE_STDOUT_OVERFLOW"), + ("sys.stderr.write('x' * 2048)", "NATIVE_STDERR_OVERFLOW"), + ("time.sleep(2)", "NATIVE_EXECUTION_TIMEOUT"), + ("raise SystemExit(7)", "NATIVE_EXECUTION_NONZERO"), + ], +) +def test_non_utf8_overflow_timeout_and_nonzero_fail_closed_with_unknown_stages( + tmp_path: Path, body: str, expected_code: str +) -> None: + request, _executable, _artifact = _fixture(tmp_path, body) + evidence = execute_plan(build_plan(request, tmp_path), tmp_path) + + assert evidence.status == CompatibilityStatus.NOT_VERIFIED + assert all(item.status == StageStatus.UNKNOWN for item in evidence.stages) + assert expected_code in {item.code for item in evidence.findings} + assert len(evidence.unknowns) == 5 + assert evidence.compatibility_execution is not None + assert evidence.compatibility_execution.stdout.captured_bytes <= 1024 + + +def test_self_authored_omiv_pass_report_is_not_privileged(tmp_path: Path) -> None: + artifact = tmp_path / "synthetic.gguf" + artifact.write_bytes(b"synthetic artifact\n") + report = { + "schema": "omiv.runtime-compatibility-runner-report.v1", + "profile_id": PROFILE_ID, + "status": "PASS", + "stages": [ + {"stage": stage, "status": "PASS"} + for stage in ("LOAD", "TOKENIZER", "PREFILL", "DECODE", "OUTPUT") + ], + } + authored_output = json.dumps(report, separators=(",", ":")) + executable = _script( + tmp_path, + f"sys.stdout.write({authored_output!r})", + ) + request = _request(tmp_path, executable, artifact) + request = request.model_copy( + update={ + "test_vector": request.test_vector.model_copy( + update={"expected_output": authored_output} + ) + } + ) + evidence = execute_plan(build_plan(request, tmp_path), tmp_path) + assert evidence.status == CompatibilityStatus.NOT_VERIFIED + assert [item.status for item in evidence.stages] == [ + StageStatus.UNKNOWN, + StageStatus.UNKNOWN, + StageStatus.UNKNOWN, + StageStatus.UNKNOWN, + StageStatus.PASS, + ] + assert all("runtime-compatibility-runner-report" not in item.basis for item in evidence.stages) + + +def test_ambiguous_runtime_version_prevents_compatibility_invocation(tmp_path: Path) -> None: + request, executable, _artifact = _fixture(tmp_path) + source = executable.read_text(encoding="utf-8").replace(VERSION, "ambiguous-version") + executable.write_text(source, encoding="utf-8") + executable.chmod(0o755) + request = request.model_copy(update={"executable_sha256": _digest(executable)}) + + evidence = execute_plan(build_plan(request, tmp_path), tmp_path) + + assert evidence.status == CompatibilityStatus.NOT_VERIFIED + assert evidence.compatibility_execution is None + assert all(item.status == StageStatus.NOT_TESTED for item in evidence.stages) + assert "RUNTIME_VERSION_MISMATCH" in {item.code for item in evidence.findings} + + +def test_exact_output_mismatch_fails_output_stage(tmp_path: Path) -> None: + artifact = tmp_path / "synthetic.gguf" + artifact.write_bytes(b"synthetic artifact\n") + executable = _script(tmp_path, "sys.stdout.write('Lyon')") + evidence = execute_plan( + build_plan(_request(tmp_path, executable, artifact), tmp_path), tmp_path + ) + assert evidence.status == CompatibilityStatus.NOT_VERIFIED + assert evidence.stages[-1].status == StageStatus.FAIL + assert "EXPECTED_OUTPUT_MISMATCH" in {item.code for item in evidence.findings} + + +def test_unexpected_runtime_file_fails_closed_and_is_bounded(tmp_path: Path) -> None: + artifact = tmp_path / "synthetic.gguf" + artifact.write_bytes(b"synthetic artifact\n") + executable = _script( + tmp_path, + "Path('unexpected.txt').write_text('bounded')\n" + "sys.stdout.write('Paris')", + ) + evidence = execute_plan( + build_plan(_request(tmp_path, executable, artifact), tmp_path), tmp_path + ) + assert evidence.status == CompatibilityStatus.NOT_VERIFIED + assert evidence.work_files[0].path == "unexpected.txt" + assert evidence.work_files[0].size == len(b"bounded") + assert "UNEXPECTED_WORK_FILE" in {item.code for item in evidence.findings} + + +def test_unsupported_schema_and_profile_fail_closed(tmp_path: Path) -> None: + request = json.loads( + (ROOT / "examples/runtime-compatibility/request.json").read_text(encoding="utf-8") + ) + request["schema"] = "omiv.runtime-compatibility-request.v999" + path = tmp_path / "unsupported-schema.json" + path.write_text(json.dumps(request), encoding="utf-8") + with pytest.raises(OmivInputError, match="invalid runtime compatibility request"): + load_request(path) + + request["schema"] = "omiv.runtime-compatibility-request.v1" + request["profile_id"] = "omiv.runtime-compatibility-profile.unknown.v1" + path.write_text(json.dumps(request), encoding="utf-8") + with pytest.raises(OmivInputError, match="invalid runtime compatibility request"): + load_request(path) + + +def test_cli_exit_codes_distinguish_blocked_unverified_and_malformed( + tmp_path: Path, +) -> None: + request, _executable, artifact = _fixture(tmp_path) + request_path = tmp_path / "request.json" + request_path.write_text( + json.dumps(request.model_dump(mode="json", by_alias=True)), encoding="utf-8" + ) + plan_path = tmp_path / "plan.json" + evidence_path = tmp_path / "evidence.json" + planned = runner.invoke( + app, + [ + "runtime-compat", + "plan", + "--request", + str(request_path), + "--root", + str(tmp_path), + "--output", + str(plan_path), + ], + ) + assert planned.exit_code == 0, planned.output + executed = runner.invoke( + app, + [ + "runtime-compat", + "run", + "--plan", + str(plan_path), + "--root", + str(tmp_path), + "--output", + str(evidence_path), + ], + ) + assert executed.exit_code == 1, executed.output + assert load_evidence(evidence_path).status == CompatibilityStatus.NOT_VERIFIED + + missing = request.model_copy( + update={ + "request_id": "blocked-cli", + "artifact_path": "missing.gguf", + "artifact_sha256": "0" * 64, + } + ) + request_path.write_text( + json.dumps(missing.model_dump(mode="json", by_alias=True)), encoding="utf-8" + ) + blocked = runner.invoke( + app, + [ + "runtime-compat", + "plan", + "--request", + str(request_path), + "--root", + str(tmp_path), + "--output", + str(tmp_path / "blocked-plan.json"), + ], + ) + assert blocked.exit_code == 1, blocked.output + assert "executable_present=yes executable=yes executable_pinned=yes" in blocked.stdout + assert "artifact_present=no artifact_pinned=no" in blocked.stdout + + malformed = json.loads(request_path.read_text(encoding="utf-8")) + malformed["profile_id"] = "omiv.runtime-compatibility-profile.unsupported.v1" + request_path.write_text(json.dumps(malformed), encoding="utf-8") + rejected = runner.invoke( + app, + [ + "runtime-compat", + "plan", + "--request", + str(request_path), + "--root", + str(tmp_path), + "--output", + str(tmp_path / "rejected-plan.json"), + ], + ) + assert rejected.exit_code == 2, rejected.output + assert artifact.exists() + + +def test_offline_verify_reconstructs_required_captured_execution(tmp_path: Path) -> None: + request, _executable, _artifact = _fixture(tmp_path) + evidence = execute_plan(build_plan(request, tmp_path), tmp_path) + value = evidence.model_dump(mode="json", by_alias=True) + value["version_execution"] = None + _rehash_evidence(value) + path = tmp_path / "tampered-evidence.json" + path.write_text(json.dumps(value), encoding="utf-8") + + with pytest.raises(OmivInputError, match="captured version observation"): + load_evidence(path) + + +@pytest.mark.parametrize( + "mutation", + [ + "request_id", + "plan_id", + "plan_digest", + "plan_content", + "invocation", + "environment", + "limits", + "artifact_binding", + "runtime_binding", + "test_vector", + ], +) +def test_offline_verification_rejects_rehashed_incoherent_plan_projection( + tmp_path: Path, mutation: str +) -> None: + request, _executable, _artifact = _fixture(tmp_path) + evidence = execute_plan(build_plan(request, tmp_path), tmp_path) + value = json.loads(json.dumps(evidence.model_dump(mode="json", by_alias=True))) + if mutation == "request_id": + value["request_id"] = "replacement-request" + elif mutation == "plan_id": + value["plan_id"] = "runtime_compat_plan_" + "f" * 32 + elif mutation == "plan_digest": + value["plan_digest"] = "f" * 64 + elif mutation == "plan_content": + value["plan"]["expected_work"][0] = "Replaced plan content." + elif mutation in {"invocation", "environment"}: + invocation = value["invocation"] + if mutation == "invocation": + invocation["run_arguments"][4] = "replacement prompt" + else: + invocation["environment"][0]["value"] = "replacement" + invocation_body = { + key: item for key, item in invocation.items() if key != "command_digest" + } + invocation["command_digest"] = canonical_sha256({"invocation": invocation_body}) + elif mutation == "limits": + value["limits"]["timeout_seconds"] = 2 + elif mutation == "artifact_binding": + value["artifact"]["execution_sha256"] = "f" * 64 + elif mutation == "runtime_binding": + value["runtime"]["execution_sha256"] = "f" * 64 + elif mutation == "test_vector": + value["test_vector"]["expected_output"] = "replacement" + value["test_vector_digest"] = canonical_sha256( + {"runtime-test-vector": value["test_vector"]} + ) + _rehash_evidence(value) + path = tmp_path / f"tampered-{mutation}.json" + path.write_text(json.dumps(value), encoding="utf-8") + + with pytest.raises(OmivInputError, match="invalid runtime compatibility evidence"): + load_evidence(path) + + +def test_execution_surface_uses_argument_arrays_and_no_shell() -> None: + source = (ROOT / "src/omiv/runtime_compatibility/operations.py").read_text( + encoding="utf-8" + ) + assert "subprocess.Popen(" in source + assert "shell=False" in source + assert "cwd=cwd" in source + assert "env=environment" in source + assert "start_new_session=True" in source + for forbidden in ("shell=True", "socket", "urllib", "requests", "httpx", "cuda"): + assert forbidden not in source + assert os.access( + ROOT / "examples/runtime-compatibility/synthetic-llama-runner.py", os.X_OK + ) diff --git a/tests/test_smart_preflight.py b/tests/test_smart_preflight.py new file mode 100644 index 0000000..f207c35 --- /dev/null +++ b/tests/test_smart_preflight.py @@ -0,0 +1,214 @@ +"""Candidate Phase 7A Smart Preflight / Auto Planner vertical-slice tests.""" + +from __future__ import annotations + +import json +from pathlib import Path + +import pytest +from typer.testing import CliRunner + +from omiv.assurance.models import PreflightStatus, VerdictRole +from omiv.assurance.operations import build_preflight, load_request +from omiv.cli import app +from omiv.smart_preflight.models import ( + CoverageStatus, + SmartPreflightIntent, + SmartPreflightStatus, +) +from omiv.smart_preflight.operations import build_smart_preflight + +ROOT = Path(__file__).parents[1] +runner = CliRunner() + + +def _intent(search_paths: list[str], dimensions: list[str]) -> SmartPreflightIntent: + return SmartPreflightIntent( + intent_id="phase7a-test", + subject="synthetic local Phase 7A candidate", + search_paths=search_paths, + required_dimensions=dimensions, + ) + + +def test_tracked_example_generates_phase6f_handoff_without_costly_work() -> None: + plan = build_smart_preflight( + _intent( + ["payload-integrity/evidence/authorized-complete.json"], + ["STRUCTURE"], + ), + ROOT, + ) + + assert plan.status == SmartPreflightStatus.READY + assert plan.coverage[0].status == CoverageStatus.COVERED + assert len(plan.candidates) == 1 + assert plan.candidates[0].selected is True + assert plan.candidates[0].proposed_verdict_role == VerdictRole.DIMENSION_VERDICT + assert plan.assurance_request is not None + assert plan.assurance_request.planned_operations == [] + assert not any( + ( + plan.costs.download, + plan.costs.network, + plan.costs.conversion, + plan.costs.remote_collector, + plan.costs.gpu, + ) + ) + + assurance_plan = build_preflight(plan.assurance_request, ROOT) + assert assurance_plan.status == PreflightStatus.READY + # Phase 7A does not extend the Phase 6F semantic-value allowlist or promote a + # canonical-but-unrecognized outcome to PASS. + assert assurance_plan.members[0].semantic_status.value == "UNKNOWN" + + +def test_cli_vertical_slice_writes_request_consumed_by_unchanged_assurance_plan( + tmp_path: Path, +) -> None: + smart_plan = tmp_path / "smart-plan.json" + assurance_request = tmp_path / "assurance-request.json" + result = runner.invoke( + app, + [ + "smart-preflight", + "plan", + "--intent", + str(ROOT / "examples/smart-preflight/intent.json"), + "--root", + str(ROOT), + "--output", + str(smart_plan), + "--assurance-request-output", + str(assurance_request), + ], + ) + assert result.exit_code == 0, result.output + assert "READY candidates=1 selected=1 gaps=0 costly=none" in result.stdout + + request = load_request(assurance_request) + assert request.schema_id == "omiv.assurance-request.v1" + assert request.requirements[0].source_path == ( + "payload-integrity/evidence/authorized-complete.json" + ) + phase6f_plan = tmp_path / "assurance-plan.json" + handoff = runner.invoke( + app, + [ + "assurance", + "plan", + "--request", + str(assurance_request), + "--root", + str(ROOT), + "--output", + str(phase6f_plan), + ], + ) + assert handoff.exit_code == 0, handoff.output + assert handoff.stdout.startswith("READY ") + + +def test_distinct_verdict_candidates_are_ambiguous_and_not_auto_selected() -> None: + first = build_smart_preflight( + _intent(["quantization-fidelity/evidence"], ["FIDELITY"]), ROOT + ) + second = build_smart_preflight( + _intent(["quantization-fidelity/evidence"], ["FIDELITY"]), ROOT + ) + + assert first == second + assert first.status == SmartPreflightStatus.BLOCKED + assert first.coverage[0].status == CoverageStatus.AMBIGUOUS + assert first.assurance_request is None + assert not any(item.selected for item in first.candidates) + assert "VERDICT_CANDIDATE_AMBIGUOUS" in {item.code for item in first.findings} + + +def test_invalid_canonical_object_is_reported_without_becoming_evidence(tmp_path: Path) -> None: + source = ROOT / "payload-integrity/evidence/authorized-complete.json" + value = json.loads(source.read_text(encoding="utf-8")) + value["outcome"] = "NOT_A_CANONICAL_OUTCOME" + (tmp_path / "invalid.json").write_text(json.dumps(value), encoding="utf-8") + + plan = build_smart_preflight(_intent(["invalid.json"], ["STRUCTURE"]), tmp_path) + + assert plan.status == SmartPreflightStatus.BLOCKED + assert plan.candidates == [] + assert plan.assurance_request is None + assert {item.code for item in plan.findings} >= { + "DISCOVERY_SCHEMA_VALIDATION_FAILED", + "REQUIRED_DIMENSION_MISSING", + } + + +def test_phase7b_candidate_evidence_is_not_auto_selected(tmp_path: Path) -> None: + (tmp_path / "runtime-compatibility.json").write_text( + json.dumps({"schema": "omiv.runtime-compatibility-evidence.v1"}), + encoding="utf-8", + ) + + plan = build_smart_preflight(_intent(["."], ["RUNTIME"]), tmp_path) + + assert plan.status == SmartPreflightStatus.BLOCKED + assert plan.candidates == [] + assert plan.assurance_request is None + assert {item.code for item in plan.findings} >= { + "DISCOVERY_SCHEMA_UNSUPPORTED", + "REQUIRED_DIMENSION_MISSING", + } + + +def test_symlinks_are_never_followed(tmp_path: Path) -> None: + target = tmp_path / "target.json" + target.write_bytes((ROOT / "payload-integrity/evidence/authorized-complete.json").read_bytes()) + link = tmp_path / "linked.json" + try: + link.symlink_to(target) + except OSError: + pytest.skip("symlinks are unavailable on this platform") + + plan = build_smart_preflight(_intent(["."], ["STRUCTURE"]), tmp_path) + + assert plan.status == SmartPreflightStatus.READY + assert len(plan.candidates) == 1 + assert plan.candidates[0].source_path == "target.json" + assert "DISCOVERY_SYMLINK_SKIPPED" in {item.code for item in plan.findings} + + +def test_discovery_limit_blocks_partial_auto_plan( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + raw = (ROOT / "payload-integrity/evidence/authorized-complete.json").read_bytes() + (tmp_path / "a.json").write_bytes(raw) + (tmp_path / "b.json").write_bytes(raw) + monkeypatch.setattr("omiv.smart_preflight.operations.MAX_DISCOVERY_JSON_FILES", 1) + + plan = build_smart_preflight(_intent(["."], ["STRUCTURE"]), tmp_path) + + assert plan.status == SmartPreflightStatus.BLOCKED + assert plan.costs.download is False + assert "LIMIT_EXCEEDED:DISCOVERY_JSON_FILES" in {item.code for item in plan.findings} + + +def test_cli_refuses_to_overwrite_discovered_evidence() -> None: + evidence = ROOT / "payload-integrity/evidence/authorized-complete.json" + before = evidence.read_bytes() + result = runner.invoke( + app, + [ + "smart-preflight", + "plan", + "--intent", + str(ROOT / "examples/smart-preflight/intent.json"), + "--root", + str(ROOT), + "--output", + str(evidence), + ], + ) + + assert result.exit_code == 2 + assert "collides with an input path" in result.output + assert evidence.read_bytes() == before diff --git a/tests/test_v0_10_0_release_preparation.py b/tests/test_v0_10_0_release_preparation.py index c401bf6..9aafb04 100644 --- a/tests/test_v0_10_0_release_preparation.py +++ b/tests/test_v0_10_0_release_preparation.py @@ -3,6 +3,7 @@ import json import runpy import subprocess +import sys import tomllib from pathlib import Path @@ -17,7 +18,7 @@ def _audit() -> dict[str, object]: def test_release_preparation_audit_passes_and_is_deterministic() -> None: - command = ["python", "tools/audit_v0_10_0_release_preparation.py", "--json"] + command = [sys.executable, "tools/audit_v0_10_0_release_preparation.py", "--json"] first = subprocess.run(command, cwd=ROOT, check=True, capture_output=True, text=True) second = subprocess.run(command, cwd=ROOT, check=True, capture_output=True, text=True) assert first.stdout == second.stdout diff --git a/tools/audit_github_publication_controls.py b/tools/audit_github_publication_controls.py index 69aea0f..662cfe5 100644 --- a/tools/audit_github_publication_controls.py +++ b/tools/audit_github_publication_controls.py @@ -1038,10 +1038,13 @@ def run_audit(root: Path) -> list[Check]: ) checks.append( Check( - "phase6f_unimplemented", - "Phase 6F | PLANNED, NOT IMPLEMENTED" in roadmap - and policy["implementation"]["phase6f_status"] == "PLANNED_NOT_IMPLEMENTED", - "phase6f=planned", + "phase6f_boundary", + ( + "Phase 6F | COMPLETE; NOT RELEASED" in roadmap + and "Phase 7 | FUTURE, SCOPE NOT FROZEN" in roadmap + and policy["implementation"]["phase6f_status"] == "PLANNED_NOT_IMPLEMENTED" + ), + "r1f_historical=planned current=complete_not_released phase7=unfrozen", ) ) diff --git a/tools/audit_offline_evidence_walkthrough.py b/tools/audit_offline_evidence_walkthrough.py index 3213bd8..9c9b57e 100644 --- a/tools/audit_offline_evidence_walkthrough.py +++ b/tools/audit_offline_evidence_walkthrough.py @@ -509,7 +509,7 @@ def run_audit(root: Path) -> list[Check]: "| R1D offline walkthrough | COMPLETE |", "| R1E GitHub metadata/security | COMPLETE |", "| R1F final publication audit | COMPLETE; public controls verified |", - "| Phase 6F | PLANNED, NOT IMPLEMENTED |", + "| Phase 6F | COMPLETE; NOT RELEASED |", ) checks.append( Check( @@ -518,12 +518,12 @@ def run_audit(root: Path) -> list[Check]: f"required={len(roadmap_terms)}", ) ) - phase6f_complete = any( - re.search(r"Phase 6F.{0,30}\b(COMPLETE|IMPLEMENTED)\b", line, re.IGNORECASE) - and "not implemented" not in line.lower() + phase6f_released = any( + re.search(r"Phase 6F.{0,30}\bRELEASED\b", line, re.IGNORECASE) + and "NOT RELEASED" not in line.upper() for line in documentation.splitlines() ) - checks.append(Check("phase6f_unimplemented", not phase6f_complete, "completion_claims=0")) + checks.append(Check("phase6f_not_released", not phase6f_released, "release_claims=0")) forbidden_claim_patterns = ( r"available on PyPI", diff --git a/tools/audit_public_launch_ux.py b/tools/audit_public_launch_ux.py index 89d94cb..32b0291 100644 --- a/tools/audit_public_launch_ux.py +++ b/tools/audit_public_launch_ux.py @@ -349,7 +349,7 @@ def run_audit(root: Path = ROOT) -> list[Check]: "Phase 6C | RELEASED", "Phase 6D | RELEASED", "Phase 6E | RELEASED", - "Phase 6F | PLANNED, NOT IMPLEMENTED", + "Phase 6F | COMPLETE; NOT RELEASED", "Phase 7 | FUTURE, SCOPE NOT FROZEN", "R1A readiness | COMPLETE", "R1B private clean-clone CI | COMPLETE", diff --git a/tools/audit_r1f_publication_readiness.py b/tools/audit_r1f_publication_readiness.py index 6c6eea4..6e793ba 100644 --- a/tools/audit_r1f_publication_readiness.py +++ b/tools/audit_r1f_publication_readiness.py @@ -853,7 +853,7 @@ def run_audit(root: Path) -> list[Check]: marker in roadmap for marker in ( "R1E GitHub metadata/security | COMPLETE", - "Phase 6F | PLANNED, NOT IMPLEMENTED", + "Phase 6F | COMPLETE; NOT RELEASED", "Phase 7 | FUTURE, SCOPE NOT FROZEN", ) ),