diff --git a/README.md b/README.md index 9849d44..ffb899e 100644 --- a/README.md +++ b/README.md @@ -1,46 +1,69 @@ # Open Model Integration Validator (OMIV) -OMIV is an offline-first evidence and verification framework for AI artifacts, -transformations, deployments, and runtime identity. +OMIV is an offline-first framework +for verifying AI model artifacts, +transformations, deployments, +and runtime identity. [![CI](https://github.com/200lz/open-model-integration-validator/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/200lz/open-model-integration-validator/actions/workflows/ci.yml) [![Python 3.11+](https://img.shields.io/badge/Python-3.11%2B-3776AB)](pyproject.toml) [![License: Apache-2.0](https://img.shields.io/badge/License-Apache--2.0-blue.svg)](LICENSE) [![Status: public preview](https://img.shields.io/badge/status-public--preview-orange)](docs/roadmap.md) -It preserves the difference between what was declared, what was observed, what -was cryptographically verified, what policy permits, and what remains -unavailable. OMIV is a pre-1.0 framework, not a certification authority, safety -evaluator, authenticity oracle, or production-ready compliance product. +Know what model artifact you have, what changed, what evidence supports it, +and what remains unknown. + +It is for model release teams, +inference/runtime maintainers, +conversion and quantization vendors, +chip and platform teams, +and enterprise AI teams. + +It verifies bounded evidence for artifacts, +shards, GGUF comparisons, transformations, +configuration, trust, policy, +and runtime records. + +Try the offline quickstart below, then see the real +[Unsloth Gemma 4 case study](case-studies/unsloth-gemma4-e2b-it-q8/README.md). + +OMIV is a pre-1.0 Alpha public preview, +not a certification authority, safety evaluator, +authenticity oracle, +or production compliance product. ## Why OMIV -A model name is not a model identity. A signature does not prove runtime-loaded -weights. A deployment declaration does not prove which weights produced an output. -Missing evidence remains unavailable. Canonical validity is not authenticity, and -structural parity is not behavioral equivalence. +A model name is not a model identity. +A signature does not prove which weights +a runtime loaded or produced an output. -OMIV gives developers strict schemas, canonical identities and digests, -scope-qualified verification, signatures, policy results, and portable evidence. -It does not turn those records into claims about model safety, provider authority, -complete behavioral equivalence, or runtime weight attribution. +OMIV separates declarations, observations, +cryptographic verification, policy decisions, +and unavailable evidence. +Missing evidence stays unavailable. + +Canonical validity is not authenticity. +Structural parity is not behavioral equivalence. +OMIV does not diagnose or fix +every integration problem. ## What OMIV verifies -| Evidence surface | OMIV can verify | OMIV does not establish | +| Evidence surface | Verifiable within scope | Not established | | --- | --- | --- | -| Local artifacts | Declared byte identity, manifests, structure, and canonical digests | Safety, authenticity, or publisher authority | -| Transformations | Supplied lineage, mappings, quantization representation, and bounded numerical evidence | Complete semantic or behavioral equivalence | -| Trust and policy | Signature validity, stated authority, policy evaluation, custody, and historical linkage | That a signer is the rightful publisher or a policy is appropriate | -| Deployment and runtime | Supplied declarations, observations, bindings, and output-provenance records | Which weights actively produced an output without adequate evidence | -| Remote metadata | Bounded supplied snapshots and shard reconciliation | Current provider state or payload identity by default | +| Artifacts | Bytes, manifests, structure, digests | Safety or publisher authority | +| Shards/GGUF | Completeness and structural comparison | Universal compatibility | +| Changes | Lineage, mappings, representation, samples | Complete semantic equivalence | +| Trust | Signatures, authority, custody, policy | Policy appropriateness | +| Runtime | Supplied resolution, deployment, output bindings | Active weights without evidence | -An absent, unsupported, or unchecked evidence item never becomes `PASS`. +An absent, unsupported, or unchecked item never becomes `PASS`. ## 30-second quickstart -From a clean clone, install OMIV from source. Dependency installation may contact -a package index; the verification command itself is offline. +From a clean clone, install from source. Installation may contact a package +index; the verification command itself is offline. ```bash python -m venv .venv @@ -50,7 +73,7 @@ omiv --version omiv runtime-resolution verify runtime-resolution-parity/scenarios/immutable-pinned.json ``` -Expected verification output and exit status: +Expected output and exit status: ```text 0.10.0 @@ -58,221 +81,176 @@ VALID_CANONICAL_RUNTIME_RESOLUTION_OBJECT schema=omiv.runtime-resolution-scenari exit code: 0 ``` -The tracked input is synthetic demonstration data. This command establishes only -strict parsing, supported-schema recognition, and canonical identity/digest -relationships for that supplied object. It makes no provider request, resolves no -live alias, observes no deployment or runtime-loaded weights, and runs no inference. -It does not establish provider authenticity, publisher authority, safety, or -production readiness. See the [complete quickstart](docs/quickstart.md) and [offline -demonstration](examples/offline-quickstart/README.md), including PowerShell activation -and exit-code semantics. +The input is synthetic. +The command verifies parsing, schema support, +and identity/digest relationships for that object. +It makes no provider request, +resolves no live alias, +observes no deployment or loaded weights, +and runs no inference. + +See the [complete quickstart](docs/quickstart.md) +and [offline demonstration](examples/offline-quickstart/README.md). ## Evidence chain ```mermaid flowchart TD - A[Source / registry / publisher] --> B[Resolution and acquisition evidence] - B --> C[Exact artifact identity] - C --> D[Payload manifest and shard reconciliation] - D --> E[Transformation, quantization, tokenizer evidence] - E --> F[Signature, authority, and policy] - F --> G[Deployment and runtime binding] - G --> H[Output provenance and historical audit] + A[Source or registry] --> B[Acquisition evidence] + B --> C[Artifact identity] + C --> D[Manifest and shards] + D --> E[Transformation evidence] + E --> F[Signature and policy] + F --> G[Deployment binding] + G --> H[Output provenance] A -. declaration is not observation .-> B - C -. artifact identity is not runtime identity .-> G - F -. signature validity is not publisher authority .-> A - E -. finite probes are not behavioral equivalence .-> H - B -. absent evidence is not PASS .-> H + C -. artifact is not runtime identity .-> G + E -. finite evidence is bounded .-> H + B -. unavailable is not PASS .-> H ``` -The arrows describe possible evidence flow, not universal automation. Read the -[architecture overview](docs/architecture.md) for component boundaries and the -current Phase 5/6 evidence model. +This is possible evidence flow, not universal automation. See +[architecture](docs/architecture.md). ## Current capabilities -| Area | Available now | Bounded, local, or offline scope | Explicit non-goals | -| --- | --- | --- | --- | -| Phase 5 | Passports, custody, attestations, trust, governance, security, runtime continuity, and historical audit | Verification of supplied evidence and policy-scoped results | Safety certification, real-world truth, or automatic approval | -| Phase 6A | Payload manifests and exact local byte identity | Local files and declared expectation scope | Publisher identity or safety | -| Phase 6B | Remote metadata and shard reconciliation | Supplied/pinned metadata; collectors are separate and opt-in | Payload download or inferred weight identity | -| Phase 6C | Quantization representation and bounded numerical-fidelity evidence | Declared samples, limits, and Decimal semantics | Whole-model behavioral equivalence | -| Phase 6D | Tokenizer/configuration structural parity and supplied probes | Supplied local artifacts and finite probes | Complete tokenizer or behavioral equivalence | -| Phase 6E | Mutable resolution, deployment binding, supplied backend results, and output-provenance schemas | Offline verification of supplied records | Live resolution, inference, or provable inference | - -The [documentation index](docs/README.md) links the design and implementation -documents for every released engineering phase. Phase 6F Assurance Bundle -interoperability is planned and is not implemented. - -### How to read an OMIV result - -OMIV results are deliberately narrower than labels such as “verified model.” Read -each result through five questions: - -1. **What is the subject?** A file, logical model, transformation, deployment, - runtime observation, request, and output are different subjects. -2. **What is the scope?** A selected member set, local directory, finite probe set, - policy profile, environment, and historical cutoff limit what was evaluated. -3. **What evidence was supplied?** A declaration, direct observation, digest-only - reference, signature, and provider document carry different authority. -4. **What was reconstructed?** Schema validity, canonical digest identity, reference - linkage, signature validity, and policy evaluation are separate checks. -5. **What remains unavailable?** Limitations and missing evidence are part of the - result, not footnotes to discard. - -For example, an intact signature can show that bytes were signed by a particular -key. A separate trust record is needed to state what that key is authorized to sign, -and neither record alone shows that the signed artifact was loaded by a runtime. -Likewise, an exact finite probe result applies only to the declared probes and does -not establish identical weights or behavior outside that set. - -### Evidence result vocabulary - -Different OMIV subsystems use typed, domain-specific outcomes, but the following -reading rules are consistent: - -| Result shape | Interpretation | -| --- | --- | -| Valid canonical object | Strict parsing and identity reconstruction succeeded for the supplied object | -| Exact match for declared scope | All requirements inside an explicit finite scope matched | -| Valid with limitations | Integrity succeeded while named evidence or scope remains incomplete | -| Not available / not checked | Required evidence was not supplied or was intentionally outside the operation | -| Policy nonpassing | The supplied evidence did not satisfy the selected policy; this is not necessarily malformed input | -| Integrity invalid | Schema, digest, reference, signature, or canonical reconstruction failed | - -Always keep the subject, scope, evidence source, policy identity, and limitations -beside the headline outcome. Portable evidence is designed to preserve those -qualifiers when it moves between tools or organizations. - -### Verification posture - -OMIV defaults to offline verification. Network-capable collection is a separate, -explicit operation with its own bounds; verification never fills gaps by contacting -a provider. Canonical output avoids host-specific paths, clock-dependent identities, -and accidental mutation of tracked evidence. - -This posture makes results reviewable in a clean checkout and usable in air-gapped -or controlled environments after installation. It also keeps collection authority -separate from verification authority: the verifier can check a portable supplied -record without claiming to have witnessed how it was collected. - -When integrating OMIV into automation, treat nonzero scope/policy outcomes, -integrity errors, missing external artifacts, and unsupported evidence as distinct -states. Do not flatten them into a generic success or failure that loses why the -result was bounded. - -### Common offline workflows - -The root quickstart exercises one safe verification path. The -[offline evidence walkthrough](docs/offline-evidence-walkthrough.md) continues -through tracked Phase 5 and Phase 6A–6E evidence, including expected semantic -exit-`1` results and a temporary malformed-input exit-`2` demonstration. The -repository also contains bounded offline interfaces for: - -- validating canonical inventory structure; -- comparing GGUF inventories under explicit policies; -- validating semantic mappings and transformation lineage; -- verifying Model Passport and custody integrity; -- checking signed attestations, trust bundles, and revocation records; -- evaluating governance, security, and historical trust evidence; -- verifying local payload manifests and declared expectations; -- reconstructing remote metadata and shard-reconciliation objects; -- inspecting quantization and tokenizer/configuration evidence; -- verifying supplied deployment, runtime, and output-provenance records. - -These commands do not all have the same exit-code policy. Consult the linked phase -document and command help before automating a gate: +OMIV currently covers: -```bash -omiv --help -omiv runtime-resolution verify --help -``` +- artifact identity and manifests; +- shard completeness and reconciliation; +- GGUF/source-to-target comparison; +- transformation and quantization evidence; +- tokenizer/configuration evidence; +- signatures, trust, and policy; +- deployment, runtime, and output provenance. -Collectors and conversion runners are distinct from offline verification. Some are -explicitly opt-in or can execute external tools; they are not part of the quickstart -and should be reviewed under their own documented threat and resource boundaries. +Released boundaries remain narrow: -### Repository evidence versus external artifacts +| Phase | Released scope | Boundary | +| --- | --- | --- | +| Phase 5 | Passports, custody, attestations, trust, governance, security, runtime continuity, historical audit | Supplied evidence; no automatic approval | +| Phase 6A | Local manifests and byte identity | No publisher or semantic claim | +| Phase 6B | Pinned metadata and shard reconciliation | Collection stays separate | +| Phase 6C | Representation and bounded numerical-fidelity evidence | Declared samples and limits only | +| Phase 6D | Tokenizer/configuration parity and supplied probes | Finite scope only | +| Phase 6E | Runtime resolution, deployment binding, supplied results, output provenance | No implicit live inference | -Tracked synthetic and canonical examples make verifier behavior reviewable without -large payloads. They do not pretend to be fresh provider observations. When a -referenced external artifact is absent, OMIV either reports that absence or offers a -clearly named reduced-scope verification mode where the schema supports one. +Phase 6F Assurance Bundle interoperability is planned, not implemented. +See the [roadmap](docs/roadmap.md). -In particular, the ignored Kimi raw inventory is not a quickstart dependency. Do not -create or download it to run the demonstration. Remote practice collectors are also -unnecessary for normal offline verification and are never invoked implicitly. +### Reading results -## Real-world case studies +Keep subject, scope, source, policy, +and limits together. Unavailable stays unavailable. +Signatures do not prove authority or runtime use. -- **Case Study 01 — Unsloth Gemma 4 E2B IT Q8_0 GGUF:** The export succeeded, and - both current artifact identities matched retained historical C1 size/SHA-256 - observations. OMIV also made - source-provenance and main/mmproj companion-binding observability gaps explicit. - Read the [case study](case-studies/unsloth-gemma4-e2b-it-q8/README.md) with its - methodology, claim registry, evidence index, results, and limitations. +### Offline and fail-closed posture -## Practice-profile limitations +Verification defaults to offline operation. Collection is explicit and separate; +verification does not contact providers to fill gaps. -Practice profiles illustrate bounded evidence contracts; they are not endorsements, -partnerships, or claims about current provider state. +Missing evidence, unsupported schemas, integrity errors, +and policy failures remain distinct. -| Profile | Current scope | Important limitation | -| --- | --- | --- | -| Qwen | Existing local model-pack and format examples | Example coverage does not make a universal integration claim | -| Kimi | Structural and reference evidence | The large raw local inventory is not distributed | -| xAI | Bounded public metadata and documented mutable-alias/roadmap evidence | No observed runtime weight identity | -| DeepSeek | Readiness and missing-snapshot contract | Readiness is not a completed integration | -| Hugging Face | Provider-neutral bounded metadata interface | No default network use and no provider endorsement | +The [offline walkthrough](docs/offline-evidence-walkthrough.md) covers tracked +Phase 5 and Phase 6A–6E records and exit-code semantics. + +Collectors and conversion runners may execute tools or access networks. +They are outside the quickstart. + +## Real-world case study + +**Case Study 01 — Unsloth Gemma 4 E2B IT Q8_0 GGUF** +is an independent, offline structural +and provenance-observability analysis. + +The export succeeded. +Current main-GGUF and mmproj sizes/SHA-256 values +match retained historical C1 observations. +Source-revision and format-native companion binding +remained unavailable. + +The historical C1 bytes are unavailable, +so this is `HISTORICAL_C1_MATCH`, +not a new direct C1/C2 file comparison. + +The study makes no semantic-fidelity, +numerical-fidelity, or runtime-compatibility claim. +Structural validation is not behavioral equivalence. + +We warmly thank Daniel Han +for suggesting Gemma 4 E2B IT +and the opportunity to test a real Unsloth export. +OMIV worked independently; +the exchange does not imply partnership, endorsement, +approval, certification, or joint work. + +Read the +[case study](case-studies/unsloth-gemma4-e2b-it-q8/README.md) +with its claims, evidence, results, and limits. + +## Practice-profile limitations + +Practice profiles are bounded examples, +not partnerships, endorsements, +or claims about current provider state. +See the [technical reference](docs/reference/technical-reference.md). ## Public and future commercial boundary -The open core owns canonical schemas, canonicalization, evidence semantics, -signatures and verification, the offline CLI, portable evidence, policy-result -semantics, and future public Assurance Bundle verification. +The public open core owns schemas, +canonicalization, evidence semantics, +signatures, offline verification, +portable evidence, policy results, +and public Assurance Bundle verification. + +Possible future commercial work may add +managed collection, private registries, +monitoring, organization policy, +RBAC/SSO, KMS/HSM, deployment admission, +connectors, history, and support. -Possible future commercial operation may add managed collection, private -registries, continuous monitoring, organization-wide policy operation, RBAC/SSO, -KMS/HSM integration, deployment admission, managed history, connectors, and -support. No such product or repository is claimed here, and it may not secretly -redefine canonical public OMIV semantics. Read the -[full boundary](docs/public-commercial-boundary.md). +No commercial product is claimed. Future services may not redefine public OMIV +semantics. See the [full boundary](docs/public-commercial-boundary.md). ## Documentation -Start with the [documentation index](docs/README.md): +Start with the [documentation index](docs/README.md). - [Quickstart](docs/quickstart.md) - [Offline evidence walkthrough](docs/offline-evidence-walkthrough.md) - [Architecture](docs/architecture.md) -- [Technical reference migrated from the historical README](docs/reference/technical-reference.md) -- [README migration map](docs/reference/readme-migration-map.md) - [Security and privacy](docs/public-release-security-and-privacy.md) - [GitHub publication controls](docs/github-publication-controls.md) -- [R1F final-publication audit](docs/r1f-final-publication-audit.md) -- [v0.10.0 release notes](docs/v0.10.0-release-notes.md) -- [v0.10.0 publication recovery](docs/v0.10.0-publication-recovery.md) +- [R1F publication audit](docs/r1f-final-publication-audit.md) +- [Release notes](docs/v0.10.0-release-notes.md) - [Roadmap](docs/roadmap.md) +Detailed command and phase guidance lives in the documentation. + ## Project status and roadmap -OMIV is public and publicly available as a pre-1.0 Alpha public preview. The repository is public. -The signed annotated `v0.10.0` tag and its GitHub pre-release exist, while -the PyPI project and version remain absent after the first Trusted Publishing workflow -failed before its publish job. Engineering Phases 5 and 6A–6E are released in repository history; -Phase 6F is planned and not implemented. Public -availability is not release completion and does not establish production readiness, -certification, safety, provider authenticity, or publisher authority. See the -[roadmap](docs/roadmap.md), [release notes](docs/v0.10.0-release-notes.md), and -[changelog](CHANGELOG.md). +The repository is public. OMIV remains a pre-1.0 Alpha public preview, +not a production, certification, safety, or authenticity claim. + +Phases 5 and 6A–6E are released in history. Phase 6F is planned, +not implemented. The signed annotated `v0.10.0` tag and GitHub pre-release exist; +the PyPI project/version remain absent after a pre-publish failure. + +Public availability is not release completion. See the [roadmap](docs/roadmap.md) +and [release notes](docs/v0.10.0-release-notes.md). ## Contributing, security, support, and license Contributions follow [CONTRIBUTING.md](CONTRIBUTING.md) and -[GOVERNANCE.md](GOVERNANCE.md). Report vulnerabilities through -[SECURITY.md](SECURITY.md), and use [SUPPORT.md](SUPPORT.md) for support scope. +[GOVERNANCE.md](GOVERNANCE.md). + +Report vulnerabilities through [SECURITY.md](SECURITY.md). Use +[SUPPORT.md](SUPPORT.md) for the support boundary. + +OMIV uses [Apache-2.0](LICENSE). See [NOTICE](NOTICE), +[third-party notices](THIRD_PARTY_NOTICES.md), +and [trademarks](TRADEMARKS.md). -OMIV is licensed under [Apache-2.0](LICENSE); see [NOTICE](NOTICE) and -[third-party notices](THIRD_PARTY_NOTICES.md). Project and provider names remain -subject to [trademark guidance](TRADEMARKS.md). OMIV is independent and is not an -official, approved, endorsed, or affiliated tool of any model provider. +OMIV is independent and is not an official, approved, endorsed, affiliated, or +certified tool of any model provider. diff --git a/docs/github-publication-controls.md b/docs/github-publication-controls.md index 146fb48..f35e418 100644 --- a/docs/github-publication-controls.md +++ b/docs/github-publication-controls.md @@ -188,6 +188,19 @@ No workflow may gain write permissions or execute untrusted pull-request code wi privileged event. Publication automation, if separately designed later, must not hold write credentials while evaluating untrusted code. +The public-release identity audit keeps ordinary authoritative-main GitHub squash +commits with a terminal `(#N)` subject association fully offline. A reviewed, +GitHub-signed, single-parent author/committer pair on authoritative `main` whose subject +lacks that suffix uses a separate bounded public-corroboration path: unauthenticated, +read-only GitHub REST responses must identify exactly one merged same-repository pull +request and must match repository name and ID, base `main`, result/base/head/tree +topology, reviewed actors, signature reason, local signer key, and identity +fingerprints. Responses are size-bounded and cached within the audit process; absence, +ambiguity, malformed data, extra associations, or contradiction fails closed. No +credential is read or serialized. This classification grants no owner, publisher, +maintainer, release, or repository authority and does not prove review, required +checks, branch protection, safety, authenticity, or behavioral equivalence. + ## Public-main enforcement policy After public visibility, normal changes to `main` require a pull request and the diff --git a/tests/test_public_release_readiness.py b/tests/test_public_release_readiness.py index de0542b..7d88990 100644 --- a/tests/test_public_release_readiness.py +++ b/tests/test_public_release_readiness.py @@ -4,6 +4,7 @@ import runpy import subprocess import tomllib +from copy import deepcopy from dataclasses import asdict, replace from pathlib import Path from typing import Any @@ -242,6 +243,141 @@ def _signed_squash_fixture( return observations, (evidence,) +def _valid_live_squash_metadata( + namespace: dict[str, object], + *, + commit_sha: str = "6" * 40, + tree_sha: str = "7" * 40, + parent_sha: str = "8" * 40, + head_sha: str = "9" * 40, +) -> tuple[list[dict[str, Any]], dict[str, Any], dict[str, Any]]: + repository = { + "id": namespace["GITHUB_REPOSITORY_ID"], + "full_name": namespace["GITHUB_REPOSITORY_FULL_NAME"], + } + associated = [ + { + "number": 17, + "state": "closed", + "merged_at": "2026-08-13T00:00:00Z", + "merge_commit_sha": commit_sha, + "user": {"id": namespace["GITHUB_OWNER_ACTOR_ID"]}, + "base": {"ref": "main", "sha": parent_sha, "repo": deepcopy(repository)}, + "head": {"sha": head_sha, "repo": deepcopy(repository)}, + } + ] + result_commit = { + "sha": commit_sha, + "parents": [{"sha": parent_sha}], + "author": {"id": namespace["GITHUB_OWNER_ACTOR_ID"]}, + "committer": {"id": namespace["GITHUB_WEB_FLOW_ACTOR_ID"]}, + "commit": { + "tree": {"sha": tree_sha}, + "verification": {"verified": True, "reason": "valid"}, + }, + } + head_commit = {"sha": head_sha, "commit": {"tree": {"sha": tree_sha}}} + return associated, result_commit, head_commit + + +def _install_live_squash_metadata( + namespace: dict[str, object], + associated: object, + result_commit: object, + head_commit: object, + *, + unavailable_endpoint: str | None = None, + unavailable_reason: str = "NETWORK_ERROR", + unavailable_status: int = 0, +) -> None: + github_result = namespace["GitHubJsonResult"] + builder_globals = namespace["_live_public_squash_commit_identity_evidence"].__globals__ + try: + associated_head_sha = associated[0]["head"]["sha"] + except (KeyError, IndexError, TypeError): + associated_head_sha = None + + def response(payload: object, endpoint: str) -> object: + if unavailable_endpoint == endpoint: + return github_result(False, unavailable_reason, unavailable_status) + return github_result(True, "AVAILABLE", 200, payload) + + def fake_array(url: str) -> object: + assert url.endswith("/pulls?per_page=2") + return response(associated, "associated") + + def fake_object(url: str) -> object: + if associated_head_sha is not None and url.endswith("/" + associated_head_sha): + return response(head_commit, "head") + return response(result_commit, "result") + + builder_globals["_github_json_array"] = fake_array + builder_globals["_github_json"] = fake_object + + +def _run_live_squash_corroboration( + namespace: dict[str, object], + associated: object, + result_commit: object, + head_commit: object, + *, + unavailable_endpoint: str | None = None, + signer_fingerprint: str | None = None, +) -> object: + _install_live_squash_metadata( + namespace, + associated, + result_commit, + head_commit, + unavailable_endpoint=unavailable_endpoint, + ) + return namespace["_live_public_squash_commit_identity_evidence"]( + repository=namespace["GITHUB_REPOSITORY_FULL_NAME"], + commit_sha="6" * 40, + tree_sha="7" * 40, + parents=("8" * 40,), + author_fingerprint=namespace["GITHUB_SQUASH_AUTHOR_FINGERPRINT"], + committer_fingerprint=namespace["GITHUB_SQUASH_COMMITTER_FINGERPRINT"], + signer_fingerprint=( + namespace["GITHUB_WEB_FLOW_SIGNING_KEY_FINGERPRINT"] + if signer_fingerprint is None + else signer_fingerprint + ), + authoritative_ref_classifications=("LOCAL_MAIN", "REMOTE_MAIN"), + ) + + +def _install_current_main_live_metadata(namespace: dict[str, object]) -> None: + commit_sha = subprocess.run( + ["git", "rev-parse", "refs/remotes/origin/main"], + cwd=ROOT, + check=True, + capture_output=True, + text=True, + ).stdout.strip() + parent_sha = subprocess.run( + ["git", "show", "-s", "--format=%P", commit_sha], + cwd=ROOT, + check=True, + capture_output=True, + text=True, + ).stdout.strip() + tree_sha = subprocess.run( + ["git", "show", "-s", "--format=%T", commit_sha], + cwd=ROOT, + check=True, + capture_output=True, + text=True, + ).stdout.strip() + associated, result_commit, head_commit = _valid_live_squash_metadata( + namespace, + commit_sha=commit_sha, + tree_sha=tree_sha, + parent_sha=parent_sha, + ) + _install_live_squash_metadata(namespace, associated, result_commit, head_commit) + + def _valid_pull_request_event(namespace: dict[str, object]) -> dict[str, Any]: def repository() -> dict[str, Any]: return { @@ -1278,35 +1414,26 @@ def test_ci_has_read_only_permissions_and_immutable_action_pins() -> None: assert "actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97" in workflow -def test_public_release_audit_is_privacy_safe_and_passes() -> None: - result = subprocess.run( - ["python", "tools/audit_public_release_readiness.py", "--json"], - cwd=ROOT, - check=False, - capture_output=True, - text=True, - ) - assert result.returncode == 0, _privacy_safe_audit_failure(result) - report = json.loads(result.stdout) +def test_public_release_audit_is_privacy_safe_and_passes(monkeypatch: Any, capsys: Any) -> None: + namespace = _audit_namespace() + namespace["_current_pull_request_evidence"]() + _install_current_main_live_metadata(namespace) + monkeypatch.setattr("sys.argv", ["audit_public_release_readiness.py", "--json"]) + assert namespace["main"]() == 0 + output = capsys.readouterr().out + report = json.loads(output) assert report["classification"] == "PASS" assert report["privacy"]["sensitive_values_serialized"] == 0 assert report["privacy"]["approved_historical_path_fingerprints"] == 5 - assert "@" not in result.stdout - assert "/home/" not in result.stdout - assert "/tmp/" not in result.stdout - assert "\x1b" not in result.stdout + assert "@" not in output + assert "/home/" not in output + assert "/tmp/" not in output + assert "\x1b" not in output assert report["coverage"]["history_surfaces"] == sorted(report["coverage"]["history_surfaces"]) assert any("heuristic" in limitation for limitation in report["limitations"]) - repeated = subprocess.run( - ["python", "tools/audit_public_release_readiness.py", "--json"], - cwd=ROOT, - check=False, - capture_output=True, - text=True, - ) - assert repeated.returncode == 0, _privacy_safe_audit_failure(repeated) - assert repeated.stdout == result.stdout + assert namespace["main"]() == 0 + assert capsys.readouterr().out == output def test_public_release_audit_failure_diagnostics_are_privacy_safe() -> None: @@ -1661,6 +1788,285 @@ def test_signed_main_squash_identity_pair_passes_and_is_forward_safe() -> None: assert "bd53015609b7c3a08e106e0f5600dcd6c4fabc01" not in source +def test_subject_suffix_squash_recognition_remains_offline() -> None: + namespace = _audit_namespace() + observations, _ = _signed_squash_fixture(namespace) + builder = namespace["_github_signed_squash_commit_identity_evidence"] + builder_globals = builder.__globals__ + builder_globals["_normalized_origin_repository"] = lambda: namespace[ + "GITHUB_REPOSITORY_FULL_NAME" + ] + builder_globals["_cryptographically_verified_signer"] = lambda _: namespace[ + "GITHUB_WEB_FLOW_SIGNING_KEY_FINGERPRINT" + ] + + def fake_git(*args: str, input_bytes: bytes | None = None) -> bytes: + del input_bytes + if args[:3] == ("show", "-s", "--format=%s"): + return b"ordinary squash subject (#17)\n" + if args[:3] == ("show", "-s", "--format=%T"): + return ("7" * 40 + "\n").encode() + raise AssertionError(f"unexpected git call: {args!r}") + + def network_forbidden(_: str) -> object: + raise AssertionError("offline subject association must not use GitHub REST") + + builder_globals["_git"] = fake_git + builder_globals["_github_json_array"] = network_forbidden + builder_globals["_github_json"] = network_forbidden + result = builder(observations) + assert (result.status, result.reason_code) == ("AVAILABLE", "AVAILABLE") + assert len(result.evidence) == 1 + assert result.evidence[0].association_source == "OFFLINE_SUBJECT_SUFFIX" + assert result.evidence[0].live_actor_observation_supplied is False + + +def test_missing_subject_association_requires_complete_live_corroboration() -> None: + namespace = _audit_namespace() + associated, result_commit, head_commit = _valid_live_squash_metadata(namespace) + result = _run_live_squash_corroboration(namespace, associated, result_commit, head_commit) + assert (result.status, result.reason_code) == ("AVAILABLE", "AVAILABLE") + assert len(result.evidence) == 1 + evidence = result.evidence[0] + assert evidence.association_source == "LIVE_PUBLIC_ASSOCIATED_PULL_REQUEST" + assert evidence.live_actor_observation_supplied is True + assert evidence.repository_id == namespace["GITHUB_REPOSITORY_ID"] + assert evidence.base_sha == "8" * 40 + assert evidence.head_tree_sha == evidence.tree_sha == "7" * 40 + observations, _ = _signed_squash_fixture(namespace) + categories = [ + namespace["_classify_identity"](item, signed_squash_evidence=result.evidence) + for item in observations + ] + assert categories == [ + "VERIFIED_PLATFORM_MEDIATED_ACCOUNT_IDENTITY", + "VERIFIED_PLATFORM_SERVICE_IDENTITY", + ] + + +def test_missing_subject_zero_or_multiple_associations_fail_closed() -> None: + for associated_count in (0, 2): + namespace = _audit_namespace() + associated, result_commit, head_commit = _valid_live_squash_metadata(namespace) + selected = associated * associated_count + result = _run_live_squash_corroboration(namespace, selected, result_commit, head_commit) + assert (result.status, result.reason_code, result.evidence) == ( + "INVALID", + "ASSOCIATED_PR_COUNT_INVALID", + (), + ) + + +def test_missing_subject_open_or_unmerged_association_fails_closed() -> None: + for changes in ( + {"state": "open", "merged_at": None, "merge_commit_sha": None}, + {"state": "closed", "merged_at": None, "merge_commit_sha": None}, + ): + namespace = _audit_namespace() + associated, result_commit, head_commit = _valid_live_squash_metadata(namespace) + associated[0].update(changes) + result = _run_live_squash_corroboration(namespace, associated, result_commit, head_commit) + assert result.status == "INVALID" + assert result.evidence == () + + +def test_missing_subject_repository_id_and_base_branch_fail_closed() -> None: + mutations = ( + ("base", "repo", "full_name", "example/unrelated"), + ("head", "repo", "full_name", "example/unrelated"), + ("base", "repo", "id", 1), + ("head", "repo", "id", 1), + ("base", "ref", None, "release"), + ) + for first, second, third, value in mutations: + namespace = _audit_namespace() + associated, result_commit, head_commit = _valid_live_squash_metadata(namespace) + selected = associated[0][first][second] + if third is None: + associated[0][first][second] = value + else: + selected[third] = value + result = _run_live_squash_corroboration(namespace, associated, result_commit, head_commit) + assert result.status == "INVALID" + assert result.evidence == () + + +def test_missing_subject_result_parent_base_head_and_tree_bindings_fail_closed() -> None: + cases = ( + ("associated_result", "a" * 40), + ("associated_base", "a" * 40), + ("result_sha", "a" * 40), + ("result_parent", "a" * 40), + ("result_tree", "a" * 40), + ("head_sha", "a" * 40), + ("head_tree", "a" * 40), + ) + for field, value in cases: + namespace = _audit_namespace() + associated, result_commit, head_commit = _valid_live_squash_metadata(namespace) + if field == "associated_result": + associated[0]["merge_commit_sha"] = value + elif field == "associated_base": + associated[0]["base"]["sha"] = value + elif field == "result_sha": + result_commit["sha"] = value + elif field == "result_parent": + result_commit["parents"] = [{"sha": value}] + elif field == "result_tree": + result_commit["commit"]["tree"]["sha"] = value + elif field == "head_sha": + head_commit["sha"] = value + else: + head_commit["commit"]["tree"]["sha"] = value + result = _run_live_squash_corroboration(namespace, associated, result_commit, head_commit) + assert result.status == "INVALID" + assert result.evidence == () + + +def test_missing_subject_author_committer_signature_key_and_fingerprint_fail_closed() -> None: + cases = ( + ("pr_author", 1), + ("result_author", 1), + ("result_committer", 1), + ("signature_verified", False), + ("signature_reason", "unsigned"), + ) + for field, value in cases: + namespace = _audit_namespace() + associated, result_commit, head_commit = _valid_live_squash_metadata(namespace) + if field == "pr_author": + associated[0]["user"]["id"] = value + elif field == "result_author": + result_commit["author"]["id"] = value + elif field == "result_committer": + result_commit["committer"]["id"] = value + elif field == "signature_verified": + result_commit["commit"]["verification"]["verified"] = value + else: + result_commit["commit"]["verification"]["reason"] = value + result = _run_live_squash_corroboration(namespace, associated, result_commit, head_commit) + assert result.status == "INVALID" + assert result.evidence == () + + namespace = _audit_namespace() + associated, result_commit, head_commit = _valid_live_squash_metadata(namespace) + wrong_fingerprint = _run_live_squash_corroboration( + namespace, + associated, + result_commit, + head_commit, + signer_fingerprint="A" * 40, + ) + assert (wrong_fingerprint.status, wrong_fingerprint.reason_code) == ( + "INVALID", + "LOCAL_CANDIDATE_BINDING_INVALID", + ) + + observations, evidence = _signed_squash_fixture(namespace) + live = replace( + evidence[0], + association_source="LIVE_PUBLIC_ASSOCIATED_PULL_REQUEST", + live_actor_observation_supplied=True, + repository_id=namespace["GITHUB_REPOSITORY_ID"], + base_ref="main", + base_sha="8" * 40, + head_sha="9" * 40, + head_tree_sha="7" * 40, + author_actor_id=namespace["GITHUB_OWNER_ACTOR_ID"], + committer_actor_id=namespace["GITHUB_WEB_FLOW_ACTOR_ID"], + signature_verified=True, + signature_reason="valid", + ) + for changed in ( + replace(live, signature_key_id="DEADBEEFDEADBEEF"), + replace(live, signer_fingerprint="A" * 40), + replace(live, author_fingerprint="1" * 64), + replace(live, committer_fingerprint="2" * 64), + ): + categories = [ + namespace["_classify_identity"](item, signed_squash_evidence=(changed,)) + for item in observations + ] + assert "UNVERIFIED_PLATFORM_SERVICE_CLAIM" in categories + + +def test_missing_subject_malformed_and_unavailable_responses_fail_closed() -> None: + malformed_values = (None, {}, "unexpected", [None]) + for malformed in malformed_values: + namespace = _audit_namespace() + _, result_commit, head_commit = _valid_live_squash_metadata(namespace) + result = _run_live_squash_corroboration(namespace, malformed, result_commit, head_commit) + assert result.status == "INVALID" + assert result.evidence == () + + for endpoint in ("associated", "result", "head"): + namespace = _audit_namespace() + associated, result_commit, head_commit = _valid_live_squash_metadata(namespace) + result = _run_live_squash_corroboration( + namespace, + associated, + result_commit, + head_commit, + unavailable_endpoint=endpoint, + ) + assert result.status == "NOT_AVAILABLE" + assert result.evidence == () + + +def test_github_json_reads_are_bounded_shape_checked_and_cached(monkeypatch: Any) -> None: + namespace = _audit_namespace() + document = namespace["_github_json_document"] + object_reader = namespace["_github_json"] + array_reader = namespace["_github_json_array"] + document.cache_clear() + object_reader.cache_clear() + array_reader.cache_clear() + calls: list[str] = [] + payloads = [b"{}"] + + class Response: + status = 200 + + def __enter__(self) -> Response: + return self + + def __exit__(self, *args: object) -> None: + del args + + def read(self, limit: int) -> bytes: + del limit + return payloads[0] + + def fake_urlopen(*args: object, **kwargs: object) -> Response: + del kwargs + calls.append(str(args[0])) + return Response() + + monkeypatch.setattr(namespace["urllib"].request, "urlopen", fake_urlopen) + repository = namespace["GITHUB_REPOSITORY_FULL_NAME"] + url = f"https://api.github.com/repos/{repository}/commits/{'1' * 40}" + assert object_reader(url).available is True + assert object_reader(url).available is True + assert len(calls) == 1 + + cases = ( + (b"[]", object_reader, "2" * 40, "RESPONSE_SHAPE_INVALID"), + (b"{}", array_reader, "3" * 40, "RESPONSE_SHAPE_INVALID"), + (b'{"a":1,"a":2}', object_reader, "4" * 40, "RESPONSE_JSON_INVALID"), + ( + b" " * (namespace["MAX_GITHUB_RESPONSE_BYTES"] + 1), + object_reader, + "5" * 40, + "RESPONSE_LIMIT_EXCEEDED", + ), + ) + for payload, reader, sha, reason in cases: + payloads[0] = payload + selected = f"https://api.github.com/repos/{repository}/commits/{sha}" + result = reader(selected) + assert (result.available, result.reason_code) == (False, reason) + + def test_signed_squash_roles_categories_and_authority_are_distinct() -> None: namespace = _audit_namespace() observations, evidence = _signed_squash_fixture(namespace) @@ -1788,16 +2194,32 @@ def test_protected_squash_merge_evidence_is_separate_and_fail_closed() -> None: assert builder({**valid, "branch_protection": changed_protection}).status == "INVALID" -def test_real_authoritative_main_squash_evidence_is_offline_and_redacted() -> None: +def test_current_main_squash_evidence_passes_with_mocked_public_corroboration() -> None: namespace = _audit_namespace() + current_pull_request = namespace["_current_pull_request_evidence"]() + _install_current_main_live_metadata(namespace) result = namespace["_github_signed_squash_commit_identity_evidence"]() assert result.status == "AVAILABLE" - assert len(result.evidence) >= 1 - assert all(item.live_actor_observation_supplied is False for item in result.evidence) + assert result.safe_facts["live_public_associations"] == 1 + assert {item.association_source for item in result.evidence} == { + "OFFLINE_SUBJECT_SUFFIX", + "LIVE_PUBLIC_ASSOCIATED_PULL_REQUEST", + } + observations = namespace["_history_identity_observations"]() + check = namespace["_identity_classification_check"]( + observations, + pull_request_evidence=current_pull_request.evidence, + signed_squash_evidence=result.evidence, + ) + assert check.passed is True + assert "unverified_platform_claims=0" in check.detail + assert "unknown=0" in check.detail + assert "invalid=0" in check.detail serialized = json.dumps(asdict(result), sort_keys=True) assert "@" not in serialized assert "/home/" not in serialized assert "/tmp/" not in serialized + assert "token" not in serialized.lower() def test_existing_main_and_annotated_tag_privacy_behavior_is_unchanged() -> None: diff --git a/tools/audit_public_release_readiness.py b/tools/audit_public_release_readiness.py index ca5f46a..0ec888a 100644 --- a/tools/audit_public_release_readiness.py +++ b/tools/audit_public_release_readiness.py @@ -160,7 +160,7 @@ "GITHUB_REST_PULL_REQUEST_CORROBORATION", } ) -REQUIRED_SIGNED_SQUASH_PROVENANCE = frozenset( +REQUIRED_SIGNED_SQUASH_COMMON_PROVENANCE = frozenset( { "AUTHORITATIVE_MAIN_REACHABILITY", "EXACT_AUTHOR_COMMITTER_FINGERPRINT_PAIR", @@ -168,9 +168,26 @@ "LOCAL_CRYPTOGRAPHIC_SIGNATURE_VERIFICATION", "NORMALIZED_ORIGIN_REPOSITORY_IDENTITY", "REVIEWED_GITHUB_ACTOR_ASSOCIATION_POLICY", + } +) +OFFLINE_SIGNED_SQUASH_PROVENANCE = REQUIRED_SIGNED_SQUASH_COMMON_PROVENANCE | frozenset( + { "SQUASH_PR_SUBJECT_ASSOCIATION_SIGNAL", } ) +LIVE_SIGNED_SQUASH_PROVENANCE = REQUIRED_SIGNED_SQUASH_COMMON_PROVENANCE | frozenset( + { + "GITHUB_REST_ASSOCIATED_PULL_REQUEST_CORROBORATION", + "GITHUB_REST_COMMIT_ACTOR_AND_SIGNATURE_CORROBORATION", + "GITHUB_REST_HEAD_TREE_CORROBORATION", + "NORMALIZED_REPOSITORY_ID_CORROBORATION", + } +) +REQUIRED_SIGNED_SQUASH_PROVENANCE = OFFLINE_SIGNED_SQUASH_PROVENANCE | LIVE_SIGNED_SQUASH_PROVENANCE +SQUASH_ASSOCIATION_SOURCES = ( + "LIVE_PUBLIC_ASSOCIATED_PULL_REQUEST", + "OFFLINE_SUBJECT_SUFFIX", +) GITHUB_REPOSITORY_FULL_NAME = "200lz/open-model-integration-validator" GITHUB_REPOSITORY_ID = 1316060005 GITHUB_OWNER_ACTOR_ID = 145014769 @@ -204,6 +221,8 @@ SYNTHETIC_HOME_USERS = {"example", "synthetic", "user"} MAX_PUBLIC_FILE_BYTES = 5 * 1024 * 1024 MAX_PAYLOAD_FIXTURE_BYTES = 1024 +MAX_GITHUB_RESPONSE_BYTES = 1024 * 1024 +MAX_LIVE_SQUASH_CANDIDATES = 8 REQUIRED_PUBLIC_FILES = { "CHANGELOG.md", "CITATION.cff", @@ -372,6 +391,16 @@ class GithubSignedSquashCommitIdentityEvidence: observed_git_identity: bool reviewed_github_actor_association: bool live_actor_observation_supplied: bool + association_source: str = "OFFLINE_SUBJECT_SUFFIX" + repository_id: int | None = None + base_ref: str | None = None + base_sha: str | None = None + head_sha: str | None = None + head_tree_sha: str | None = None + author_actor_id: int | None = None + committer_actor_id: int | None = None + signature_verified: bool | None = None + signature_reason: str | None = None authority_limit: str = NO_PLATFORM_AUTHORITY @@ -395,7 +424,7 @@ class GitHubJsonResult: available: bool reason_code: str status_code: int - payload: dict[str, Any] | None = None + payload: Any = None PR1_HEAD_ALLOWED_REFS = ( @@ -790,8 +819,18 @@ def add( ] -def _github_json(url: str) -> GitHubJsonResult: - if not url.startswith("https://api.github.com/"): +def _reject_duplicate_json_keys(pairs: list[tuple[str, Any]]) -> dict[str, Any]: + result: dict[str, Any] = {} + for key, value in pairs: + if key in result: + raise ValueError("duplicate JSON key") + result[key] = value + return result + + +@functools.cache +def _github_json_document(url: str) -> GitHubJsonResult: + if not url.startswith(f"https://api.github.com/repos/{GITHUB_REPOSITORY_FULL_NAME}/"): return GitHubJsonResult(False, "URL_REJECTED", 0) request = urllib.request.Request( url, @@ -805,23 +844,37 @@ def _github_json(url: str) -> GitHubJsonResult: with urllib.request.urlopen( request, timeout=10, context=ssl.create_default_context() ) as response: - payload = response.read(1_048_577) + payload = response.read(MAX_GITHUB_RESPONSE_BYTES + 1) if response.status != 200: return GitHubJsonResult(False, "HTTP_STATUS_NOT_SUCCESS", response.status) - if len(payload) > 1_048_576: + if len(payload) > MAX_GITHUB_RESPONSE_BYTES: return GitHubJsonResult(False, "RESPONSE_LIMIT_EXCEEDED", response.status) - parsed = json.loads(payload) + parsed = json.loads(payload, object_pairs_hook=_reject_duplicate_json_keys) except urllib.error.HTTPError as exc: return GitHubJsonResult(False, "HTTP_STATUS_NOT_SUCCESS", exc.code) except (OSError, urllib.error.URLError): return GitHubJsonResult(False, "NETWORK_ERROR", 0) except (UnicodeError, ValueError, json.JSONDecodeError): return GitHubJsonResult(False, "RESPONSE_JSON_INVALID", 200) - if not isinstance(parsed, dict): - return GitHubJsonResult(False, "RESPONSE_SHAPE_INVALID", 200) return GitHubJsonResult(True, "AVAILABLE", 200, parsed) +@functools.cache +def _github_json(url: str) -> GitHubJsonResult: + result = _github_json_document(url) + if result.available and not isinstance(result.payload, dict): + return GitHubJsonResult(False, "RESPONSE_SHAPE_INVALID", result.status_code) + return result + + +@functools.cache +def _github_json_array(url: str) -> GitHubJsonResult: + result = _github_json_document(url) + if result.available and not isinstance(result.payload, list): + return GitHubJsonResult(False, "RESPONSE_SHAPE_INVALID", result.status_code) + return result + + def _pull_request_evidence_result( status: str, reason_code: str, @@ -1467,6 +1520,250 @@ def _cryptographically_verified_signer(commit_sha: str) -> str | None: ) +def _live_public_squash_commit_identity_evidence( + *, + repository: str, + commit_sha: str, + tree_sha: str, + parents: tuple[str, ...], + author_fingerprint: str, + committer_fingerprint: str, + signer_fingerprint: str, + authoritative_ref_classifications: tuple[str, ...], +) -> GithubSignedSquashEvidenceResult: + facts: dict[str, bool | int] = {"live_public_corroboration_attempted": True} + + def result( + status: str, + reason_code: str, + evidence: tuple[GithubSignedSquashCommitIdentityEvidence, ...] = (), + ) -> GithubSignedSquashEvidenceResult: + return GithubSignedSquashEvidenceResult( + status, reason_code, dict(sorted(facts.items())), evidence + ) + + local_bindings_valid = bool( + repository == GITHUB_REPOSITORY_FULL_NAME + and re.fullmatch(r"[0-9a-f]{40}", commit_sha) + and re.fullmatch(r"[0-9a-f]{40}", tree_sha) + and len(parents) == 1 + and re.fullmatch(r"[0-9a-f]{40}", parents[0]) + and author_fingerprint == GITHUB_SQUASH_AUTHOR_FINGERPRINT + and committer_fingerprint == GITHUB_SQUASH_COMMITTER_FINGERPRINT + and signer_fingerprint == GITHUB_WEB_FLOW_SIGNING_KEY_FINGERPRINT + and authoritative_ref_classifications + and set(authoritative_ref_classifications).issubset({"LOCAL_MAIN", "REMOTE_MAIN"}) + ) + facts["local_candidate_bindings_valid"] = local_bindings_valid + if not local_bindings_valid: + return result("INVALID", "LOCAL_CANDIDATE_BINDING_INVALID") + + associated_result = _github_json_array( + f"https://api.github.com/repos/{GITHUB_REPOSITORY_FULL_NAME}/commits/" + f"{commit_sha}/pulls?per_page=2" + ) + facts["associated_pr_metadata_available"] = associated_result.available + facts["associated_pr_metadata_status_code"] = associated_result.status_code + if not associated_result.available: + status = ( + "NOT_AVAILABLE" if _bounded_live_metadata_unavailable(associated_result) else "INVALID" + ) + return result(status, f"ASSOCIATED_PR_{associated_result.reason_code}") + associated = associated_result.payload + if not isinstance(associated, list): + return result("INVALID", "ASSOCIATED_PR_RESPONSE_SHAPE_INVALID") + facts["associated_pr_count"] = len(associated) + if len(associated) != 1: + return result("INVALID", "ASSOCIATED_PR_COUNT_INVALID") + pull_request = associated[0] + try: + if not isinstance(pull_request, dict): + raise TypeError + pr_number = pull_request["number"] + state = pull_request["state"] + merged_at = pull_request["merged_at"] + merge_commit_sha = pull_request["merge_commit_sha"] + pr_author_actor_id = pull_request["user"]["id"] + base_ref = pull_request["base"]["ref"] + base_sha = pull_request["base"]["sha"] + base_repository_id = pull_request["base"]["repo"]["id"] + base_repository_full_name = pull_request["base"]["repo"]["full_name"] + head_sha = pull_request["head"]["sha"] + head_repository_id = pull_request["head"]["repo"]["id"] + head_repository_full_name = pull_request["head"]["repo"]["full_name"] + except (KeyError, TypeError): + return result("INVALID", "ASSOCIATED_PR_RESPONSE_SCHEMA_INVALID") + pr_shape_valid = bool( + isinstance(pr_number, int) + and not isinstance(pr_number, bool) + and pr_number > 0 + and isinstance(state, str) + and (merged_at is None or isinstance(merged_at, str)) + and (merge_commit_sha is None or isinstance(merge_commit_sha, str)) + and isinstance(pr_author_actor_id, int) + and not isinstance(pr_author_actor_id, bool) + and isinstance(base_ref, str) + and isinstance(base_sha, str) + and isinstance(base_repository_id, int) + and not isinstance(base_repository_id, bool) + and isinstance(base_repository_full_name, str) + and isinstance(head_sha, str) + and isinstance(head_repository_id, int) + and not isinstance(head_repository_id, bool) + and isinstance(head_repository_full_name, str) + and re.fullmatch(r"[0-9a-f]{40}", base_sha) + and re.fullmatch(r"[0-9a-f]{40}", head_sha) + ) + facts["associated_pr_shape_valid"] = pr_shape_valid + if not pr_shape_valid: + return result("INVALID", "ASSOCIATED_PR_RESPONSE_SCHEMA_INVALID") + repository_bindings_match = bool( + base_repository_full_name == repository + and head_repository_full_name == repository + and base_repository_id == GITHUB_REPOSITORY_ID + and head_repository_id == GITHUB_REPOSITORY_ID + ) + facts["associated_pr_repository_matches"] = repository_bindings_match + if not repository_bindings_match: + return result("INVALID", "ASSOCIATED_PR_REPOSITORY_MISMATCH") + associated_pr_merged = bool( + state == "closed" + and isinstance(merged_at, str) + and merged_at + and isinstance(merge_commit_sha, str) + ) + facts["associated_pr_merged"] = associated_pr_merged + if not associated_pr_merged: + return result("INVALID", "ASSOCIATED_PR_NOT_MERGED") + if re.fullmatch(r"[0-9a-f]{40}", merge_commit_sha) is None: + return result("INVALID", "ASSOCIATED_PR_RESPONSE_SCHEMA_INVALID") + facts["associated_pr_base_ref_matches"] = base_ref == "main" + if base_ref != "main": + return result("INVALID", "ASSOCIATED_PR_BASE_REF_MISMATCH") + facts["associated_pr_result_matches"] = merge_commit_sha == commit_sha + if merge_commit_sha != commit_sha: + return result("INVALID", "ASSOCIATED_PR_RESULT_SHA_MISMATCH") + facts["associated_pr_base_matches_local_parent"] = base_sha == parents[0] + if base_sha != parents[0]: + return result("INVALID", "ASSOCIATED_PR_BASE_SHA_MISMATCH") + facts["associated_pr_author_actor_matches"] = pr_author_actor_id == GITHUB_OWNER_ACTOR_ID + if pr_author_actor_id != GITHUB_OWNER_ACTOR_ID: + return result("INVALID", "ASSOCIATED_PR_AUTHOR_ACTOR_MISMATCH") + + commit_result = _github_json( + f"https://api.github.com/repos/{GITHUB_REPOSITORY_FULL_NAME}/commits/{commit_sha}" + ) + facts["result_commit_metadata_available"] = commit_result.available + facts["result_commit_metadata_status_code"] = commit_result.status_code + if not commit_result.available: + status = "NOT_AVAILABLE" if _bounded_live_metadata_unavailable(commit_result) else "INVALID" + return result(status, f"RESULT_COMMIT_{commit_result.reason_code}") + commit = commit_result.payload + try: + if not isinstance(commit, dict): + raise TypeError + api_commit_sha = commit["sha"] + api_tree_sha = commit["commit"]["tree"]["sha"] + api_parents = tuple(item["sha"] for item in commit["parents"]) + author_actor_id = commit["author"]["id"] + committer_actor_id = commit["committer"]["id"] + signature_verified = commit["commit"]["verification"]["verified"] + signature_reason = commit["commit"]["verification"]["reason"] + except (KeyError, TypeError): + return result("INVALID", "RESULT_COMMIT_RESPONSE_SCHEMA_INVALID") + commit_shape_valid = bool( + isinstance(api_commit_sha, str) + and isinstance(api_tree_sha, str) + and all(isinstance(item, str) for item in api_parents) + and isinstance(author_actor_id, int) + and not isinstance(author_actor_id, bool) + and isinstance(committer_actor_id, int) + and not isinstance(committer_actor_id, bool) + and isinstance(signature_verified, bool) + and isinstance(signature_reason, str) + and re.fullmatch(r"[0-9a-f]{40}", api_commit_sha) + and re.fullmatch(r"[0-9a-f]{40}", api_tree_sha) + and all(re.fullmatch(r"[0-9a-f]{40}", item) for item in api_parents) + ) + facts["result_commit_shape_valid"] = commit_shape_valid + if not commit_shape_valid: + return result("INVALID", "RESULT_COMMIT_RESPONSE_SCHEMA_INVALID") + result_bindings_match = bool( + api_commit_sha == commit_sha and api_tree_sha == tree_sha and api_parents == parents + ) + facts["result_commit_git_bindings_match"] = result_bindings_match + if not result_bindings_match: + return result("INVALID", "RESULT_COMMIT_GIT_BINDING_MISMATCH") + actor_bindings_match = bool( + author_actor_id == GITHUB_OWNER_ACTOR_ID and committer_actor_id == GITHUB_WEB_FLOW_ACTOR_ID + ) + facts["result_commit_actor_bindings_match"] = actor_bindings_match + if not actor_bindings_match: + return result("INVALID", "RESULT_COMMIT_ACTOR_MISMATCH") + signature_bindings_match = bool(signature_verified is True and signature_reason == "valid") + facts["result_commit_signature_bindings_match"] = signature_bindings_match + if not signature_bindings_match: + return result("INVALID", "RESULT_COMMIT_SIGNATURE_MISMATCH") + + head_result = _github_json( + f"https://api.github.com/repos/{GITHUB_REPOSITORY_FULL_NAME}/commits/{head_sha}" + ) + facts["head_commit_metadata_available"] = head_result.available + facts["head_commit_metadata_status_code"] = head_result.status_code + if not head_result.available: + status = "NOT_AVAILABLE" if _bounded_live_metadata_unavailable(head_result) else "INVALID" + return result(status, f"HEAD_COMMIT_{head_result.reason_code}") + head = head_result.payload + try: + if not isinstance(head, dict): + raise TypeError + api_head_sha = head["sha"] + head_tree_sha = head["commit"]["tree"]["sha"] + except (KeyError, TypeError): + return result("INVALID", "HEAD_COMMIT_RESPONSE_SCHEMA_INVALID") + head_bindings_match = bool( + isinstance(api_head_sha, str) + and isinstance(head_tree_sha, str) + and re.fullmatch(r"[0-9a-f]{40}", api_head_sha) + and re.fullmatch(r"[0-9a-f]{40}", head_tree_sha) + and api_head_sha == head_sha + and head_tree_sha == tree_sha + and head_tree_sha == api_tree_sha + ) + facts["head_tree_matches_result_tree"] = head_bindings_match + if not head_bindings_match: + return result("INVALID", "HEAD_COMMIT_TREE_BINDING_MISMATCH") + + facts["all_live_public_bindings_match"] = True + evidence = GithubSignedSquashCommitIdentityEvidence( + valid=True, + repository_full_name=repository, + commit_sha=commit_sha, + tree_sha=tree_sha, + parents=parents, + pr_number=pr_number, + author_fingerprint=author_fingerprint, + committer_fingerprint=committer_fingerprint, + signature_key_id=GITHUB_WEB_FLOW_SIGNING_KEY_ID, + signer_fingerprint=signer_fingerprint, + authoritative_ref_classifications=authoritative_ref_classifications, + observed_git_identity=True, + reviewed_github_actor_association=True, + live_actor_observation_supplied=True, + association_source="LIVE_PUBLIC_ASSOCIATED_PULL_REQUEST", + repository_id=GITHUB_REPOSITORY_ID, + base_ref="main", + base_sha=base_sha, + head_sha=head_sha, + head_tree_sha=head_tree_sha, + author_actor_id=author_actor_id, + committer_actor_id=committer_actor_id, + signature_verified=signature_verified, + signature_reason=signature_reason, + ) + return result("AVAILABLE", "AVAILABLE", (evidence,)) + + def _github_signed_squash_commit_identity_evidence( observations: list[IdentityObservation] | None = None, ) -> GithubSignedSquashEvidenceResult: @@ -1494,6 +1791,8 @@ def _github_signed_squash_commit_identity_evidence( item.object_sha: item for item in committer.occurrences if item.role == "COMMITTER" } result: list[GithubSignedSquashCommitIdentityEvidence] = [] + live_results: list[GithubSignedSquashEvidenceResult] = [] + live_candidate_count = 0 for commit_sha in sorted(set(author_occurrences) & set(committer_occurrences)): author_occurrence = author_occurrences[commit_sha] committer_occurrence = committer_occurrences[commit_sha] @@ -1524,14 +1823,37 @@ def _github_signed_squash_commit_identity_evidence( tree_sha = _git("show", "-s", "--format=%T", commit_sha).decode("ascii").strip() except (OSError, UnicodeError, subprocess.CalledProcessError): continue - association = re.search(r"\(#([1-9][0-9]*)\)$", subject) signer = _cryptographically_verified_signer(commit_sha) if ( - association is None - or re.fullmatch(r"[0-9a-f]{40}", tree_sha) is None + re.fullmatch(r"[0-9a-f]{40}", tree_sha) is None or signer != GITHUB_WEB_FLOW_SIGNING_KEY_FINGERPRINT ): continue + association = re.search(r"\(#([1-9][0-9]*)\)$", subject) + if association is None: + live_candidate_count += 1 + if live_candidate_count > MAX_LIVE_SQUASH_CANDIDATES: + live_results.append( + GithubSignedSquashEvidenceResult( + "INVALID", + "LIVE_PUBLIC_CANDIDATE_LIMIT_EXCEEDED", + {"live_public_candidate_limit": MAX_LIVE_SQUASH_CANDIDATES}, + ) + ) + continue + live_result = _live_public_squash_commit_identity_evidence( + repository=repository, + commit_sha=commit_sha, + tree_sha=tree_sha, + parents=author_occurrence.parents, + author_fingerprint=author.fingerprint, + committer_fingerprint=committer.fingerprint, + signer_fingerprint=signer, + authoritative_ref_classifications=authoritative, + ) + live_results.append(live_result) + result.extend(live_result.evidence) + continue result.append( GithubSignedSquashCommitIdentityEvidence( valid=True, @@ -1548,9 +1870,34 @@ def _github_signed_squash_commit_identity_evidence( observed_git_identity=True, reviewed_github_actor_association=True, live_actor_observation_supplied=False, + association_source="OFFLINE_SUBJECT_SUFFIX", ) ) + facts["offline_subject_associations"] = sum( + item.association_source == "OFFLINE_SUBJECT_SUFFIX" for item in result + ) + facts["live_public_association_candidates"] = live_candidate_count + facts["live_public_associations"] = sum( + item.association_source == "LIVE_PUBLIC_ASSOCIATED_PULL_REQUEST" for item in result + ) + facts["live_public_corroboration_failures"] = sum( + item.status != "AVAILABLE" for item in live_results + ) facts["verified_squash_commits"] = len(result) + failed_live_results = [item for item in live_results if item.status != "AVAILABLE"] + if failed_live_results: + failed = failed_live_results[0] + status = ( + "INVALID" + if any(item.status == "INVALID" for item in failed_live_results) + else failed.status + ) + return GithubSignedSquashEvidenceResult( + status, + failed.reason_code, + dict(sorted(facts.items())), + tuple(result), + ) return GithubSignedSquashEvidenceResult( "AVAILABLE" if result else "INVALID", "AVAILABLE" if result else "REVIEWED_SQUASH_IDENTITY_NOT_VERIFIED", @@ -1787,7 +2134,34 @@ def _platform_occurrence_matches( == item.paired_fingerprint and squash.observed_git_identity and squash.reviewed_github_actor_association - and not squash.live_actor_observation_supplied + and ( + ( + squash.association_source == "OFFLINE_SUBJECT_SUFFIX" + and not squash.live_actor_observation_supplied + and squash.repository_id is None + and squash.base_ref is None + and squash.base_sha is None + and squash.head_sha is None + and squash.head_tree_sha is None + and squash.author_actor_id is None + and squash.committer_actor_id is None + and squash.signature_verified is None + and squash.signature_reason is None + ) + or ( + squash.association_source == "LIVE_PUBLIC_ASSOCIATED_PULL_REQUEST" + and squash.live_actor_observation_supplied + and squash.repository_id == GITHUB_REPOSITORY_ID + and squash.base_ref == "main" + and squash.base_sha == squash.parents[0] + and re.fullmatch(r"[0-9a-f]{40}", squash.head_sha or "") + and squash.head_tree_sha == squash.tree_sha + and squash.author_actor_id == GITHUB_OWNER_ACTOR_ID + and squash.committer_actor_id == GITHUB_WEB_FLOW_ACTOR_ID + and squash.signature_verified is True + and squash.signature_reason == "valid" + ) + ) and squash.authority_limit == NO_PLATFORM_AUTHORITY for item in policy.signed_squash_roles for squash in signed_squash_evidence @@ -2309,10 +2683,11 @@ def main() -> int: "purpose": policy.purpose, "signed_squash_roles": [ { + "association_sources": list(SQUASH_ASSOCIATION_SOURCES), "authoritative_ref_classifications": list( item.authoritative_ref_classifications ), - "live_actor_observation_required": False, + "live_actor_observation_required_for_missing_subject": True, "paired_fingerprint": item.paired_fingerprint, "parent_count": item.parent_count, "repository_full_name": item.repository_full_name, @@ -2359,7 +2734,8 @@ def main() -> int: "exact_local_and_remote_ref_scopes", "github_actions_pull_request_event_binding", "github_rest_actor_and_signature_corroboration", - "github_signed_authoritative_main_squash_identity", + "github_signed_authoritative_main_squash_live_public_association", + "github_signed_authoritative_main_squash_offline_subject_association", "pull_request_base_head_merge_parent_topology", "reachable_blob_sizes", "reachable_commit_patches", @@ -2377,6 +2753,9 @@ def main() -> int: "Offline GitHub-signed squash identity evidence proves only a reviewed Git " "identity occurrence and never proves PR approval, branch protection, checks, " "owner authority, release authority, or publication authority.", + "A missing squash-subject association requires bounded live public GitHub commit " + "and associated-PR corroboration; absence, ambiguity, or contradiction fails closed " + "and grants no owner, publisher, maintainer, release, or repository authority.", ], "privacy": { "approved_author_identity": "OWNER_APPROVED", @@ -2392,21 +2771,31 @@ def main() -> int: "evidence": [ { "author_fingerprint": item.author_fingerprint, + "association_source": item.association_source, "authoritative_ref_classifications": list( item.authoritative_ref_classifications ), "authority_limit": item.authority_limit, "commit_sha": item.commit_sha, "committer_fingerprint": item.committer_fingerprint, + "author_actor_id": item.author_actor_id, + "base_ref": item.base_ref, + "base_sha": item.base_sha, + "committer_actor_id": item.committer_actor_id, + "head_sha": item.head_sha, + "head_tree_sha": item.head_tree_sha, "live_actor_observation_supplied": (item.live_actor_observation_supplied), "observed_git_identity": item.observed_git_identity, "parents": list(item.parents), "pr_number": item.pr_number, "repository_full_name": item.repository_full_name, + "repository_id": item.repository_id, "reviewed_github_actor_association": ( item.reviewed_github_actor_association ), "signature_key_id": item.signature_key_id, + "signature_reason": item.signature_reason, + "signature_verified": item.signature_verified, "signer_fingerprint": item.signer_fingerprint, "tree_sha": item.tree_sha, "valid": item.valid,