Verify PRML (Pre-Registered ML Manifest) commitments in CI. Block merges when an evaluation claim was tampered, regressed below threshold, or never locked. Optionally anchor the hash to the public registry.

TL;DR — drop this in your workflow:

- uses: studio-11-co/prml-verify-action@v1

If .falsify/ exists in your repo, this will fail the build on any tampered or regressed claim. That's it. The defaults are sane for most CI setups.

PRML is a small open spec (CC BY 4.0) for committing an ML evaluation claim — metric, threshold, dataset split, model version — to a SHA-256 hash before the run. The hash is a tamper-evident receipt anyone can re-derive against the canonical bytes.

This Action wraps the falsify reference CLI (and its falsify-engine companion) as a composite GitHub Action. Modes:

It can also optionally POST your manifest to registry.falsify.dev so the receipt is publicly re-derivable from a permalink (with a README badge).

• Block post-hoc threshold tuning at merge time. If someone edits the locked manifest after the experiment ran, the hash changes and CI fails with exit code 3.
• Make eval claims falsifiable. A reviewer or regulator who reads your paper or model card can re-derive the SHA-256 from the canonical bytes and confirm the threshold was fixed before the data.
• Five-line integration. No service to deploy, no account to register. The Action runs falsify from PyPI; the optional registry is content-addressed and stateless beyond the hash.

What this does not solve, named explicitly in §8.1 of the spec: selective non-publication. You can pre-register ten claims and report two. PRML is a commitment primitive, not a complete publication-integrity system.

pip install falsify
falsify init accuracy.prml.yaml    # skeleton manifest
# edit accuracy.prml.yaml: metric, comparator, threshold, dataset.hash, seed, producer
falsify lock accuracy.prml.yaml    # canonicalize + SHA-256 + write accuracy.prml.prml.sha256

Commit accuracy.prml.yaml (and its .prml.sha256 sidecar) to your repo, then gate CI with mode: manifest.

The .falsify/<name>/ workflow-engine flow (init → lock → run → verdict → guard) uses the falsify-engine command from the same install; drive it with mode: guard / verdict.

# .github/workflows/prml.yml
name: PRML
on: [push, pull_request]
jobs:
  verify:
    runs-on: ubuntu-22.04
    steps:
      - uses: actions/checkout@v4
      - uses: studio-11-co/prml-verify-action@v1

That's it. Now any commit that tampers with a locked claim, or any model-version drift that pushes the metric below the threshold, fails the build.

- uses: actions/checkout@v4
- uses: studio-11-co/prml-verify-action@v1

- uses: actions/checkout@v4
- uses: studio-11-co/prml-verify-action@v1
  with:
    mode: verdict
    claim: imagenet-resnet50-baseline
    expected-hash: fb7403c40afe63d892bf4aea2c123fdd7fe85366b74a277875465c4cb3cbf19c

For repos that ship one PRML manifest at a known path and only want a schema-shape check in CI:

- uses: actions/checkout@v6
- uses: studio-11-co/prml-verify-action@v1
  with:
    mode: schema-only
    manifest-path: claims/cv-screening.prml.yaml
    schema-version: v0.1

Exit codes: 0 PASS, 10 schema validation FAIL, 2 IO / bad inputs. Works against the live JSON Schemas at https://spec.falsify.dev/schema/ (the same ones merged into the SchemaStore catalog).

- uses: actions/checkout@v4
- id: prml
  uses: studio-11-co/prml-verify-action@v1
  with:
    mode: verdict
    claim: imagenet-resnet50-baseline
    anchor-to-registry: 'true'
- name: Show badge snippet
  run: echo "${{ steps.prml.outputs.badge-snippet }}"

The badge-snippet output is a paste-ready Markdown string like:

[![PRML locked · fb7403c40afe](https://registry.falsify.dev/badge/<hash>.svg)](https://registry.falsify.dev/<hash>)

- uses: actions/checkout@v4
- uses: studio-11-co/prml-verify-action@v2
  with:
    mode: guard
    falsify-version: 0.3.1
- run: ./deploy-to-prod.sh
  if: steps.prml.outputs.status == 'pass'

More examples in the examples/ directory.

When anchor-to-registry: 'true':

1. The Action POSTs the canonical manifest YAML to ${registry-url}/commit
2. The registry recomputes SHA-256 over the same canonical bytes
3. The registry returns {hash, permalink, timestamp} and stores the manifest publicly
4. The Action emits the permalink and a paste-ready badge-snippet output

The registry has no auth and no server-side state beyond the hash. It exists for discoverability — your manifest can be anchored anywhere (a public git repo, an arXiv preprint, a Sigstore attestation, your own static host). The registry is one option, not a requirement.

Read more: What is PRML? · PRML vs in-toto, SLSA, Model Cards, HELM

If your workflow already runs through MLflow, prml-verify-action pairs well with mlflow-falsify — a tiny MLflow Run Context Provider plugin (pip install mlflow-falsify) that auto-tags every mlflow.start_run() with the PRML manifest hash plus the metric, comparator, threshold, and dataset id. Zero workflow change required. Source: studio-11-co/mlflow-falsify.

For the JavaScript / Node.js side of the same canonicalization: falsify-js on npm.

For execution-integrity beyond what PRML §8.1 covers, pair this action with cosign — the cookbook walks through it as Pattern 11: PRML + Sigstore.

Subcategory-by-subcategory maps from major AI governance frameworks to PRML fields (FULL / PARTIAL / NONE tagged):

• EU AI Act Article 12 — code-level pattern for the 2 December 2027 high-risk deadline
• NIST AI RMF 1.0 — GOVERN / MAP / MEASURE / MANAGE subcategory map
• ISO/IEC 42001:2023 — AI Management System clause-by-clause evidence map

• PRML v0.1 stable; falsify CLI 0.3.1 (PyPI)
• v0.2 RFC frozen: 2026-05-22 (comment window closed)
• Spec license: CC BY 4.0
• Reference implementations: Python · JavaScript · Go · Rust (byte-equivalent across 21 conformance vectors: 13 v0.1 + 8 v0.2)

If you find a gap or edge case the spec doesn't handle, open an issue before the freeze date — that's how v0.2 gets sharper.

• Spec format dual-track. The PRML manifest format described at spec.falsify.dev/v0.1 (9 YAML fields, content-addressed) and the falsify CLI's internal hypothesis.schema.yaml (claim / falsification / experiment) are currently distinct. The reconciliation work is part of the v0.2 RFC freeze (2026-05-22). Until then, use the PRML format for registry commits and the hypothesis schema for falsify lock flows.

This Action is MIT-licensed. The PRML specification is CC BY 4.0. The reference falsify toolkit is MIT.

Maintained by Cüneyt Öztürk · PRML / falsify track lead: Cüneyt Öztürk · hello@falsify.dev

• v0.1 stable. v0.2 RFC open through 2026-05-22 — spec.falsify.dev/v0.2-rfc.
• The PRML JSON Schema is in the SchemaStore catalog (merged 2026-05-11), so *.prml.yaml files autocomplete in VS Code, JetBrains, Helix, Zed, and Cursor out of the box.

See CONTRIBUTING.md and the good first issue label for scoped work.

Cite the spec: Öztürk, C. (2026). PRML v0.1. Zenodo. https://doi.org/10.5281/zenodo.20177839