RFC v0.2 — five proposals open for comment until 2026-05-22

[sk8ordie84]

RFC v0.2 — open for community comment

The full draft is at spec.falsify.dev/v0.2-rfc.
Freeze: 2026-05-22 23:59 UTC.

Five proposals

• P-01 Streaming variant (prml_mode: streaming for live evaluations)
• P-02 Runner attestation (optional URI to an out-of-band attestation)
• P-03 Revocation primitive (revoked_at, revocation_reason)
• P-04 Conformance vector format (standardise the test-vector layout)
• P-05 Patent grant placement (move grant text from appendix to preamble)

How to comment

Reply to this discussion with format:

Proposal: P-0X
Position: support / oppose / clarify
Rationale: [≤ 200 words]
Affected impl(s): py / js / go / rust / all (if applicable)
Suggested edit (optional): [diff]

Or open a new discussion in the RFC v0.2 — comments category for a deeper thread.

What this RFC does and doesn't do

• ✅ Adds optional fields. Every v0.1 manifest stays a valid v0.2 manifest.
• ✅ Documents the patent non-assertion grant in a more standards-friendly location.
• ❌ Breaks v0.1 hashes. Non-negotiable: the canonicalisation rules preserve hash-equivalence for v0.1-shaped inputs.
• ❌ Solves selective publication. PRML §8.1 explicitly does not.

Comment summary template (optional but useful)

If your comment is longer than ~3 paragraphs, a TL;DR at the top helps maintainers triage.

— Cüneyt

[sk8ordie84]

Status update — 8 days to freeze

A few signals have accumulated since this RFC was opened, worth surfacing for anyone tracking:

• 2026-05-11: PRML JSON Schema merged into the SchemaStore catalog by Mads Kristensen (Microsoft). *.prml.yaml files now autocomplete in VS Code, JetBrains, Helix, Zed, and Cursor out of the box.
• 2026-05-14: v0.1 published on Zenodo with a citable DOI: 10.5281/zenodo.20177839. The concept DOI always resolves to the latest version; v0.2 will get its own version DOI after freeze.
• 2026-05-14: Submission filed to the OECD.AI Catalogue of Tools & Metrics for Trustworthy AI; secretariat vetting in progress.
• 2026-05-14: Late comment filed to NIST AI 800-2 (AI800-2@nist.gov) mapping PRML to §4 contamination risk and §6 reporting integrity.

Freeze: 2026-05-22 23:59 UTC, 8 days from now.

If you have substantive feedback on any of P-01 through P-05, this is the leverage window. After freeze:

• 2026-05-22: spec text frozen, conformance vectors finalized
• 2026-05-23+: v0.2 schema PR to SchemaStore, reference impls published with v0.2 support
• 2026-05-26+: Zenodo deposit for v0.2 with new version DOI

No formal endorsement required for comment. Reply here, or open a new Discussion in the RFC v0.2 — comments category for a deeper thread.

— Cüneyt

[topeuph-ai]

Proposal: P-02
Position: Support + concrete example

Rationale:

Pattern 10 in the cookbook names the gap well: the auditor role
(aggregating per-hash observations from multiple producers) has no
tooling yet, and the v0.3 registry page is the planned answer.
P-02's attestation_uri is the right hook for a stronger solution
at the protocol layer.

We're building ValiChord ( github.com/topeuph-ai/ValiChord ) — a
commit-reveal protocol where independent validators each commit a
hashed verdict before any reveals happen. No party sees another's
result before committing. The resulting HarmonyRecord — a
tamper-evident aggregated outcome record written to a public DHT
once all validators have revealed — would be a natural
attestation_uri value.

The property that makes attestation_uri useful beyond what
Pattern 11 + Sigstore already gives you: the referenced system must
enforce pre-commitment by parties who had no access to each other's
verdicts. Sigstore proves who ran and when; it doesn't prevent
running 10 times and signing only the best result. Blind
commit-reveal does.

Suggested addition to the P-02 field note: distinguish between
execution attestation (who ran, when — Sigstore) and
independence attestation (distinct parties couldn't coordinate
outcomes — commit-reveal). They're both useful; they solve
different parts of the §8.1 gap.

Affected impl(s): None (optional field, no parsing changes required)

Suggested addition to the P-02 field note: distinguish between
execution attestation (who ran, when — Sigstore) and
independence attestation (distinct parties couldn't coordinate
outcomes — commit-reveal). They're both useful; they solve
different parts of the §8.1 gap.

Affected impl(s): None (optional field, no parsing changes required)

[sk8ordie84]

Ceri, this is the kind of comment the RFC was opened to find. The execution vs independence split is the right distinction, and it cleans up confusion P-02 had baked in. Two things worth flagging before tomorrow's freeze.

First, the field-note language you suggested ("distinguish execution attestation from independence attestation") will go into the v0.2 closing decisions verbatim if you're fine with attribution. If you'd rather stay uncredited, say the word.

Second, ValiChord plus a PRML attestation_uri pointing at a HarmonyReceipt is the exact pattern §8.1 calls out as needing a companion mechanism. If you'd write Cookbook Pattern 13 (PRML + commit-reveal) as a co-authored entry alongside Pattern 11 (PRML + Sigstore), happy to make space.

Freeze is 2026-05-22 23:59 UTC. Anything that doesn't land in v0.2 lands in v0.3 backlog with attribution.

[topeuph-ai]

Glad it landed. Two answers:

Attribution: yes, use it.

Pattern 13: I'd write it.

The reason I recognised the synergy immediately is that ValiChord has a researcher commit built into the protocol — the researcher seals their result hash to the DHT before any validator begins work, which is structurally the same move as falsify lock. The difference is that ValiChord wraps that commitment in independent multi-party verification on top.

The problem is what Pattern 11 leaves open. Sigstore gives you execution attestation — who ran, when, against which artefacts. Pattern 13 adds independence attestation: distinct parties
committed to their verdicts blind, before any reveals happened, so coordination after the fact is structurally impossible rather than just contractually prohibited.

The workflow stacks three layers. First, falsify lock pre-registers the claim (metric, threshold, dataset hash, seed) before the experiment runs — that's PRML doing its job. Second, the researcher runs the eval and produces a valichord_attestation bundle: a SHA-256 Merkle root over per-sample outputs that binds the reported aggregate to the underlying run. The bundle's bundle_hash is what gets submitted to ValiChord as the dataset commitment. Third, independent validators each reproduce the eval blind — they each seal a hashed verdict before any reveals happen, the reveal window opens only once all validators have committed, and attestations are immutable once written. The resulting HarmonyRecord (our term for the aggregated outcome written to a public DHT) becomes the attestation_uri value in the PRML manifest.

"What it doesn't fix" will be honest: the blind commit prevents verdict coordination, not withdrawal — a validator who dislikes their result can still go silent. PRML §8.1 stands on selective non-publication; the DHT makes divergence visible but doesn't compel publication.

One note on the integration status: valichord_attestation and the Holochain protocol have all the right primitives — wiring the bundle hash as the DHT commitment is planned work, so the pattern will describe the intended architecture clearly rather than claiming it's fully deployed today.

Happy to submit as a PR to falsify-cookbook. What's your preferred attribution format for co-authored patterns?

[sk8ordie84]

Three things back.

Attribution format for co-authored patterns: top-of-file metadata block, paper-style.

Authors: Cüneyt Öztürk (Studio 11, falsify) and Ceri John (Topeuph AI, ValiChord)
License: CC0-1.0 (cookbook default)

I'll add this to Pattern 11 retroactively so the cookbook stays consistent. If you'd prefer a different framing (single author with acknowledgment, or order reversed), happy to adjust.

The in-progress vs deployed honesty in your last paragraph is the right call. PRML §8.1 readers are skeptical of "solved everything" claims, and Pattern 11 already names what Sigstore doesn't cover. Pattern 13 doing the same for ValiChord lands cleaner than overpromising.

PR target: falsify-cookbook repo, file at patterns/13-commit-reveal-validation.md following the existing numbering. I'll review within 24 hours of submission. If you'd rather draft in a gist first for back-and-forth before the PR, that works too.

v0.2 closing note will reference Pattern 13 as "draft in progress with Ceri John, lands in v0.2 closing addendum or v0.3 backlog depending on PR timing." Attribution appears tomorrow regardless of PR merge timing.

[topeuph-ai]

Attribution works as proposed.

Direct PR rather than gist — pattern is clear enough. Filing to patterns/13-commit-reveal-validation.md today.

One correction to my earlier sketch before it goes in writing: I described bundle_hash as "what gets submitted to ValiChord as the dataset commitment." That was imprecise. ValiChord's data_hash is the SHA-256 of the raw data deposit (code + data archive) — that's what validators download and reproduce against. The attestation bundle is a separate artefact published
alongside the dataset; validators can independently verify the researcher's claimed outputs_merkle_root by building their own bundle from the same run. Two distinct commitments, not one.

The "what this still doesn't cover" section will be explicit about both: (a) validators don't currently commit to their own bundle hashes — the protocol binds to verdict, not to a verified reproduction bundle; (b) the integration is manual stitching today, not a single command.

[topeuph-ai]

Freeze notes for v0.3 claim-tree

Pattern 13 is filed; thank you for the co-author credit in the changelog.

Looking at the v0.3 backlog before the window closes — the claim-tree open question on partial publication detection is one we have solved at a different layer and worth cross-pollinating.

In valichord_attestation (our Merkle library for AI evaluation attestations), the bundle commits a samples_total count upfront — the full number of evaluation instances the researcher processed — before any selective disclosure. A verifier who later receives a Merkle-proof batch smaller than samples_total knows unconditionally that items were omitted. No suite-level wrapper needed; the count is part of the leaf commitment.

That approach could answer the backlog fork of "enumerate required leaves up front vs list observed leaves": the suite document could commit to a leaves_total count at lock-time (analogous to samples_total), making silent drops detectable without requiring every leaf to be enumerated in the suite document itself.

On format inputs: we have an adapter for inspect_ai .eval files. The lm-eval-harness JSON format is structurally similar. Is there a concrete sample output you would want mapped when the rfc-v0.3 issue opens, or is the backlog's "link to the format docs" ask sufficient for now?

[sk8ordie84]

Both notes land. The leaves_total analog to samples_total is the cleaner answer to the enumerate-up-front-vs-observed question. I just added it to spec/v0.3-backlog/01-claim-tree.md with attribution and the symmetry note (commit efcf86f).

On format inputs: link to docs is enough right now. The v0.3 RFC issue will probably want one worked example, but selecting which one is itself part of the v0.3 design conversation, not a pre-freeze decision. When the rfc-v0.3 issue opens (target Q4 2026), the inspect_ai .eval adapter you have will be the natural starting case to test the claim-tree shape against.

The valichord_attestation library is referenced in Pattern 13 already; linking it from the v0.3 backlog as a precedent for the bundle-side count commitment is the addition I just made. Want me to use a stable release tag once you have one, or is the current main branch citation acceptable?

[topeuph-ai]

Stable release tag please — v0.5.3 just cut: https://github.com/topeuph-ai/ValiChord/releases/tag/v0.5.3

valichord_attestation is at 1.2.0 in that tag: format v1.2 (Metric.filter, Bundle.meta, dual bundle_hash/content_hash, samples_total), 326 tests. Source archives attached.

[sk8ordie84]

Pinned. Updated 01-claim-tree.md to cite ValiChord v0.5.3 with valichord_attestation 1.2.0 format spec. Congrats on the release.