Skip to content

Commit dbc9c14

Browse files
authored
fix: require locked pilot preregistration
Fail closed on unlocked PRR records while preserving editable drafts; aligned tests and docs with the existing pre-experiment freeze contract.
1 parent 8e4cc5b commit dbc9c14

5 files changed

Lines changed: 43 additions & 8 deletions

File tree

‎docs/evidence/METRICS_CURRENT.md‎

Lines changed: 9 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -5,9 +5,9 @@ Machine-readable snapshot: `outputs/metrics_snapshot.json` regenerated with `mak
55
> **Purpose:** One authoritative table of current pipeline metrics. If any doc disagrees
66
> with this file, this file wins. Updated whenever benchmark/benchmark config changes.
77
>
8-
> **Last updated:** 2026-08-22 (lab-result report schema validation; benchmark values unchanged)
8+
> **Last updated:** 2026-08-23 (pilot pre-registration lock enforcement; benchmark values unchanged)
99
10-
> **Current verification note (2026-08-22):** Phase AA AA6 exposes the AARG-
10+
> **Current verification note (2026-08-23):** Phase AA AA6 exposes the AARG-
1111
> reproducibility aggregate through a repeatable CLI/make workflow, while Phase
1212
> AB AB5 exposes the ABAG- claim-integrity aggregate, Phase AC AC3 exposes the
1313
> ACDG- aggregate, Phase Y Y5 exposes the YAG- aggregate, and Phase Z Z5
@@ -100,10 +100,16 @@ Machine-readable snapshot: `outputs/metrics_snapshot.json` regenerated with `mak
100100
> Phase AC AC3 exposes the ACDG-
101101
> aggregate disconfirming-evidence gate as a repeatable CLI/make workflow. It
102102
> has 18 focused gate tests plus 2 CLI integration tests. Full pytest
103-
> collection succeeds at 12,423 tests; the live count is enforced by the
103+
> collection succeeds at 12,424 tests; the live count is enforced by the
104104
> current-state alignment test. This artifact does not establish
105105
> biological validation or benchmark improvement.
106106
107+
> **Pilot pre-registration lock note (2026-08-23):** The PRR- validator now
108+
> rejects unlocked records as invalid for experiment start. Unlocked drafts can
109+
> still be edited, but a valid pre-registration must freeze its selection
110+
> criteria and thresholds first. This is anti-cherry-picking infrastructure,
111+
> not assay validation or biological evidence.
112+
107113
> **Canonical ERP generator note (2026-08-05):** `make generate-review-packet`
108114
> now exercises the V4 component-based ERP contract and emits a draft when no
109115
> artifact references are supplied. The legacy placeholder generator remains

‎docs/research/ROADMAP.md‎

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Roadmap
22

3-
## Current state — 2026-08-22
3+
## Current state — 2026-08-23
44

55
Phase AB is complete as of 2026-07-26. AB5 exposes the existing CSD-, RDR-,
66
EGN-, and EHP- claim-integrity artifacts through an ABAG- aggregate, CLI
@@ -183,6 +183,12 @@ control-failure, raw-data, and input-blocker fields. This makes the report
183183
interoperable without treating report structure as assay validation, lab
184184
authentication, biological evidence, or recalibration authority.
185185

186+
On 2026-08-23, pilot pre-registration validation began failing closed on
187+
unlocked records. Drafts remain editable, but only a locked PRR- record can
188+
validate as the pre-experiment contract. This enforces the existing
189+
pre-registration boundary; it does not validate assay procedures, biological
190+
claims, or partner readiness.
191+
186192
This file is the current milestone authority. The older
187193
[`50_LOOP_PLAN.md`](50_LOOP_PLAN.md) is a historical execution record, not a
188194
live status page. Select the next bottleneck from

‎docs/review/PRE_REGISTERED_PILOT_TEMPLATE.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -189,6 +189,10 @@ Required result context:
189189

190190
Do not publish unsafe operational details.
191191

192+
The machine validator treats `is_locked: true` as a validity requirement. An
193+
unlocked draft can be useful for internal editing, but it must not pass as a
194+
valid pre-registration for experiment start.
195+
192196
## Recalibration decision
193197

194198
After result intake, run the pre-registered recalibration gate.

‎src/openamp_foundry/evidence/pilot_preregistration.py‎

Lines changed: 9 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -158,15 +158,22 @@ def validate_pilot_preregistration(
158158
if not record.frozen_at.strip():
159159
violations.append("frozen_at must not be empty")
160160

161-
# Rule 13: threshold values in [0, 1]
161+
# Rule 13: lock required before a record can be valid
162+
if not record.is_locked:
163+
violations.append(
164+
"is_locked must be True — selection criteria and thresholds must be "
165+
"frozen before any experiment begins"
166+
)
167+
168+
# Rule 14: threshold values in [0, 1]
162169
for t in record.score_thresholds:
163170
if not (0.0 <= t.threshold_value <= 1.0):
164171
violations.append(
165172
f"score_thresholds[{t.score_name}].threshold_value must be in [0, 1], "
166173
f"got {t.threshold_value}"
167174
)
168175

169-
# Rule 14: amendment_reasons vocabulary
176+
# Rule 15: amendment_reasons vocabulary
170177
for reason in record.amendment_reasons:
171178
if reason not in VALID_AMENDMENT_REASONS:
172179
violations.append(

‎tests/evidence/test_pilot_preregistration.py‎

Lines changed: 14 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -37,7 +37,7 @@ def _make_record(**kwargs) -> PilotPreregistration:
3737
negative_control="pbs_vehicle",
3838
outcome_metric="minimum_inhibitory_concentration",
3939
dry_lab_only_declaration=True,
40-
is_locked=False,
40+
is_locked=True,
4141
)
4242
defaults.update(kwargs)
4343
return PilotPreregistration(**defaults)
@@ -82,7 +82,14 @@ def test_default_amendment_count(self):
8282
assert r.amendment_count == 0
8383

8484
def test_is_locked_default_false(self):
85-
r = _make_record()
85+
r = PilotPreregistration(
86+
record_id="PRR-DEFAULT-001",
87+
version="1.0.0",
88+
frozen_at="2026-07-10T00:00:00Z",
89+
pipeline_version="0.9.0",
90+
git_sha="abc1234",
91+
primary_hypothesis="draft",
92+
)
8693
assert r.is_locked is False
8794

8895

@@ -112,6 +119,11 @@ def test_valid_record_passes(self):
112119
assert result.is_valid is True
113120
assert result.violations == []
114121

122+
def test_unlocked_record_is_not_valid_for_experiment_start(self):
123+
result = validate_pilot_preregistration(_make_record(is_locked=False))
124+
assert result.is_valid is False
125+
assert any("is_locked" in violation for violation in result.violations)
126+
115127
def test_returns_validation_result(self):
116128
result = validate_pilot_preregistration(_make_record())
117129
assert isinstance(result, PreregistrationValidationResult)

0 commit comments

Comments
 (0)