Skip to content

Commit cb12f03

Browse files
authored
fix: harden pilot preregistration CLI input boundary
Reject malformed PRR CLI field types with structured input errors while preserving valid and semantic-invalid exit behavior.
1 parent b1ad98a commit cb12f03

5 files changed

Lines changed: 118 additions & 4 deletions

File tree

‎docs/evidence/METRICS_CURRENT.md‎

Lines changed: 9 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -5,9 +5,9 @@ Machine-readable snapshot: `outputs/metrics_snapshot.json` regenerated with `mak
55
> **Purpose:** One authoritative table of current pipeline metrics. If any doc disagrees
66
> with this file, this file wins. Updated whenever benchmark/benchmark config changes.
77
>
8-
> **Last updated:** 2026-09-02 (pilot pre-registration CLI check; benchmark values unchanged)
8+
> **Last updated:** 2026-09-05 (pilot pre-registration input boundary; benchmark values unchanged)
99
10-
> **Current verification note (2026-09-02):** Phase AA AA6 exposes the AARG-
10+
> **Current verification note (2026-09-05):** Phase AA AA6 exposes the AARG-
1111
> reproducibility aggregate through a repeatable CLI/make workflow, while Phase
1212
> AB AB5 exposes the ABAG- claim-integrity aggregate, Phase AC AC3 exposes the
1313
> ACDG- aggregate, Phase Y Y5 exposes the YAG- aggregate, and Phase Z Z5
@@ -100,7 +100,7 @@ Machine-readable snapshot: `outputs/metrics_snapshot.json` regenerated with `mak
100100
> Phase AC AC3 exposes the ACDG-
101101
> aggregate disconfirming-evidence gate as a repeatable CLI/make workflow. It
102102
> has 18 focused gate tests plus 2 CLI integration tests. Full pytest
103-
> collection succeeds at 12,435 tests; the live count is enforced by the
103+
> collection succeeds at 12,438 tests; the live count is enforced by the
104104
> current-state alignment test. This artifact does not establish
105105
> biological validation or benchmark improvement.
106106
@@ -116,6 +116,12 @@ Machine-readable snapshot: `outputs/metrics_snapshot.json` regenerated with `mak
116116
> contract executable in the review loop; it does not authenticate signers,
117117
> validate assays, or establish biological evidence.
118118
119+
> **Pilot pre-registration input-boundary note (2026-09-05):** The PRR CLI now
120+
> rejects malformed scalar, list, boolean, integer, and threshold field types
121+
> as structured input errors. This prevents permissive coercion and unhandled
122+
> exceptions; it does not authenticate signers or establish assay or biological
123+
> evidence.
124+
119125
> **Pilot freeze-integrity note (2026-08-24):** Locked PRR- records now require
120126
> a deterministic SHA-256 over their canonical content. This catches edits to
121127
> frozen selection criteria and thresholds; it does not authenticate the

‎docs/getting-started/COMMAND_SURFACE.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -140,6 +140,8 @@ qualified pilot. A valid record must be locked and carry a matching
140140
`freeze_sha256`; an editable draft is expected to fail this check. The command
141141
checks record integrity only. It does not authenticate a signer, validate an
142142
assay, or establish biological evidence.
143+
Malformed field types are reported as input errors rather than being coerced
144+
or allowed to crash the command.
143145

144146
## Recalibration workflow
145147

‎docs/research/ROADMAP.md‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Roadmap
22

3-
## Current state — 2026-09-02
3+
## Current state — 2026-09-05
44

55
Phase AB is complete as of 2026-07-26. AB5 exposes the existing CSD-, RDR-,
66
EGN-, and EHP- claim-integrity artifacts through an ABAG- aggregate, CLI
@@ -204,6 +204,10 @@ On 2026-09-02, the PRR contract became reachable through the
204204
checks in the normal review loop, while remaining an integrity check rather
205205
than signer authentication, assay validation, or biological evidence.
206206

207+
On 2026-09-05, the PRR CLI began rejecting malformed field types as structured
208+
input errors. This prevents permissive coercion or handler crashes at the JSON
209+
boundary while leaving valid locked and editable records unchanged.
210+
207211
This file is the current milestone authority. The older
208212
[`50_LOOP_PLAN.md`](50_LOOP_PLAN.md) is a historical execution record, not a
209213
live status page. Select the next bottleneck from

‎src/openamp_foundry/cli/commands/reports.py‎

Lines changed: 66 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2043,6 +2043,72 @@ def _run_pilot_preregistration_check(args: argparse.Namespace) -> int:
20432043
print(json.dumps({"status": "error", "error": "--entry-json must be a JSON object"}))
20442044
return 2
20452045

2046+
string_fields = (
2047+
"record_id", "version", "frozen_at", "pipeline_version", "git_sha",
2048+
"primary_hypothesis", "positive_control", "negative_control",
2049+
"outcome_metric", "notes", "freeze_sha256",
2050+
)
2051+
for field_name in string_fields:
2052+
if field_name in entry_dict and not isinstance(entry_dict[field_name], str):
2053+
print(json.dumps({
2054+
"status": "error",
2055+
"error": f"Malformed PRR JSON: {field_name} must be a string",
2056+
}))
2057+
return 2
2058+
list_fields = ("selection_criteria", "amendment_reasons", "score_thresholds")
2059+
for field_name in list_fields:
2060+
if field_name in entry_dict and not isinstance(entry_dict[field_name], list):
2061+
print(json.dumps({
2062+
"status": "error",
2063+
"error": f"Malformed PRR JSON: {field_name} must be a list",
2064+
}))
2065+
return 2
2066+
for field_name in ("dry_lab_only_declaration", "is_locked"):
2067+
if field_name in entry_dict and not isinstance(entry_dict[field_name], bool):
2068+
print(json.dumps({
2069+
"status": "error",
2070+
"error": f"Malformed PRR JSON: {field_name} must be a boolean",
2071+
}))
2072+
return 2
2073+
for field_name in ("n_candidates_planned", "amendment_count"):
2074+
if field_name in entry_dict and (
2075+
not isinstance(entry_dict[field_name], int)
2076+
or isinstance(entry_dict[field_name], bool)
2077+
):
2078+
print(json.dumps({
2079+
"status": "error",
2080+
"error": f"Malformed PRR JSON: {field_name} must be an integer",
2081+
}))
2082+
return 2
2083+
for threshold in entry_dict.get("score_thresholds", []):
2084+
if not isinstance(threshold, dict):
2085+
print(json.dumps({
2086+
"status": "error",
2087+
"error": "Malformed PRR JSON: score_thresholds entries must be objects",
2088+
}))
2089+
return 2
2090+
if not isinstance(threshold.get("score_name"), str):
2091+
print(json.dumps({
2092+
"status": "error",
2093+
"error": "Malformed PRR JSON: score_thresholds.score_name must be a string",
2094+
}))
2095+
return 2
2096+
if (
2097+
not isinstance(threshold.get("threshold_value"), (int, float))
2098+
or isinstance(threshold.get("threshold_value"), bool)
2099+
):
2100+
print(json.dumps({
2101+
"status": "error",
2102+
"error": "Malformed PRR JSON: score_thresholds.threshold_value must be numeric",
2103+
}))
2104+
return 2
2105+
if not isinstance(threshold.get("direction"), str):
2106+
print(json.dumps({
2107+
"status": "error",
2108+
"error": "Malformed PRR JSON: score_thresholds.direction must be a string",
2109+
}))
2110+
return 2
2111+
20462112
try:
20472113
thresholds = [
20482114
ScoreThreshold(**threshold)

‎tests/integration/test_pilot_preregistration_cli.py‎

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -72,3 +72,39 @@ def test_invalid_preregistration_json_returns_input_error(capsys):
7272
assert rc == 2
7373
result = json.loads(capsys.readouterr().out)
7474
assert result["status"] == "error"
75+
76+
77+
def test_invalid_preregistration_field_type_returns_input_error(capsys):
78+
payload = _locked_payload()
79+
payload["selection_criteria"] = "not-a-list"
80+
rc = main([
81+
"pilot-preregistration-check",
82+
"--entry-json", json.dumps(payload),
83+
])
84+
assert rc == 2
85+
result = json.loads(capsys.readouterr().out)
86+
assert "selection_criteria must be a list" in result["error"]
87+
88+
89+
def test_invalid_preregistration_threshold_type_returns_input_error(capsys):
90+
payload = _locked_payload()
91+
payload["score_thresholds"][0]["threshold_value"] = "0.75"
92+
rc = main([
93+
"pilot-preregistration-check",
94+
"--entry-json", json.dumps(payload),
95+
])
96+
assert rc == 2
97+
result = json.loads(capsys.readouterr().out)
98+
assert "threshold_value must be numeric" in result["error"]
99+
100+
101+
def test_invalid_preregistration_scalar_type_returns_input_error(capsys):
102+
payload = _locked_payload()
103+
payload["primary_hypothesis"] = 42
104+
rc = main([
105+
"pilot-preregistration-check",
106+
"--entry-json", json.dumps(payload),
107+
])
108+
assert rc == 2
109+
result = json.loads(capsys.readouterr().out)
110+
assert "primary_hypothesis must be a string" in result["error"]

0 commit comments

Comments
 (0)