Skip to content

Commit 65dfc4d

Browse files
authored
fix: surface lab result data origin (#27)
Merged after self-review and focused verification. Full-suite limitations remain documented in the PR.
1 parent bd3d252 commit 65dfc4d

5 files changed

Lines changed: 76 additions & 9 deletions

File tree

‎docs/evidence/METRICS_CURRENT.md‎

Lines changed: 11 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -5,9 +5,9 @@ Machine-readable snapshot: `outputs/metrics_snapshot.json` regenerated with `mak
55
> **Purpose:** One authoritative table of current pipeline metrics. If any doc disagrees
66
> with this file, this file wins. Updated whenever benchmark/benchmark config changes.
77
>
8-
> **Last updated:** 2026-08-18 (canonical V4 packet validator/schema parity; benchmark values unchanged)
8+
> **Last updated:** 2026-08-21 (standalone lab-result data-origin visibility; benchmark values unchanged)
99
10-
> **Current verification note (2026-08-18):** Phase AA AA6 exposes the AARG-
10+
> **Current verification note (2026-08-21):** Phase AA AA6 exposes the AARG-
1111
> reproducibility aggregate through a repeatable CLI/make workflow, while Phase
1212
> AB AB5 exposes the ABAG- claim-integrity aggregate, Phase AC AC3 exposes the
1313
> ACDG- aggregate, Phase Y Y5 exposes the YAG- aggregate, and Phase Z Z5
@@ -100,7 +100,7 @@ Machine-readable snapshot: `outputs/metrics_snapshot.json` regenerated with `mak
100100
> Phase AC AC3 exposes the ACDG-
101101
> aggregate disconfirming-evidence gate as a repeatable CLI/make workflow. It
102102
> has 18 focused gate tests plus 2 CLI integration tests. Full pytest
103-
> collection succeeds at 12,416 tests; the live count is enforced by the
103+
> collection succeeds at 12,417 tests; the live count is enforced by the
104104
> current-state alignment test. This artifact does not establish
105105
> biological validation or benchmark improvement.
106106
@@ -232,6 +232,14 @@ Machine-readable snapshot: `outputs/metrics_snapshot.json` regenerated with `mak
232232
- This is provenance visibility only; legacy results remain accepted and no
233233
recalibration or biological claim policy changed.
234234

235+
### External-result reporting integrity — expose data origin in every report
236+
- Standalone `lab-result-report` JSON and Markdown outputs now include the
237+
shared `data_origin` summary used by calibration intake.
238+
- Synthetic labels are visible by result ID, and unclassified records remain
239+
explicitly unclassified rather than being inferred to be real wet-lab evidence.
240+
- This is audit visibility only; it does not validate assay contents, authenticate
241+
the performing lab, establish biological evidence, or authorize recalibration.
242+
235243
### External-result reporting integrity — opt-in raw-file hash verification
236244
- Added `verify_raw_data_provenance()` and an optional `raw_data_file` field to
237245
bind a declared SHA-256 to independently hashed bytes under a supplied

‎docs/research/ROADMAP.md‎

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Roadmap
22

3-
## Current state — 2026-08-18
3+
## Current state — 2026-08-21
44

55
Phase AB is complete as of 2026-07-26. AB5 exposes the existing CSD-, RDR-,
66
EGN-, and EHP- claim-integrity artifacts through an ABAG- aggregate, CLI
@@ -169,6 +169,14 @@ CLI-generated packets. This is packaging integrity only; it does not
169169
authenticate artifacts or reviewers, validate science, or establish biological
170170
evidence.
171171

172+
On 2026-08-21, standalone `lab-result-report` JSON and Markdown outputs began
173+
carrying the shared data-origin summary used by calibration intake. Synthetic
174+
labels are therefore visible in both result-reporting paths, while unclassified
175+
records remain explicitly unclassified rather than being presented as verified
176+
wet-lab evidence. This closes an intake-audit visibility gap; it does not
177+
validate assay contents, authenticate a lab, establish biology, or permit
178+
recalibration.
179+
172180
This file is the current milestone authority. The older
173181
[`50_LOOP_PLAN.md`](50_LOOP_PLAN.md) is a historical execution record, not a
174182
live status page. Select the next bottleneck from

‎src/openamp_foundry/reports/AGENTS.md‎

Lines changed: 6 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -7,11 +7,12 @@ review artifacts without strengthening scientific claims.
77

88
## Key Components
99

10-
- `lab_result_report.py`: result counts, candidate rollups, controls, and input
11-
validation blockers. Markdown and JSON distinguish control-passing outcome
12-
counts from raw audit observations, including failed-control qualitative
13-
results. They also show declared raw-data hash coverage without calling it
14-
independently verified.
10+
- `lab_result_report.py`: result counts, candidate rollups, controls, data-origin
11+
status, and input validation blockers. Markdown and JSON distinguish
12+
control-passing outcome counts from raw audit observations, including
13+
failed-control qualitative results. They also show synthetic/unclassified
14+
origin and declared raw-data hash coverage without calling either biological
15+
provenance or a declared hash independently verified.
1516
- `recalibration_report.py`: proposal/gate summaries; proposals are not applied.
1617

1718
## Diagrams (Mermaid)

‎src/openamp_foundry/reports/lab_result_report.py‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,7 @@
1616
candidate_result_map,
1717
duplicate_result_ids,
1818
load_lab_results_dir_with_errors,
19+
summarise_data_origin,
1920
summarise_candidate_outcomes,
2021
summarise_lab_results,
2122
verify_raw_data_provenance,
@@ -28,6 +29,7 @@ def build_lab_result_report(
2829
"""Build a machine-readable wet-lab result report from a directory of JSON files."""
2930
results, invalid_lab_result_files = load_lab_results_dir_with_errors(results_dir)
3031
summary = summarise_lab_results(results)
32+
data_origin = summarise_data_origin(results)
3133
by_candidate = summarise_candidate_outcomes(results)
3234
duplicate_ids = duplicate_result_ids(results)
3335
raw_data_provenance = verify_raw_data_provenance(results, raw_data_dir)
@@ -49,6 +51,7 @@ def build_lab_result_report(
4951

5052
return {
5153
"summary": summary,
54+
"data_origin": data_origin,
5255
"raw_data_provenance": raw_data_provenance,
5356
"raw_data_verification_issues": raw_data_provenance["verification_issues"],
5457
"n_invalid_lab_result_files": len(invalid_lab_result_files),
@@ -97,6 +100,8 @@ def write_lab_result_markdown(report: dict[str, Any], out_path: str | Path) -> N
97100
f"- Results with both controls passing: {s.get('n_valid_controls', 0)}",
98101
f"- Invalid result files: {report.get('n_invalid_lab_result_files', 0)}",
99102
f"- Duplicate result IDs: {report.get('n_duplicate_lab_result_ids', 0)}",
103+
f"- Data-origin status: {report.get('data_origin', {}).get('status', 'unclassified')}",
104+
f"- Synthetic results: {report.get('data_origin', {}).get('n_synthetic_results', 0)}",
100105
f"- Raw-data hash coverage: {raw_data_status}",
101106
f"- Raw-data hash verification: {report.get('raw_data_provenance', {}).get('verification_status', 'not_requested')}",
102107
"",

‎tests/integration/test_cli.py‎

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -997,14 +997,59 @@ def test_lab_result_report_creates_outputs(tmp_path, capsys):
997997
assert "toxic" not in report["summary"]["by_usable_qualitative_result"]
998998
assert report["n_candidates"] == 1
999999
assert len(report["control_failures"]) == 1
1000+
assert report["data_origin"]["status"] == "unclassified"
1001+
assert report["data_origin"]["n_synthetic_results"] == 0
10001002
text = out_md.read_text(encoding="utf-8")
10011003
assert "Wet-Lab Result Report" in text
10021004
assert "Usable Qualitative Outcome Counts" in text
10031005
assert "Raw Qualitative Observations (Audit Only)" in text
1006+
assert "Data-origin status: unclassified" in text
10041007
assert "CAND-001" in text
10051008
assert "RES-002" in text
10061009

10071010

1011+
def test_lab_result_report_surfaces_synthetic_origin(tmp_path):
1012+
results_dir = tmp_path / "lab_results"
1013+
results_dir.mkdir()
1014+
result = {
1015+
"result_id": "RES-SYNTHETIC-001",
1016+
"candidate_id": "SYNTHETIC-CAND-001",
1017+
"assay_type": "MIC",
1018+
"organism_or_cell_line": "SYNTHETIC TEST - E. coli",
1019+
"result_value": 8.0,
1020+
"result_unit": "µg/mL",
1021+
"result_qualitative": "active",
1022+
"positive_control_passed": True,
1023+
"negative_control_passed": True,
1024+
"assay_date": "2026-07-01",
1025+
"replicate_count": 1,
1026+
"performed_by_lab": "SYNTHETIC TEST - fixture",
1027+
"raw_data_sha256": None,
1028+
"computational_candidate_certificate_hash": "abc123def456",
1029+
"notes": "SYNTHETIC TEST DATA - not a real assay.",
1030+
"disclaimer": (
1031+
"SYNTHETIC TEST. This is not a real experimental result on a "
1032+
"computationally nominated candidate and does not constitute a "
1033+
"drug or clinical claim."
1034+
),
1035+
}
1036+
(results_dir / "synthetic.json").write_text(json.dumps(result), encoding="utf-8")
1037+
1038+
from openamp_foundry.reports.lab_result_report import (
1039+
build_lab_result_report,
1040+
write_lab_result_markdown,
1041+
)
1042+
1043+
report = build_lab_result_report(results_dir)
1044+
assert report["data_origin"]["status"] == "synthetic_present"
1045+
assert report["data_origin"]["n_synthetic_results"] == 1
1046+
assert report["data_origin"]["synthetic_result_ids"] == ["RES-SYNTHETIC-001"]
1047+
1048+
out_md = tmp_path / "report.md"
1049+
write_lab_result_markdown(report, out_md)
1050+
assert "Data-origin status: synthetic_present" in out_md.read_text(encoding="utf-8")
1051+
1052+
10081053
def test_lab_result_report_blocks_invalid_files(tmp_path, capsys):
10091054
results_dir = tmp_path / "lab_results"
10101055
results_dir.mkdir()

0 commit comments

Comments
 (0)