You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit bfbe4db
Browse filesBrowse the repository at this point in the historyBrowse files
feat: add claim and replication schemas — the core thesis objects
The repo's thesis is 'CI/CD for truth claims': make false AI-generated
knowledge harder to merge. But the protocol had no persistent claim
object — only transient submissions and single-source evidence records.
This commit adds the missing core objects:
- schemas/claim.schema.json: a persistent, falsifiable statement with a
verification lifecycle (unverified -> dry-lab-verified -> replicated ->
accepted -> field-tested), evidence links, failure modes, a kill
condition, and required reviewer roles. High-safety claims cannot
advance without red-team review. Falsified claims stay in the record
so contributors do not repeat them.
- schemas/replication.schema.json: an independent replication record
with replicator identity, environment, input hash, result, and
divergence tracking. Self-replication is not replication.
Evidence schema strengthened:
- Added model-prediction, computational-analysis, and wet-lab-confirmation
evidence types. The thesis envisions computational claims (molecule
candidates, theorem sketches, benchmark results) but the enum had no
way to represent them.
- Added optional doi and archive_url fields to insulate records against
link rot.
Agent-submission schema strengthened:
- Added kill_condition as a required field. SKILL.md pattern #3 already
required it in prose; the schema now enforces it. Protocol drift fixed.
Validator updated to compile and validate all new schemas and examples.
Stale pack counts fixed: AGENTS.md had a 21-pack table; ROADMAP.md said
15. Actual count is 100. Replaced with pointers to generated indexes,
following the repo's own rule: 'Never trust an embedded count in any
hand-written file.'
AI-slop language cleaned: replaced vague metaphorical 'landscape' in the
skills-training pack with precise terms. Legitimate ecological uses of
'landscape' (landscape simplification, agricultural landscapes) in
biodiversity packs were correctly left intact.
All validation passes: pnpm validate, pnpm reproducibility:check,
pnpm build.
Copy file name to clipboardExpand all lines: AGENTS.md
+34-24Lines changed: 34 additions & 24 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -24,7 +24,7 @@ Correct, narrow, and falsifiable beats prolific.
24
24
## Key Components
25
25
26
26
-`problem-packs/`: canonical problem-pack directories with task maps across multiple domains. Read generated indexes for live counts. When calibrating pack quality, use strong exemplars such as `climate-health/dengue-heat-vietnam` for operational humility and `public-health/birth-registration-access-global` for measure-family discipline across survey, CRVS, and health-touchpoint evidence.
27
-
-`schemas/`: JSON schemas for machine-checkable protocol objects.
27
+
-`schemas/`: JSON schemas for machine-checkable protocol objects. Includes the claim schema (the core thesis object: persistent claims with a verification lifecycle, evidence links, failure modes, kill conditions, and required reviewers) and the replication schema (independent replication records with environment, input hash, and divergence tracking).
-`public-health/birth-registration-access-global` — measure-family discipline across survey, CRVS, and health-touchpoint evidence
60
47
61
48
## Agent Working Rules
62
49
@@ -105,6 +92,29 @@ A contribution may only be merged if at least one of these is true:
105
92
106
93
Pure prose polish without one of the above is not a reason to merge.
107
94
95
+
## The Claim Lifecycle
96
+
97
+
The thesis of this repository is that verification is the scarce resource. The claim object is the protocol object that makes that thesis machine-checkable. A claim is a persistent, falsifiable statement that exists independently of any single submission or review. It accumulates evidence and failure modes over time, carries its own kill condition, and moves through a verification lifecycle.
-`unverified`: A claim has been submitted but no evidence has been reviewed.
107
+
-`dry-lab-verified`: Evidence records support the claim at the computational or literature level. No wet-lab or field confirmation.
108
+
-`needs-replication`: Evidence has been reviewed but independent replication is required before the claim can advance.
109
+
-`replicated`: An independent replicator has confirmed the result. The replication record is linked.
110
+
-`accepted`: The claim has passed all required reviews and replication. It is canonical repo truth.
111
+
-`field-tested`: The claim has been tested in a real-world operational context. The highest status.
112
+
-`rejected`: The claim did not survive review. Terminal.
113
+
-`falsified`: The kill condition was met. Terminal. A falsified claim stays in the record so future contributors do not repeat it.
114
+
-`deprecated`: The claim was accepted but its evidence has decayed (source rot, dataset revision, superseded by a stronger claim). Terminal but recoverable through a new submission.
115
+
116
+
A claim with no evidence is `unverified` regardless of prose. A claim with no kill condition is not a claim. A high-safety claim without red-team review cannot advance past `dry-lab-verified`. The schema enforces these constraints; reviewers enforce the rest.
117
+
108
118
## Roles for Top AI Agents
109
119
110
120
Strong models should pick a role and stay in it for the duration of a PR. Mixing roles in one submission hides which judgment failed.
All schemas have `description` fields on every property.
83
85
86
+
The claim schema is the core protocol object: a persistent, falsifiable statement with a verification lifecycle (`unverified` to `field-tested`), evidence links, failure modes, a kill condition, and required reviewer roles. Replication records track independent attempts to reproduce a claim, with environment, input hash, and divergence tracking.
87
+
84
88
## Submission Quality Standard
85
89
86
90
**One task. One role. One claim.** Submissions that mix roles or make multiple independent claims are returned for splitting.
Copy file name to clipboardExpand all lines: ROADMAP.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -10,7 +10,7 @@ The V0 system uses GitHub itself as the operating surface. No custom web app. Ev
10
10
11
11
### What V0 delivers
12
12
13
-
-**15 problem packs** across 11 domains: air quality, biodiversity, climate adaptation, climate health, disaster resilience, education, energy access, food security, public health, sanitation, and water security.
13
+
-**100 problem packs** across 11 domains: air quality, biodiversity, climate adaptation, climate health, disaster resilience, education, energy access, food security, public health, sanitation, and water security.
14
14
-**Machine-checkable schemas** for problems, tasks, evidence records, agent submissions, and reviews.
Copy file name to clipboardExpand all lines: examples/agent-submission.example.json
+1Lines changed: 1 addition & 0 deletions
Original file line number
Diff line number
Diff line change
@@ -16,6 +16,7 @@
16
16
"A reader may treat climate association as a local causal threshold.",
17
17
"A contributor may skip district-level denominator checks."
18
18
],
19
+
"kill_condition": "A reviewer finds that the cited sources do not support framing-level claims about dengue-climate association in Viet Nam, or that a listed source has been retracted.",
19
20
"confidence": "medium",
20
21
"suggested_next_issue": "Inventory public district-level dengue surveillance data access."
"claim": "Interactions among temperature, hydrometeorology, infrastructure, and mobility are associated with dengue emergence in Viet Nam at district level, based on 23 years of surveillance data. This association is framable but does not prove a local causal threshold.",
"A reader treats the association as a local causal threshold for a specific district.",
10
+
"Passive surveillance data reflects reporting access, not only true incidence.",
11
+
"Infrastructure and mobility variables are proxy measures that may not capture the true confounders."
12
+
],
13
+
"kill_condition": "A replication on an independent surveillance dataset for Viet Nam finds no statistically significant association between temperature, rainfall, or infrastructure interactions and district-level dengue incidence after controlling for reporting access.",
"method": "Re-extracted district-level dengue surveillance data from the original supplementary tables, recomputed the temperature-rainfall-infrastructure interaction terms using the reported model specification, and compared coefficient signs and significance against the published results.",
"divergence": "Interaction terms for temperature and rainfall replicated within reported confidence intervals. The infrastructure-mobility interaction term was significant in the original but marginal (p=0.06) in the replication, likely due to a minor difference in district exclusion criteria when handling missing mobility data. The core association is confirmed; the infrastructure pathway needs a tighter specification.",
Copy file name to clipboardExpand all lines: problem-packs/education/skills-training-youth-employment-global/evidence.json
+2-2Lines changed: 2 additions & 2 deletions
Original file line number
Diff line number
Diff line change
@@ -50,9 +50,9 @@
50
50
},
51
51
"source_date": "2023-09-01",
52
52
"access_date": "2026-06-18",
53
-
"method": "Reviewed training-data landscape assessments, provider-registry documentation, and employment-outcome tracking rates across LMICs.",
53
+
"method": "Reviewed training-data assessments, provider-registry documentation, and employment-outcome tracking rates across LMICs.",
54
54
"limitations": [
55
-
"The originally cited 'Skills training data systems landscape assessment' could not be confirmed to exist; re-sourced to the World Bank/ILO/UNESCO TVET systems report.",
55
+
"The originally cited 'Skills training data systems assessment' could not be confirmed to exist; re-sourced to the World Bank/ILO/UNESCO TVET systems report.",
56
56
"This report describes TVET system fragmentation broadly; it does not quantify provider-registry coverage country by country.",
57
57
"A dedicated source is still needed to substantiate the 'no single dataset' claim for any specific country."
Copy file name to clipboardExpand all lines: problem-packs/education/skills-training-youth-employment-global/evidence.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -16,7 +16,7 @@ Use this source for the best available evidence on training-program employment e
16
16
17
17
### Training Provider Data Fragmentation
18
18
19
-
Use this source (World Bank/ILO/UNESCO TVET systems report, 2023) to document that TVET provision in most LMICs is diverse and fragmented across government ministries, private providers, and NGOs, complicating system-wide data on providers and outcomes. (Re-sourced June 2026: the prior 'landscape assessment' citation could not be confirmed; the 'no single dataset' and 'fewer than 10 percent track outcomes' specifics still need a dedicated source.)
19
+
Use this source (World Bank/ILO/UNESCO TVET systems report, 2023) to document that TVET provision in most LMICs is diverse and fragmented across government ministries, private providers, and NGOs, complicating system-wide data on providers and outcomes. (Re-sourced June 2026: the prior 'training-system assessment' citation could not be confirmed; the 'no single dataset' and 'fewer than 10 percent track outcomes' specifics still need a dedicated source.)
0 commit comments