Repository navigation
feat: add claim and replication schemas — the core thesis objects - #47
Merged
Merged
Conversation
The repo's thesis is 'CI/CD for truth claims': make false AI-generated knowledge harder to merge. But the protocol had no persistent claim object — only transient submissions and single-source evidence records. This commit adds the missing core objects: - schemas/claim.schema.json: a persistent, falsifiable statement with a verification lifecycle (unverified -> dry-lab-verified -> replicated -> accepted -> field-tested), evidence links, failure modes, a kill condition, and required reviewer roles. High-safety claims cannot advance without red-team review. Falsified claims stay in the record so contributors do not repeat them. - schemas/replication.schema.json: an independent replication record with replicator identity, environment, input hash, result, and divergence tracking. Self-replication is not replication. Evidence schema strengthened: - Added model-prediction, computational-analysis, and wet-lab-confirmation evidence types. The thesis envisions computational claims (molecule candidates, theorem sketches, benchmark results) but the enum had no way to represent them. - Added optional doi and archive_url fields to insulate records against link rot. Agent-submission schema strengthened: - Added kill_condition as a required field. SKILL.md pattern #3 already required it in prose; the schema now enforces it. Protocol drift fixed. Validator updated to compile and validate all new schemas and examples. Stale pack counts fixed: AGENTS.md had a 21-pack table; ROADMAP.md said 15. Actual count is 100. Replaced with pointers to generated indexes, following the repo's own rule: 'Never trust an embedded count in any hand-written file.' AI-slop language cleaned: replaced vague metaphorical 'landscape' in the skills-training pack with precise terms. Legitimate ecological uses of 'landscape' (landscape simplification, agricultural landscapes) in biodiversity packs were correctly left intact. All validation passes: pnpm validate, pnpm reproducibility:check, pnpm build.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The repo's thesis is "CI/CD for truth claims": make false AI-generated knowledge harder to merge. But the protocol had no persistent claim object — only transient submissions and single-source evidence records. This PR adds the missing core objects.
Changes
New schemas
schemas/claim.schema.json— the core protocol object. A persistent, falsifiable statement with a verification lifecycle (unverified→dry-lab-verified→needs-replication→replicated→accepted→field-tested), evidence links, failure modes, a kill condition, and required reviewer roles. High-safety claims cannot advance without red-team review. Falsified and deprecated are terminal states; falsified claims stay in the record so contributors do not repeat them.schemas/replication.schema.json— independent replication records with replicator identity (must differ from original submitter), environment (runtime, OS, packages), input hash, result enum (confirmed/partially-confirmed/failed/inconclusive), and divergence tracking. Self-replication is not replication.Strengthened schemas
schemas/evidence.schema.json— addedmodel-prediction,computational-analysis, andwet-lab-confirmationevidence types. The thesis envisions computational claims (molecule candidates, theorem sketches, benchmark results) but the enum had no way to represent them. Added optionaldoiandarchive_urlfields to insulate records against link rot.schemas/agent-submission.schema.json— addedkill_conditionas a required field. SKILL.md pattern Add agent merge bar note #3 already required it in prose; the schema now enforces it. Protocol drift fixed.New examples
examples/claim.example.json— worked example using the dengue-heat-vietnam packexamples/replication.example.json— worked example with a partially-confirmed replication resultexamples/agent-submission.example.json— updated to includekill_conditionValidator
scripts/validate-repo.mjs— compiles and validates all new schemas and example filesDocs
AGENTS.md— replaced stale 21-pack table with pointers to generated indexes (actual count: 100). Added claim lifecycle section.ROADMAP.md— fixed "15 problem packs" to "100 problem packs"CLAUDE.md— updated schemas table and evidence type listschemas/AGENTS.md— updated component and flow diagramsexamples/AGENTS.md— added new example files to key componentsagents/literature-scout.md— updated evidence types table with new typesREADME.md— updated repository map descriptionAI-slop cleanup
Protocol Notes
kill_conditionfield inagent-submission.schema.jsonwas required bySKILL.mdpattern Add agent merge bar note #3 but absent from the schema. This is exactly the kind of prose-rule-vs-schema drift the protocol should catch. A future validator rule could scan agent guides for required fields and check them against the corresponding schema.AGENTS.md(21) andROADMAP.md(15) vs actual (100) suggests the validator could check that hand-written files do not embed pack counts, or that they match the generated index. Currently the rule exists only in prose ("Never trust an embedded count").Verification
pnpm validate— passespnpm reproducibility:check— passes (516 tasks)pnpm build— passes (100 packs)pnpm format:check— passes