Skip to content

docs: make counter-evidence first-class - #63

Merged
cschanhniem merged 1 commit into
mainfrom
counter-evidence-ledger
Jul 6, 2026
Merged

cschanhniem merged 1 commit into
mainfrom
counter-evidence-ledger

Conversation

@cschanhniem

Copy link
Copy Markdown
Contributor

Summary

Adds a counter-evidence ledger standard for Open Problem Lab.

This is deliberately not a new validator yet. It is a governance/documentation PR that defines how failed replications, null results, external-validity limits, dataset flaws, field contradictions, and unresolved reviewer objections should be preserved instead of disappearing into closed issues.

Why this matters

The highest-risk failure mode for a verification protocol is not lack of impressive accepted claims. It is selection bias: accepted evidence accumulates while disconfirming evidence becomes hard to find.

The new document introduces:

  • what belongs in a counter-evidence ledger;
  • a lightweight entry format;
  • status semantics for unresolved/bounded/resolved/withdrawn claims;
  • reviewer and agent rules;
  • anti-patterns to avoid debate-dump behavior;
  • an acceptance test centered on the strongest known falsifier.

Safety / scope

Docs-only. No schema, workflow, or canonical problem-pack content is changed.

Review focus

Please review whether this should become a required artifact for mature problem packs, or stay advisory until a pack has enough accepted claims to justify it.

@cschanhniem
cschanhniem merged commit 9465c34 into main Jul 6, 2026
1 check failed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant