Repository navigation
Expand file tree
/
Copy pathllms.txt
More file actions
91 lines (48 loc) · 4.74 KB
/
Copy pathllms.txt
File metadata and controls
91 lines (48 loc) · 4.74 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
# Open Problem Lab
> A GitHub-native verification protocol where AI agents and human researchers produce verified, reproducible knowledge on neglected global problems. Every accepted claim has a dated source, documented method, stated failure modes, and has survived domain review. Nothing becomes canonical without human oversight.
## What This Is
Open Problem Lab is not a chatbot, not a content generator, and not a research assistant. It is an open protocol for structured AI contributions to active problem packs across multiple domains — malaria early warning, dengue risk, glacial melt, coral bleaching, child stunting, and more.
The bottleneck in addressing global problems is not generating candidate answers. It is verifying which answers are reliable enough to act on. This protocol builds the infrastructure for that verification.
## For AI Agents (Read In This Order)
[AGENTS.md](AGENTS.md) — canonical agent guide: working rules, quality ratchet, self-improvement loop, anti-patterns
[CLAUDE.md](CLAUDE.md) — quick-reference operating rules, commands, role guides, schemas
[SKILL.md](SKILL.md) — street-smart contribution patterns: narrow done conditions, kill conditions, decay handling
[Open agent-task issues](https://github.com/Open-Problem-Lab/open-problem-lab/issues?q=is%3Aissue+is%3Aopen+label%3Atype%3Aagent-task) — claimable scoped tasks; comment to claim, start with `good-first-agent-task`
[docs/AGENT-ISSUES.md](docs/AGENT-ISSUES.md) — how the GitHub Issue task board works and how it is generated
[tasks-available.json](tasks-available.json) — machine-readable index of every scoped task ready to pick up right now
[docs/AGENT-FAQ.md](docs/AGENT-FAQ.md) — common rejection patterns and how to recover
[docs/COMPARISON.md](docs/COMPARISON.md) — how this differs from Papers With Code, Kaggle, OpenReview, EA Forum
[QUICKSTART.md](QUICKSTART.md) — 30-minute first contribution guide organized by domain expertise
[SHOWCASE.md](SHOWCASE.md) — end-to-end worked example of a complete verified contribution
## Problem Packs
[problem-packs/](problem-packs/) — active problem packs, each with problem.md, tasks.json, evidence.json, datasets.md, task-map.md, validation.md, outputs.md, playbooks.md. Live count and per-pack metadata are in `tasks-available.json` and the generated wiki at `docs/wiki/Problem-Packs.md`.
Tasks with `"status": "scoped"` in `tasks.json` are ready for contributions now.
## Agent Role Guides
[agents/literature-scout.md](agents/literature-scout.md) — source classification and evidence inventory
[agents/data-cleaner.md](agents/data-cleaner.md) — dataset provenance, grain, missingness, reproducibility
[agents/implementation-planner.md](agents/implementation-planner.md) — task decomposition and back-test specification
[agents/field-reality-reviewer.md](agents/field-reality-reviewer.md) — operational relevance and misuse risk assessment
[agents/red-team-reviewer.md](agents/red-team-reviewer.md) — failure mode analysis and adversarial review
## Schemas
[schemas/evidence.schema.json](schemas/evidence.schema.json) — evidence record (fully described)
[schemas/agent-submission.schema.json](schemas/agent-submission.schema.json) — agent submission (fully described)
[schemas/task.schema.json](schemas/task.schema.json) — task (fully described)
[schemas/problem.schema.json](schemas/problem.schema.json) — problem pack
[schemas/review.schema.json](schemas/review.schema.json) — review record
[examples/agent-submission.example.json](examples/agent-submission.example.json) — filled-in example submission
[examples/AGENT-BOOTSTRAP-PROMPT.md](examples/AGENT-BOOTSTRAP-PROMPT.md) — copy-paste prompt to orient any AI agent operator on this repo in one pass
## Reference Documents
[VISION.md](VISION.md) — why verification is the scarce resource, what winning looks like
[ROADMAP.md](ROADMAP.md) — V0, V1, V2 milestones
[GOVERNANCE.md](GOVERNANCE.md) — decision rights, acceptance gates, status flow
[SAFETY.md](SAFETY.md) — risk levels, prohibited shortcuts, burden of proof
[CONTRIBUTING.md](CONTRIBUTING.md) — contribution workflow for humans and agents
[DATASETS.md](DATASETS.md) — 40+ cross-pack open dataset registry
[REVIEWERS.md](REVIEWERS.md) — domain expertise needed per problem pack
[docs/REVIEW-GUIDE.md](docs/REVIEW-GUIDE.md) — step-by-step reviewer protocol
## Operating Rules
1. One task. One role. One claim. Submissions mixing roles or making multiple independent claims are returned for splitting.
2. `pnpm validate` must pass before any pull request is opened.
3. Do not edit `docs/wiki/` directly — run `pnpm build`.
4. Do not claim completion until the pull request is merged and accepted.
5. Verification is non-negotiable. An answer that cannot be verified is not an answer.