Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -159,6 +159,7 @@ Validation must pass before review. If validation fails, fix the evidence or str
| [SHOWCASE.md](SHOWCASE.md) | End-to-end worked example — what a complete contribution looks like |
| [tasks-available.json](tasks-available.json) | Machine-readable index of every scoped task ready to pick up |
| [docs/AGENT-FAQ.md](docs/AGENT-FAQ.md) | Common rejection patterns and how to recover |
| [docs/AGENT-ISSUES.md](docs/AGENT-ISSUES.md) | How the GitHub Issue task board works — claim, submit, regenerate |
| [docs/COMPARISON.md](docs/COMPARISON.md) | Positioning vs Papers With Code, Kaggle, OpenReview, EA Forum |
| [REVIEWERS.md](REVIEWERS.md) | Domain expert call to action — expertise needed per problem pack |
| [DATASETS.md](DATASETS.md) | Cross-pack open dataset registry |
Expand Down
54 changes: 54 additions & 0 deletions docs/AGENT-ISSUES.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
# Agent Task Issues

This repo publishes scoped work as **GitHub Issues** so that AI agents (and humans) can discover, claim, and complete tasks without reading the whole repository first. Each issue is one scoped step inside a problem pack.

## How the board works

| Label | Meaning |
| -------------------------- | --------------------------------------------- |
| `type:agent-task` | A scoped task ready to pick up |
| `good-first-agent-task` | Low-risk entry point for a new contributor |
| `role:<role>` | Which agent role owns the task |
| `risk:low` / `risk:medium` | Safety risk tier |
| `status:open-claim` | Unclaimed and available |
| `status:needs-review` | Submitted, awaiting domain or red-team review |
| `type:evidence` | A structured evidence submission |

Browse open work:

- All agent tasks: `is:issue is:open label:type:agent-task`
- First-timers: `is:issue is:open label:good-first-agent-task`
- By role: `label:role:literature-scout`, etc.

## Claiming and completing

1. Comment on the issue to claim it. One agent per task at a time — **Issues are work claims.**
2. Read the linked `problem.md`, role guide, and `tasks.json`.
3. Submit via the **Agent Submission** issue template, or open a pull request if you change canonical files.
4. The `done_condition` in the pack's `tasks.json` is the authority on completion — not your own assessment. A domain reviewer must sign off.

## Regenerating the board

Issue bodies are generated from `tasks-available.json` (itself generated from each pack's `tasks.json` by `pnpm build`). They never invent work — they reformat indexed tasks for browsing.

```bash
# List a diverse set of N packs to open issues for
pnpm issues:plan 24

# Print the issue title, labels, and body for one pack
pnpm issues:body air-quality/pm25-monitoring-south-asia
```

To create the issues with the GitHub CLI:

```bash
node scripts/generate-agent-issues.mjs --plan 24 | while read pk; do
node scripts/generate-agent-issues.mjs --body "$pk" > /tmp/iss.txt
title=$(grep '^TITLE' /tmp/iss.txt | cut -f2-)
labels=$(grep '^LABELS' /tmp/iss.txt | cut -f2-)
sed -n '/^BODY$/,$p' /tmp/iss.txt | tail -n +2 > /tmp/body.md
gh issue create --title "$title" --label "$labels" --body-file /tmp/body.md
done
```

Source of truth always remains the pack's `tasks.json`. If an issue and a `tasks.json` disagree, the `tasks.json` wins.
2 changes: 2 additions & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@
"reproducibility:check": "node scripts/check-reproducibility.mjs",
"validate": "node scripts/validate-repo.mjs",
"verify:sources": "node scripts/verify-sources.mjs",
"issues:plan": "node scripts/generate-agent-issues.mjs --plan",
"issues:body": "node scripts/generate-agent-issues.mjs --body",
"format": "prettier --write .",
"format:check": "prettier --check ."
},
Expand Down
139 changes: 139 additions & 0 deletions scripts/generate-agent-issues.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,139 @@
#!/usr/bin/env node
// Generate GitHub issue bodies for scoped agent tasks from tasks-available.json.
//
// Usage:
// node scripts/generate-agent-issues.mjs --list # print every task_id + pack_id
// node scripts/generate-agent-issues.mjs --body <pack_id> # print one issue body (markdown)
// node scripts/generate-agent-issues.mjs --plan [n] # print a diverse set of n picks (default 24)
//
// The issue body is deliberately self-contained: an AI agent that lands on the
// issue can understand the task, the done condition, and the exact next move
// without reading the whole repo first. Source of truth stays in tasks.json —
// this script never invents work, it only reformats indexed tasks for humans
// and agents browsing GitHub Issues.

import { readFileSync } from "node:fs";
import { fileURLToPath } from "node:url";
import { dirname, join } from "node:path";

const __dirname = dirname(fileURLToPath(import.meta.url));
const ROOT = join(__dirname, "..");

function loadTasks() {
const raw = readFileSync(join(ROOT, "tasks-available.json"), "utf8");
return JSON.parse(raw).tasks;
}

const ROLE_GUIDE = {
"literature-scout": "agents/literature-scout.md",
"data-cleaner": "agents/data-cleaner.md",
"implementation-planner": "agents/implementation-planner.md",
"field-reality-reviewer": "agents/field-reality-reviewer.md",
"red-team-reviewer": "agents/red-team-reviewer.md"
};

const REPO = "https://github.com/Open-Problem-Lab/open-problem-lab/blob/main";

function issueTitle(task) {
return `[AGENT-TASK] ${task.pack_title}: ${task.title}`;
}

function issueLabels(task) {
const labels = [
"type:agent-task",
"status:open-claim",
`role:${task.owner_role}`,
`risk:${task.safety_risk}`
];
if (task.safety_risk === "low") labels.push("good-first-agent-task");
return labels;
}

function issueBody(task) {
const roleGuide = ROLE_GUIDE[task.owner_role] || "AGENTS.md";
const domains = task.domain.join(", ");
const regions = (task.region || []).join(", ") || "global";
return `## What this task asks for

${task.outcome}

**Expected artifact:** ${task.expected_artifact}

## Done condition (the authority on completion)

> ${task.done_condition}

Your own assessment does not close this task — the done condition does, and a domain reviewer must sign off.

## Task at a glance

| Field | Value |
| --- | --- |
| Pack | \`${task.pack_id}\` |
| Task ID | \`${task.task_id}\` |
| Owner role | \`${task.owner_role}\` |
| Reviewer needed | \`${task.reviewer_needed}\` |
| Safety risk | \`${task.safety_risk}\` |
| Domain | ${domains} |
| Region | ${regions} |

## How to pick this up (AI agents welcome)

1. Read the problem framing: [\`${task.problem_file}\`](${REPO}/${task.problem_file})
2. Read your role guide: [\`${roleGuide}\`](${REPO}/${roleGuide})
3. Read the canonical task definition: [\`${task.tasks_file}\`](${REPO}/${task.tasks_file})
4. Comment here to claim the task (one agent at a time — Issues are work claims).
5. Open a submission using the **Agent Submission** issue template, or a pull request if you change canonical files.
6. Follow the submission quality standard: **one task, one role, one claim.** Every evidence record needs a falsifiable claim, a stable source URL, source + access dates, evidence type, method, limitations, and confidence.

## Why this matters

This is one scoped step inside a neglected global problem. The repo's scarce resource is *verified* work, not more prose. A small, well-sourced, reviewable contribution here is worth more than a sweeping unverifiable one.

---

🤖 Generated from \`tasks-available.json\` by \`scripts/generate-agent-issues.mjs\`. Source of truth is the pack's \`tasks.json\`.`;
}

function diversePicks(tasks, n) {
const perDomain = Math.max(1, Math.ceil(n / 11));
const cap = {};
const picks = [];
for (const t of tasks) {
const d = t.domain[0];
cap[d] = cap[d] || 0;
if (cap[d] < perDomain && picks.length < n) {
picks.push(t);
cap[d]++;
}
}
// top up if short
for (const t of tasks) {
if (picks.length >= n) break;
if (!picks.includes(t)) picks.push(t);
}
return picks;
}

const args = process.argv.slice(2);
const tasks = loadTasks();

if (args[0] === "--list") {
for (const t of tasks) console.log(`${t.pack_id}\t${t.task_id}`);
} else if (args[0] === "--body") {
const t = tasks.find((x) => x.pack_id === args[1]);
if (!t) {
console.error(`No task for pack_id ${args[1]}`);
process.exit(1);
}
console.log(`TITLE\t${issueTitle(t)}`);
console.log(`LABELS\t${issueLabels(t).join(",")}`);
console.log("BODY");
console.log(issueBody(t));
} else if (args[0] === "--plan") {
const n = Number(args[1]) || 24;
for (const t of diversePicks(tasks, n)) console.log(t.pack_id);
} else {
console.error("Usage: generate-agent-issues.mjs --list | --body <pack_id> | --plan [n]");
process.exit(1);
}
Loading