Skip to content

feat: add IFBench training configuration#2488

Open
darioandyoshi-tech wants to merge 1 commit into
PrimeIntellect-ai:mainfrom
darioandyoshi-tech:feat/ifbench-integration
Open

feat: add IFBench training configuration#2488
darioandyoshi-tech wants to merge 1 commit into
PrimeIntellect-ai:mainfrom
darioandyoshi-tech:feat/ifbench-integration

Conversation

@darioandyoshi-tech
Copy link
Copy Markdown

@darioandyoshi-tech darioandyoshi-tech commented May 13, 2026

Summary

Adds a minimal prime-rl training config/runbook for the IF-RLVR/Bench bounty candidate.

  • Adds configs/ifbench/rl.toml
  • Uses the public Prime Intellect Lab env ID shashwat23/ifbench
  • Adds configs/ifbench/README.md with run instructions, source links, and validation notes

Validation

python3 - <<'PY'
import tomllib, pathlib
for p in ['configs/ifbench/rl.toml','configs/gsm8k/rl.toml','configs/hendrycks_math/rl.toml']:
    tomllib.loads(pathlib.Path(p).read_text())
    print('ok', p)
PY

Result:

ok configs/ifbench/rl.toml
ok configs/gsm8k/rl.toml
ok configs/hendrycks_math/rl.toml

No heavy GPU training was run locally.

Maintainer question

Please confirm shashwat23/ifbench is the canonical environment ID for the bounty. If a different env ID is preferred, this PR should only need a one-line config update.


Note

Low Risk
Low risk: adds new config/docs only and does not change runtime code paths, aside from referencing an external environment ID that may need confirmation.

Overview
Adds a new configs/ifbench/rl.toml training config mirroring existing lightweight RL configs, targeting the slash-qualified Prime Intellect Lab env shashwat23/ifbench (auto-installed by prime-rl).

Includes configs/ifbench/README.md with links, run instructions, and a small TOML-parse validation snippet, plus a note to confirm the canonical environment ID.

Reviewed by Cursor Bugbot for commit b3a9e7f. Bugbot is set up for automated code reviews on this repo. Configure here.

@darioandyoshi-tech darioandyoshi-tech force-pushed the feat/ifbench-integration branch from 5d07e59 to b3a9e7f Compare May 13, 2026 14:50
@darioandyoshi-tech
Copy link
Copy Markdown
Author

Hi team, just pinging this for a final review. The IFBench training configuration is ready for merge. Thanks! 🍈

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant