CaP-X Leaderboard

A small, self-contained leaderboard for code-as-policy agents on the CaP-X manipulation benchmark. All data lives in one CSV; index.html reads it at runtime, so updating the leaderboard is just editing a CSV — no build step.

.
├── index.html    # the page (fetches + renders summary.csv)
└── summary.csv   # all teams' scores (one row per team per simulation)

Viewing it

Browsers block fetch() of local files over file://, so serve the folder over HTTP:

python -m http.server 8000
# then open http://localhost:8000

(Any static host works too — e.g. GitHub Pages serves it directly.)

Metrics

Two simulators (robosuite, libero), two metrics each:

Metric	Meaning	Direction
success_rate	Task-success rate — share of trials that complete the task (task_completed / reward == 1.0), as a percent 0–100	higher is better
inference_speed	Average wall-clock seconds per trial	lower is faster

The Overall column is the mean of the two simulators' success rates and drives the default ranking (medals). Click any column header to re-sort.

CSV schema

summary.csv is "long" format — one row per team per simulation (each team has exactly two rows: robosuite and libero):

team,category,model,simulation,success_rate,inference_speed
Gemini 3 Pro,paper-closed,google/gemini-3.1-pro-preview,robosuite,88.5,14.7
Gemini 3 Pro,paper-closed,google/gemini-3.1-pro-preview,libero,67.0,16.2
Qwen3-Coder,paper-open,Qwen/Qwen3-Coder (best OSS),robosuite,74.0,9.8
Qwen3-Coder,paper-open,Qwen/Qwen3-Coder (best OSS),libero,52.5,11.0

Column	Notes
team	Display name (repeated on both rows).
category	One of paper-closed, paper-open, baseline, community — sets the badge.
model	Optional subtitle under the team name (leave empty for none).
simulation	robosuite or libero.
success_rate	Percent, 0–100.
inference_speed	Seconds per trial.

Updating

• Change a score: edit the number in summary.csv, refresh the page.
• Add a team: append its two rows (robosuite + libero, schema above) to summary.csv.
• Add a simulator later: add a row per team and extend the columns/labels in index.html (COLUMNS + the <thead> group headers + aggregateTeam).

Hosting (GitHub Pages)

The site is plain static files with no build step, so any static host works. It's wired for GitHub Pages via .github/workflows/deploy.yml, which publishes the repo root as the site root on every push to main.

Go live (one time). Free GitHub Pages needs a public repo:

1. Make the repo public (Settings → General → Change visibility). This exposes the whole repo and its history — confirm nothing sensitive is in there first.
2. Settings → Pages → Source → GitHub Actions.
3. Push to main (or Actions → Deploy leaderboard to Pages → Run workflow) to trigger the first deploy.
4. Live at https://<user>.github.io/<repo>/ — for this repo, https://shuperposition.github.io/web/.

Weekly update (~1 min). Once live, drop the new weekly summary.csv in place, then:

git add summary.csv
git commit -m "leaderboard: <week>"
git push          # the workflow redeploys automatically; live in ~1 min

Data provenance ⚠️

These numbers are illustrative, not official results:

• Paper teams (Gemini 3 Pro, Qwen3-Coder) are marked (est.) in the UI. The CaP-X paper's result tables are not bundled in this repo, so their scores are estimates grounded in the repo's real anchors (the robosuite regression rewards in docs/development.md; LIBERO being empirically much harder than robosuite in actual outputs/ runs). Replace them with the paper's real numbers when available.
• Community teams are entirely mock submissions for demonstration.