Creative Duel

This is a small harness created to compare Claude Fable 5 vs GPT-5.6 Sol. They are instructed to direct and create four 15-second films with a the same creation process, using the same skills and using the same MCP server for video generation.

Their taste and creativity are what's being judged.

The four tasks

task	brief	the test
sunscreen	invent a men's sunscreen brand from scratch and launch it	brand invention: name, identity, packaging, concept
burger	invent a burger restaurant from scratch and sell it	appetite, place-making, a signature with a point of view
codex-micro	launch the real Work Louder Codex Micro	web research only: facts AND visual references found, then faithfully generated
bigben	a mini-documentary on Big Ben, the great clock of London	documentary craft: research-backed facts, a faithful landmark, narration over sales tone

The codex-micro task features the real Work Louder Codex Micro; this project is not affiliated with or endorsed by Work Louder.

Same brief per task, same skills, same tools, same credit budget (head-to-head runs must use equal budgets). Web search is enabled for both agents (the runner passes Codex an explicit web_search flag; Claude Code has web search on by default, so no flag is passed). Concepts are developed visually on Nano Banana with a mandatory validation loop at every stage; all footage is generated; all on-screen text is overlaid, never model-rendered.

Layout

briefs/CORE.md          shared direction: judging, pipeline, loops, rules
tasks/<task>.md         the three assignments
skills/<name>/SKILL.md  craft manuals the agents must consult:
                        magic-hour-api, concepting, boards-identity,
                        screen-test, cinematography-prompts, dailies-qc,
                        voice-audio, edit-ffmpeg
duel.config             budget, models, MCP URL, pricing rates
setup.sh                one-time: deps check, MCP server install + registration
serve.sh                runs the Magic Hour MCP server (foreground)
run.sh                  unified runner
lib_run.sh              shared run plumbing (sourced by run.sh)
monitor.py              live STATUS line (backgrounded per run)
stream_render.py        renders both agents' JSON streams for the terminal
summarize.py            writes the run receipt (summary.md / summary.json)
runs/<task>/<slug>/<timestamp>/  preserved workspaces (+ latest symlink);
                        slugs are fable-5 / sol-5.6

Run it

Requirements: Linux or WSL, Python 3, ffmpeg, Node.js (for the two agent CLIs: @anthropic-ai/claude-code and @openai/codex, both logged in), and a Magic Hour API key. ./setup.sh checks all of this and tells you what's missing (on apt systems it offers to sudo apt-get install missing packages).

git clone <this repo> && cd creative-duel

export MAGIC_HOUR_API_KEY=mhk_...
./setup.sh                      # installs MCP server, registers both CLIs

# Terminal A (keep running):
./serve.sh

# Terminal B — ONE-TIME ARMING for unattended Claude runs:
claude --dangerously-skip-permissions
# accept the warning, ask it to "call the magic-hour ping tool", then quit

# The test (model, task, optional budget)
./run.sh fable sunscreen
./run.sh sol   sunscreen
./run.sh fable burger 6000
./run.sh sol   codex-micro
./run.sh fable bigben

Note: setup.sh registers the MCP server with the two CLIs differently — Claude Code stores the API key itself as a literal header in ~/.claude.json (user scope), while Codex stores only a reference to the MAGIC_HOUR_API_KEY env var. Re-run ./setup.sh if you rotate the key, and keep it exported in the shell you run from.

What each run produces

runs/<task>/<slug>/<timestamp>/
├── TASK.md            the exact rendered brief for this run
├── skills/            the craft manuals as given
├── brand.md           (invented-brand tasks) name, identity, palette
├── facts.md           (bigben) every narrated/on-screen fact with its source
├── script.md          concept, shot list, model table, screen test,
│                      dailies log, revision log
├── assets/            concept boards, shot boards, clips, audio,
│                      final_contact_sheet.png
├── final.mp4          15.0s ± 0.5s, 16:9, H.264 + AAC
├── manifest.json      every call, credits, iterations, self-scores
├── director_statement.md
├── agent.log          raw stream (stream-json for Fable, codex --json for Sol)
├── summary.md         the run receipt (see below)
└── summary.json       same numbers, machine-readable

Run summary

Every run ends with a receipt — printed to the terminal and saved as summary.md / summary.json in the run directory:

• tokens in / out / cached (from the agent CLI's own event stream)
• tool calls — agent-level calls, plus Magic Hour generation calls from the manifest
• Magic Hour credits spent
• assets generated (files in assets/)
• iterations and wall time
• turns (in summary.json) — for Fable this is the CLI-reported num_turns; for Sol it is a count of turn.completed events. Same field, loosely comparable semantics.
• costs at API list price — both agents' costs are computed from tokens at the fixed rates in duel.config (CLI-reported costs are ignored, since they vary with account plans), so the figures are directly comparable. Set MAGIC_HOUR_USD_PER_CREDIT there to price generation spend into the total. For fair head-to-head cost comparisons, leave the pricing block at list defaults on both sides.

Watching a run

Live agent activity streams in the terminal (tool calls + the director's commentary), interleaved with cyan ── STATUS lines on every state change: stage, credits spent, calls, iterations, last call's purpose — read from the manifest the agent maintains.

Fairness notes

• Rendered briefs are byte-identical apart from model label and budget.
• Every launch is a fresh, timestamped, isolated workspace; past runs are preserved; neither agent sees the other's work.
• Head-to-head comparisons require equal budgets and the same task.
• Self-scored creativity is directional; you are the judge. The run summaries carry the stats for captions.

Troubleshooting

• Claude run stalls immediately → do the one-time arming step.
• Codex MCP calls die with "user cancelled MCP tool call" → known Codex bug (openai/codex#16685): exec cancels MCP calls under managed sandboxes; the runner uses --dangerously-bypass-approvals-and-sandbox, the mirror of Claude's --dangerously-skip-permissions.
• Model name rejected → adjust CLAUDE_MODEL / CODEX_MODEL in duel.config to what your CLI versions expose.
• 15s comes out as 13s or 17s → that's a result. Log it.

License

MIT — see LICENSE.