README.md

Live Raw-Upstream Challenges

This directory holds the live, non-gating raw-upstream challenge track for mcpcraft. Each case lives in its own folder with a canonical challenge.toml, any tiny checked-in fixtures it needs, and a generated README.md that explains why the case exists and how to run it in isolation.

Why These Cases Matter

• They pressure-test one-shot ingestion against real engineering packages and CLIs.
• Many cases assert scaffold breadth explicitly so the suite checks more than one route.
• They stay separate from the public example contract so we can keep them brutally honest.
• Unlike the real-world examples, they intentionally keep ingestion and workflow compressed into challenge.toml so contributors still have to solve the wrapping problem themselves.
• They are still documented enough to serve as worked examples for contributors.

Suite Shape

At the suite level, each family climbs a difficulty ladder while every individual case still follows the same deterministic scaffold-to-runtime path.

flowchart LR
  ladder["Per-family ladder<br/>easy -> medium -> hard -> insane"] --> case["One catalogued challenge"]
  case --> spec["challenge.toml"]
  case --> fixtures["tiny fixtures or rendered inputs"]
  case --> probe["availability probes"]
  spec --> scaffold["scaffold package or command"]
  probe --> scaffold
  scaffold --> facade["generated facade"]
  facade --> manifest["build manifest"]
  spec --> workflow["workflow step(s) through the normal runtime"]
  fixtures --> workflow["workflow step(s) through the normal runtime"]
  manifest --> workflow
  workflow --> reports["stable JSON / JUnit / Markdown reports"]
  workflow --> artifacts["retained workflow artifacts"]

Loading

Run The Suite

make challenge-deps
make challenges-subset
make challenges-full
make challenge CASE=openmdao_file_utils
PYTHONPATH=src .venv/bin/python scripts/run_challenges.py --tier all --family avl

Install Optional Runtimes

The broader challenge lane expects its extra CLI tools under the repo-local .challenge-tools/ prefix. The challenge, challenges-subset, challenges-full, and run-real-world-examples targets prepend .challenge-tools/bin to PATH automatically.

make challenge-deps
make challenge-deps PROFILE=subset

Family Ladders

Family	easy	medium	hard	insane
aerosandbox	aerosandbox_easy	aerosandbox_root	aerosandbox_hard	aerosandbox_insane
avl	avl_easy	avl_medium	avl_hard	avl_insane
build123d	build123d_easy	build123d_importers	build123d_hard	build123d_root
gmsh	gmsh_easy	gmsh_cli	gmsh_module_root	gmsh_insane
openmdao	openmdao_easy	openmdao_file_utils	openmdao_hard	openmdao_insane
pynastran	pynastran_easy	pynastran_bdf	pynastran_hard	pynastran_insane
su2	su2_easy	su2_cli	su2_hard	su2_insane
submitit	submitit_easy	submitit_medium	submitit_hard	submitit_root
xfoil	xfoil_easy	xfoil_cli	xfoil_hard	xfoil_insane

Case Inventory

Case	Family	Difficulty	Tier	Slice	Target	Summary
aerosandbox_easy	aerosandbox	easy	local_full	mission	aerosandbox	Wrap AeroSandbox's Atmosphere class, query density, and save a serialized atmosphere state.
aerosandbox_hard	aerosandbox	hard	local_full	mission	aerosandbox	Wrap AeroSandbox's Atmosphere and OperatingPoint classes, query multiple physics methods, and retain serialized state files for both.
aerosandbox_insane	aerosandbox	insane	local_full	mission	aerosandbox	Wrap AeroSandbox's Atmosphere and OperatingPoint classes, query a broader set of thermodynamic and performance methods, and retain both serialized states.
aerosandbox_root	aerosandbox	medium	local_full	mission	aerosandbox	Wrap AeroSandbox's Atmosphere class in one shot, query several real instance methods, and retain a saved state file.
avl_easy	avl	easy	local_full	aerodynamics	avl	Wrap the avl executable, solve one tiny wing operating point, and retain a total-forces report.
avl_hard	avl	hard	local_full	aerodynamics	avl	Wrap the avl executable, load stored geometry/mass/run inputs, execute a saved run case, and retain forces plus stability reports.
avl_insane	avl	insane	local_full	aerodynamics	avl	Wrap the avl executable, execute a stored run case, retain forces plus stability reports, and then write AVL eigenvalue and system-matrix outputs.
avl_medium	avl	medium	local_full	aerodynamics	avl	Wrap the avl executable, solve one tiny wing operating point, and retain both total-forces and stability-derivative reports.
build123d_easy	build123d	easy	gha_subset	manufacturing	build123d.importers	Wrap build123d.importers and import one tiny checked-in STL face.
build123d_hard	build123d	hard	local_full	manufacturing	build123d.importers	Wrap build123d.importers and exercise STL, SVG, STEP, BREP, and SVG-to-code import routes with tiny checked-in fixtures.
build123d_importers	build123d	medium	gha_subset	manufacturing	build123d.importers	Wrap build123d.importers and exercise both STL and SVG import routes with tiny checked-in fixtures.
build123d_root	build123d	insane	local_full	manufacturing	build123d	Wrap the build123d package root while filtering down to five real importer routes across STL, SVG, STEP, BREP, and SVG-to-code workflows.
gmsh_cli	gmsh	medium	gha_subset	aerodynamics	gmsh	Wrap the gmsh CLI, feed it a tiny .geo model, and verify a 3D mesh file is produced.
gmsh_easy	gmsh	easy	gha_subset	aerodynamics	gmsh	Wrap the gmsh CLI, mesh a tiny checked-in box geometry in 2D, and retain the resulting mesh file.
gmsh_insane	gmsh	insane	local_full	aerodynamics	gmsh	Wrap the gmsh CLI, generate a 3D mesh, partition it into two parts, and retain both split mesh files.
gmsh_module_root	gmsh	hard	local_full	aerodynamics	gmsh	Wrap the Python gmsh module root, open a checked-in mesh, and convert it to VTK through the top-level module API.
openmdao_easy	openmdao	easy	gha_subset	systems	openmdao.utils.file_utils	Wrap concrete openmdao.utils.file_utils helpers and inspect one checked-in Python module like a real utility workflow.
openmdao_file_utils	openmdao	medium	gha_subset	systems	openmdao.utils.file_utils	Wrap concrete openmdao.utils.file_utils helpers and inspect a checked-in demo package like a real utility workflow.
openmdao_hard	openmdao	hard	local_full	systems	openmdao.utils	Wrap openmdao.utils while filtering to file_utils, render a marked _out directory, and verify that clean_outputs removes it.
openmdao_insane	openmdao	insane	local_full	systems	openmdao	Wrap the full openmdao package root while filtering down to utils.file_utils, then verify module-path helpers, HTML rendering, tempdir lookup, and real output-directory cleanup.
pynastran_bdf	pynastran	medium	gha_subset	structures	pyNastran.bdf.bdf	Wrap pyNastran's BDF class, create a node, inspect model stats, and write a non-empty deck file.
pynastran_easy	pynastran	easy	gha_subset	structures	pyNastran.bdf.bdf	Wrap pyNastran's BDF class, add one grid point, and write a non-empty deck file.
pynastran_hard	pynastran	hard	local_full	structures	pyNastran.bdf.bdf	Wrap pyNastran's BDF class, build a tiny shell panel model, validate it, inspect stats, and write the deck.
pynastran_insane	pynastran	insane	local_full	structures	pyNastran.bdf.bdf	Wrap pyNastran's BDF class, build a tiny shell panel deck, round-trip it through a second session, and write the reloaded copy.
su2_cli	su2	medium	local_full	aerodynamics	SU2_CFD	Wrap the SU2_CFD executable and run a tiny adapted official tutorial case when it is available.
su2_easy	su2	easy	local_full	aerodynamics	SU2_CFD	Wrap the SU2_CFD executable and run a one-iteration direct solve that writes history and surface CSV outputs when SU2 is installed.
su2_hard	su2	hard	local_full	aerodynamics	SU2_CFD	Wrap the SU2_CFD executable and run a tiny direct solve that writes history, surface CSV, restart, and ParaView volume outputs when SU2 is installed.
su2_insane	su2	insane	local_full	aerodynamics	SU2_CFD	Wrap the SU2_CFD executable and run a tiny direct solve that writes history, restart, surface CSV, volume ParaView, and surface ParaView outputs when SU2 is installed.
submitit_easy	submitit	easy	gha_subset	hpc	submitit.helpers	Wrap submitit.helpers.run_cmd, execute a tiny shell command, and retain the file it creates.
submitit_hard	submitit	hard	local_full	hpc	submitit	Wrap the submitit package root, retain executor entry points plus helpers.run_cmd, and use a checked-in script to run a real LocalExecutor job.
submitit_medium	submitit	medium	local_full	hpc	submitit.helpers	Wrap submitit.helpers.run_cmd, launch a checked-in helper script, and verify that DebugExecutor creates real job logs.
submitit_root	submitit	insane	local_full	hpc	submitit	Wrap the submitit package root, retain executor entry points plus helpers.run_cmd, and use a checked-in script to run DebugExecutor, LocalExecutor, and AutoExecutor jobs.
xfoil_cli	xfoil	medium	local_full	aerodynamics	xfoil	Wrap the xfoil executable and run a tiny batch polar-generation workflow when it is installed.
xfoil_easy	xfoil	easy	local_full	aerodynamics	xfoil	Wrap the xfoil executable and save a tiny one-point polar file for NACA 0012 when XFOIL is installed.
xfoil_hard	xfoil	hard	local_full	aerodynamics	xfoil	Wrap the xfoil executable and save two viscous polar files at different Reynolds numbers when XFOIL is installed.
xfoil_insane	xfoil	insane	local_full	aerodynamics	xfoil	Wrap the xfoil executable, store two viscous polars in memory, and write them out later with PWRT when XFOIL is installed.