Repository navigation
ci: cache the cargo registry and build layers in the container jobs - #34
Open
spacedevin wants to merge 1 commit into
Open
spacedevin wants to merge 1 commit into
spacedevin wants to merge 1 commit into
Conversation
The toolchain image ships rustc, libmgba and agb-gbafix, but every crates, rom and verify
job still started from an empty CARGO_HOME registry and an empty target dir: the crates.io
index, the agb fork clone and the whole dependency layer (core/alloc from build-std, agb
and everything under it) were downloaded and compiled from scratch, ~92 times per run.
- crates: actions/cache over /usr/local/cargo/{registry/index,registry/cache,git/db} (the
image's CARGO_HOME is /usr/local/cargo, not ~/.cargo) and crates/*/target — these crates
are excluded from the root workspace, so each has its own target dir.
- rom + verify: every ROM build in a run now goes through ONE target dir, via tish's own
TISH_GBA_TARGET_DIR opt-in (the ambient CARGO_TARGET_DIR breaks tish's ELF lookup, which
is why rom.sh scrubs it). The dependency layer has the same fingerprint whichever example
built it, so a target dir warmed by grid-demo is warm for all 90. The rom job — always
present, exactly one of it — is the single writer; the verify jobs restore only, so 89
jobs do not race to save the same key.
Keys lead with the nightly pin so restore-keys never hand one rustc's artifacts to another,
then hash what fixes the dependency graph (.cargo/config.toml for the agb rev, package.json
for the tish version, the crate manifests / lockfiles). Path dependencies under crates/ and
node_modules are fingerprinted by mtime and a fresh checkout is always newer, so those and
the game crate itself still compile per job; the cache is everything beneath them.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
The
crates,romandverifyjobs run inghcr.io/schlopai/chuggie-toolchain, which ships rustc, libmgba and agb-gbafix but no warm build. Every job (92 per full run) started with an empty registry and an empty target dir: crates.io index fetch, a clone of the agb fork, and a from-scratch compile of core/alloc (-Z build-std), agb and everything under it, before the example's own code was touched. Latest main run: theverify grid-demo"Run the verifier" step (build + 4000-frame emulation) is 1m44s of a ~2m50s job.Change
cratesactions/cache@v6over the registry (/usr/local/cargo/registry/{index,cache},/usr/local/cargo/git/db— the image setsCARGO_HOME=/usr/local/cargo, not~/.cargo) andcrates/*/target(these crates are excluded from the root workspace, so each has its own target dir, asscripts/clean.shdocuments).romandverify: one shared target dirtish buildnormally writes each example's build under its own.tish/gba/<pkg>/target, a path no cache key could share across the 90 verify jobs, andCARGO_TARGET_DIRis deliberately scrubbed by tish (it breaks the ELF lookup;scripts/rom.shdocuments the trap). tish has an explicit opt-in for exactly this case —TISH_GBA_TARGET_DIR(crates/tish_native/src/build.rsin @tishlang/tish 3.10.3: "share ONE cargo target dir across every example's ROM build … measured at ~45 GB across the fleet") — which points cargo and the ELF lookup at one directory. Both jobs set it to${{ github.workspace }}/target/gba.The dependency layer has the same fingerprint whichever example compiled it (same absolute paths for the path deps, same git rev for agb, same sysroot for build-std), so a target dir warmed by
grid-demois warm for every example.romis the single writer (actions/cache@v6): it always runs, there is exactly one of it, and its post-step save only happens on success.verifyjobs restore only (actions/cache/restore@v6) with the same key and paths. 90 jobs saving the same key would be 89 "cache already exists" warnings and 89 archives built for nothing.Keys
with
restore-keyson the<prefix>-<toolchain hash>-part. The nightly pin leads so a restore never crosses a rustc change (cargo would ignore those artifacts, and they would only bloat the next save)..cargo/config.tomlcarries the agb rev,package.jsonthe exact tish version;package-lock.jsonis deliberately not in the key, since deps bumps like the deckfile chores would otherwise miss the cache for a change that does not touch the Rust graph.What this does and does not cache
Cargo fingerprints path dependencies by mtime, and a fresh checkout /
npm ciis always newer than the cached fingerprints, socrates/tish-agb,tish-gba-scenepack,tish-agb-sio,tish-gba-game-engine,node_modules/@tishlang/tish/crates/tish_runtime_gbaand the game crate itself still compile in every job. What stays warm is everything beneath them: the registry index, the agb fork clone, core/alloc/compiler_builtins from build-std (rust-src in the image has fixed mtimes), agb and its crates.io dependencies, and scenepack's proc-macro dependencies (syn/quote/proc-macro2, host builds). That is the bulk of a cold build, but not all of it.Caveats, stated plainly
zstd, soactions/cachewill fall back to gzip in these jobs (save and restore are both in the container, so they agree). Addingzstdto the image's apt list (schlopai/chuggie-toolchain, not this repo) would make both ends faster..tish/gba/<pkg>/targetafter this: the verifiers that reach into.tish/gba/readsrc/main.rsandCargo.toml(checked:grep -rn 'target/' examples/*/verify.sh scripts/*.sh scripts/*.py);check_agb_fork.shusescargo metadata, which does not involve the target dir. The twoverify.shthatunset CARGO_TARGET_DIRdo not touchTISH_GBA_TARGET_DIR.Swatinem/rust-cachewas not used: it assumes~/.cargoand a workspacetarget/, neither of which is the layout here.Both files parse (
actionlint, YAML load); the one SC2155 note actionlint reports is pre-existing on main.