The Verify live chain matches the fresh pins step in rainix-tag-release runs forge test — the consumer's whole suite, since neither caller sets test-cmd. Sorted by what actually changed between the last PR CI run and the tag, that set is wrong in both directions at once.
What the release-time run actually checks
| What runs |
Changed since PR CI? |
Information |
| Unit and other tests |
No — source is byte-identical, only generated pins and the version line differ |
none |
Tagged-constants tests (0_1_3, 0_1_4, 0_1_5) |
Cannot have — those dirs are frozen and an append-only gate enforces it |
none |
| Prod alias fork test |
Yes — the pin was regenerated by cut-release, and the manual deploy landed since |
this is the only informative check |
The dir just frozen (0_1_6/) |
Yes, it did not exist minutes ago |
nothing imports it — unchecked |
So a release re-verifies that immutable data is still immutable, re-runs a suite whose source has not moved, and publishes the one artifact nothing asserted anything about.
Why the new snapshot is uncovered
LibCloneFactoryDeployTaggedConstants.t.sol imports literal versioned paths with per-version aliases and per-version function names, and its own doc comment states the workflow: "A new release adds a tag import + a test pair here." The pair is hand-written, in a PR after the snapshot exists. At tag time the freshly frozen dir has no test pair yet, so its self-consistency and reproducibility assertions arrive only after the bytes are published and frozen under the append-only gate.
Under the rolling-candidate model, testCandidateSelfConsistent covers the regenerated candidate — but not the frozen copy, and nothing compares a numbered dir to candidate.
The assertions already exist, pointed at the wrong directories
The tagged test does three things per version, and they are the whole meaningful verification:
keccak256(RUNTIME_CODE) == BYTECODE_HASH — the snapshot is internally consistent. Pure, local, no RPC.- Zoltu-deploying
CREATION_CODE lands at DEPLOYED_ADDRESS with that codehash — the snapshot reproduces its own deployment. Pure, local, no RPC.
- The live chain carries that code at that address — the deploy landed. Needs a fork.
Run those against the directory cut-release just froze and the release is verified. Everything else in the suite is noise.
Why the noise is not free
rainix-tag-release's own header keeps the on-chain deploy out of the workflow because "a flaky, retry-prone operation must not gate a one-shot tag publish, where a single transient failure would block the release entirely."
Running the full suite re-admits exactly that risk. A tag is spent once used — recovery from a failed release is a NEW tag, not a rerun, which is why sol-v0.1.6 is dead in rain.factory.deploy. So every flaky test in an 88-file suite can burn a release, and the fork tests are the flakiest part: a free-plan RPC quota already red-lined rain.factory.deploy#5's first run on 2026-08-13.
The workflow applies its own principle to the deploy and then violates it with test-cmd: forge test.
Ask
Related: #301 and #302 (rainix owning cut-release), and the observation that pointer generation should be part of build rather than a separate script — which is what would let the tagged assertions be generated alongside the constants.
The
Verify live chain matches the fresh pinsstep inrainix-tag-releaserunsforge test— the consumer's whole suite, since neither caller setstest-cmd. Sorted by what actually changed between the last PR CI run and the tag, that set is wrong in both directions at once.What the release-time run actually checks
0_1_3,0_1_4,0_1_5)cut-release, and the manual deploy landed since0_1_6/)So a release re-verifies that immutable data is still immutable, re-runs a suite whose source has not moved, and publishes the one artifact nothing asserted anything about.
Why the new snapshot is uncovered
LibCloneFactoryDeployTaggedConstants.t.solimports literal versioned paths with per-version aliases and per-version function names, and its own doc comment states the workflow: "A new release adds a tag import + a test pair here." The pair is hand-written, in a PR after the snapshot exists. At tag time the freshly frozen dir has no test pair yet, so its self-consistency and reproducibility assertions arrive only after the bytes are published and frozen under the append-only gate.Under the rolling-candidate model,
testCandidateSelfConsistentcovers the regeneratedcandidate— but not the frozen copy, and nothing compares a numbered dir tocandidate.The assertions already exist, pointed at the wrong directories
The tagged test does three things per version, and they are the whole meaningful verification:
keccak256(RUNTIME_CODE) == BYTECODE_HASH— the snapshot is internally consistent. Pure, local, no RPC.CREATION_CODElands atDEPLOYED_ADDRESSwith that codehash — the snapshot reproduces its own deployment. Pure, local, no RPC.Run those against the directory
cut-releasejust froze and the release is verified. Everything else in the suite is noise.Why the noise is not free
rainix-tag-release's own header keeps the on-chain deploy out of the workflow because "a flaky, retry-prone operation must not gate a one-shot tag publish, where a single transient failure would block the release entirely."Running the full suite re-admits exactly that risk. A tag is spent once used — recovery from a failed release is a NEW tag, not a rerun, which is why
sol-v0.1.6is dead in rain.factory.deploy. So every flaky test in an 88-file suite can burn a release, and the fork tests are the flakiest part: a free-plan RPC quota already red-lined rain.factory.deploy#5's first run on 2026-08-13.The workflow applies its own principle to the deploy and then violates it with
test-cmd: forge test.Ask
Related: #301 and #302 (rainix owning
cut-release), and the observation that pointer generation should be part ofbuildrather than a separate script — which is what would let the tagged assertions be generated alongside the constants.