AMT coverage: g7-libdescribedbymeta-emitf - #152
Conversation
The non-meta halves of testIsRainMetaV1_2Fuzz,
testCheckMetaUnhashedV1_2Fuzz and testCheckMetaHashedV1_2BadMagicGoodHash
all treated arbitrary fuzzed bytes as never carrying the rain magic
prefix. The magic number is a PUSH8 in the suite bytecode, so it sits in
the fuzz dictionary, and any seed that emits it as the head of a bytes
argument turned the suite red on unmutated code:
args=[0xff0a89c674ee7874] -> isRainMetaV1 is true, checkMeta* passes,
the expected NotRainMetaV1 revert never comes.
The two revert tests now enforce the premise with the same
vm.assume(!LibMeta.isRainMetaV1(...)) idiom testEmitMetaNotRainMeta
already uses; the isRainMetaV1 test instead breaks a fuzzed prefix
deterministically (the magic starts 0xff, a zeroed first byte is never
the prefix) so its negative half keeps asserting over every run rather
than discarding.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
testArtifactsCommitted proves every artifact contracts() names is fresh, which a mutation probe shows cannot see: - a name dropped from (or duplicated in) contracts(): the committed copy of the dropped artifact stays behind as an orphan nothing regenerates, still compiled into the rust crate (probe: contracts()[1] replaced by a duplicate SURVIVED); - committedPath drifting inside the permitted directory: the regenerated copies land at the drifted paths and the freshness test reads its own writes back (probe: .json -> .abi.json SURVIVED); - extractStable filter drift: the committed subset is compared against the same filter that wrote it (probe: keeping the full bytecode objects SURVIVED, dropping the abi key SURVIVED). testCommittedAbiDirMatchesContracts pins contracts() to the two artifacts crates/bindings/src/lib.rs consumes, spelled as literals the way SubgraphManifest.t.sol spells the indexed signature, and holds the directory to exactly that set. testCommittedArtifactIsTheStableSubset pins the committed shape to literals and the kept values to the live artifact with this file's own jq spellings, so the filter under test has nothing to agree with but the intent. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Every Solidity test derives its meta inputs from META_MAGIC_NUMBER_V1, so no test could see the literal itself drift (probe: off-by-one on the constant SURVIVED the whole suite). MetaMagicNumberV1.t.sol spells the eight spec bytes as a preimage, the one place the sol lane can catch a mutation to either side, aligned with the independent spellings in crates/cli/src/meta/magic.rs and subgraph/tests/metaBoard.test.ts. AbiSurface.t.sol pins the artifact surface the consumers bind against: MetaV1_2 keeps all three inputs unindexed (indexing one re-layouts every log under alloy::sol! and graph codegen decoders), emitMeta is exactly (bytes32,bytes) nonpayable, and describedByMetaV1 is exactly () view returns (bytes32) — view being what lets tooling eth_call a described contract for its immutable hash. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Plus Run ID: 📒 Files selected for processing (6)
Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review. WalkthroughThe PR adds tests for ABI surfaces, the metadata magic number, fuzz-test premises, and committed artifact integrity. The artifact tests compare committed files with live build outputs and enforce the consumed artifact set. ChangesInterface and artifact validation
Estimated code review effort: 3 (Moderate) | ~20 minutes Merge Risk: ⚪ Minimal · up to The PR strengthens adversarial test coverage and makes fuzz-test premises deterministic without changing production behavior; the supplied checks are green and all targeted mutations are detected, so no actionable merge-blocking risk remains beyond normal review. 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Independent verification from a second g7 worker in the same campaign: a from-scratch probe of this slice — 27 exact-string mutants written blind to this branch — corroborates the matrix above. Pass 1, pre-existing suite only, base bba50a7: 21/27 killed. Survivors were exactly the five self-referential oracles this PR pins: contracts() dropping IDescribedByMetaV1, committedPath drifting inside the permitted dir ( Pass 3, same 27 mutants against this branch (head 1dc3bd9): 26/27 KILLED, 0 SURVIVED, baseline green (22 tests). Probe-named killers agree with the matrix: Attribution spot-checks on the pre-existing kills also agree: hash/guard/error-args/subject/verbatim mutants die to testEmitForDescribedAddressHappy/Mismatch; indexed-sender dies to testEmitMetaHappy (raw topic count) and testEmitMetaEmptyBody (raw log data) even before the new artifact pin runs; view-drop dies to testEmitForDescribedAddressHappy via the expectEmit next-call/staticcall distinction; the nine manifest mutants die to the three SubgraphManifest tests as listed. Divergences: none. Two additional findings filed from this pass's adversarial half: #205 (cross-lane residue that remains after testCommittedAbiDirMatchesContracts, narrowed in a follow-up comment there) and #206 (MetaV1 entity id collision in the subgraph handler, latent under the single-dataSource pin). |
1 participant
Adversarial mutation-test coverage for group g7:
LibDescribedByMeta.emitForDescribedAddress,LibCopyArtifacts, theCopyArtifactsscript, the interface/constant surface (IMetaV1_2,IMetaBoardV1_2,IDescribedByMetaV1,META_MAGIC_NUMBER_V1) and thesubgraph.yamlmanifest data under its Solidity oracle. Base commit bba50a7.Probe:
mutation-probe(adversarial-mutation-test flake), suite = therainix-sollane's build + artifact regeneration +forge test, with regeneration running INSIDE the per-mutant suite so the committed-ABI freshness oracle is never comparing against a stale copy, and the committed-abi dir reset per mutant so one mutant's regenerated residue cannot leak into the next verdict.Behaviour matrix
Baseline repair (pre-existing red)
Adding a new bytes spelling of the magic literal raised forge's fuzz-dictionary hit rate and turned three PRE-EXISTING LibMeta fuzz tests red on unmutated code (seed 0x5f8fac0b...): their negative halves assumed arbitrary fuzzed bytes never carry the magic prefix, while the magic is a PUSH8 in the suite's own bytecode and therefore in the fuzz dictionary. Fixed by enforcing the premise (the
vm.assume(!isRainMetaV1(...))idiomtestEmitMetaNotRainMetaalready uses) and, fortestIsRainMetaV1_2Fuzz, by deterministically breaking a fuzzed prefix so the negative half keeps asserting on every run. Filed as #150. No assertion was matched to buggy behaviour; the library behaviour is correct, the tests' premise was not.Also filed from the adversarial pass: #151 (manifest
mapping.entitiesomitsTransaction, which the handler writes).QA
--onlyover the five pass-1 survivors against the strengthened suite: 5/5 KILLED with the new test named as killer by the probe's fail-pattern) and all are green on the unmutated baseline of every pass.git checkout -- crates/bindings/abi && git clean -fdq crates/bindings/abi && forge build && (forge script script/CopyArtifacts.sol --ffi || true) && forge testundernix develop) so no verdict was read off a stale committed artifact; the mutants file is probe input and deliberately not committed.Summary by CodeRabbit