ci: enforce a 95% per-crate coverage gate in contracts.yml (#1120) - #1244
Open
jjb9707 wants to merge 5 commits into
Open
ci: enforce a 95% per-crate coverage gate in contracts.yml (#1120)#1244jjb9707 wants to merge 5 commits into
jjb9707 wants to merge 5 commits into
Conversation
Adds cargo-llvm-cov to the Contracts CI workflow with: - llvm-tools-preview component for code instrumentation - taiki-e/install-action to install cargo-llvm-cov (pre-built binary) - --fail-under-lines 95 per-crate coverage gate Closes Stellopay#1120
The gate added earlier ran `cargo llvm-cov --workspace --fail-under-lines 95`, which only enforces a workspace-wide *average*. Issue Stellopay#1120 asks for a per-crate gate ("fail the job if any crate's line coverage is below 95 percent, with a clear per-crate summary in the job output"), and with 29 crates under onchain/contracts/ an average lets one uncovered crate hide behind well-covered ones. Instrumentation is still collected in a single workspace run so the gate stays cheap, then attributed back to each crate from the llvm-cov JSON export. Files under a crate's tests/ directory or a src/tests/ module are excluded so test code cannot inflate a crate towards 100%. The step prints a per-crate table to the job log and to $GITHUB_STEP_SUMMARY, and fails listing every crate below the threshold. The threshold is a COVERAGE_THRESHOLD env var on the step. Also documents the gate in docs/ci.md, whose "Coverage thresholds" section still claimed no minimum was enforced, including how to reproduce the per-crate numbers locally. Corrects the toolchain line in the same doc, which still said "stable" after the workflow moved to nightly.
# Conflicts: # docs/ci.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a per-crate line-coverage gate to
.github/workflows/contracts.yml: the build fails when any crate underonchain/contracts/drops below 95% line coverage.Changes
.github/workflows/contracts.ymlllvm-tools-previewto the Rust toolchain components (required bycargo-llvm-covfor instrumentation)cargo-llvm-covvia taiki-e/install-action (pre-built binary, faster thancargo install)Per-crate coverage gate (fail under 95% lines)step: the workspace is instrumented once withcargo llvm-cov --workspace --json, then line counts are attributed back to each crate and every crate is checked against the threshold individually::error::annotation naming how many crates are below the barCOVERAGE_THRESHOLDenv var on that stepdocs/ci.md— documents the gate, the exact commands to reproduce the CI numbers locally, the extra prerequisites, and a troubleshooting entry.Why not
--fail-under-lineson its owncargo llvm-cov --workspace --fail-under-lines 95only enforces a workspace-wide average, so a single uncovered crate can hide behind well-covered ones. This issue asks for a per-crate bar, so instrumentation is collected once (one test run, keeping the gate cheap) and then attributed per crate. Files under atests/orsrc/tests/directory are excluded, so test code cannot inflate a crate's own score.Testing
cargo llvm-covJSON-export fixtures covering:src/tests/exclusion, crate-roottests/exclusion, multi-file crates aggregated per crate rather than judged per file, a crate exactly on the threshold (95.00 -> pass), and a crate below it (->below 95%row plus non-zero exit).Note on the current red check
The
contractsjob stops earlier, at the pre-existingcargo fmt --all -- --checkstep. That step checks the whole repository, and reports the same 143 violations across 39 files onmainas it does on this branch, so the failure is not introduced here and the coverage-gate step is never reached. Once formatting is resolved onmain, this gate will run.Closes #1120