Skip to content

ci: enforce a 95% per-crate coverage gate in contracts.yml (#1120) - #1244

Open
jjb9707 wants to merge 5 commits into
Stellopay:mainfrom
jjb9707:ci-per-crate-coverage-1120
Open

ci: enforce a 95% per-crate coverage gate in contracts.yml (#1120)#1244
jjb9707 wants to merge 5 commits into
Stellopay:mainfrom
jjb9707:ci-per-crate-coverage-1120

Conversation

@jjb9707

@jjb9707 jjb9707 commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds a per-crate line-coverage gate to .github/workflows/contracts.yml: the build fails when any crate under onchain/contracts/ drops below 95% line coverage.

Changes

.github/workflows/contracts.yml

  • Adds llvm-tools-preview to the Rust toolchain components (required by cargo-llvm-cov for instrumentation)
  • Installs cargo-llvm-cov via taiki-e/install-action (pre-built binary, faster than cargo install)
  • Adds a Per-crate coverage gate (fail under 95% lines) step: the workspace is instrumented once with cargo llvm-cov --workspace --json, then line counts are attributed back to each crate and every crate is checked against the threshold individually
  • Prints a per-crate table sorted worst-first to both the job log and the job summary, and on failure emits a ::error:: annotation naming how many crates are below the bar
  • The threshold lives in the COVERAGE_THRESHOLD env var on that step

docs/ci.md — documents the gate, the exact commands to reproduce the CI numbers locally, the extra prerequisites, and a troubleshooting entry.

Why not --fail-under-lines on its own

cargo llvm-cov --workspace --fail-under-lines 95 only enforces a workspace-wide average, so a single uncovered crate can hide behind well-covered ones. This issue asks for a per-crate bar, so instrumentation is collected once (one test run, keeping the gate cheap) and then attributed per crate. Files under a tests/ or src/tests/ directory are excluded, so test code cannot inflate a crate's own score.

Testing

  • The gate's aggregation and pass/fail logic were exercised end-to-end against cargo llvm-cov JSON-export fixtures covering: src/tests/ exclusion, crate-root tests/ exclusion, multi-file crates aggregated per crate rather than judged per file, a crate exactly on the threshold (95.00 -> pass), and a crate below it (-> below 95% row plus non-zero exit).
  • CI-config change only; no contract code is modified.

Note on the current red check

The contracts job stops earlier, at the pre-existing cargo fmt --all -- --check step. That step checks the whole repository, and reports the same 143 violations across 39 files on main as it does on this branch, so the failure is not introduced here and the coverage-gate step is never reached. Once formatting is resolved on main, this gate will run.

Closes #1120

jjb9707 and others added 5 commits July 2, 2026 15:32
Adds cargo-llvm-cov to the Contracts CI workflow with:
- llvm-tools-preview component for code instrumentation
- taiki-e/install-action to install cargo-llvm-cov (pre-built binary)
- --fail-under-lines 95 per-crate coverage gate

Closes Stellopay#1120
The gate added earlier ran `cargo llvm-cov --workspace --fail-under-lines 95`,
which only enforces a workspace-wide *average*. Issue Stellopay#1120 asks for a per-crate
gate ("fail the job if any crate's line coverage is below 95 percent, with a
clear per-crate summary in the job output"), and with 29 crates under
onchain/contracts/ an average lets one uncovered crate hide behind well-covered
ones.

Instrumentation is still collected in a single workspace run so the gate stays
cheap, then attributed back to each crate from the llvm-cov JSON export. Files
under a crate's tests/ directory or a src/tests/ module are excluded so test
code cannot inflate a crate towards 100%. The step prints a per-crate table to
the job log and to $GITHUB_STEP_SUMMARY, and fails listing every crate below the
threshold. The threshold is a COVERAGE_THRESHOLD env var on the step.

Also documents the gate in docs/ci.md, whose "Coverage thresholds" section still
claimed no minimum was enforced, including how to reproduce the per-crate
numbers locally. Corrects the toolchain line in the same doc, which still said
"stable" after the workflow moved to nightly.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add a per-crate 95 percent code-coverage gate to .github/workflows/contracts.yml

1 participant