feat: fleet visibility — gate outcomes readable across installed repos

[MCKRUZ]

Closes #12.

Stacked PR. Base is feat/dependency-axis so the diff shows only this change. Retarget to main once #39 and #40 merge.

Overrides and gate outcomes were already recorded per repo — the accepted-risk labels, the two ledgers, the PR timeline. Nothing aggregated them, so across a portfolio nobody could answer whether the gates are firing, being routinely waved through, or quietly switched off.

The third question is the one that needed building

A gate has two halves: the check, and the rule requiring it to pass. Remove the rule and the check still runs, still reports, and looks entirely normal on the pull request — a red run simply merges anyway.

From outside that repo, a disarmed gate and a gate that never caught anything produce identical evidence. No amount of counting separates them. check_standard.py catches this in our repo; nothing caught it in a client's.

So each repo commits a weekly rails-telemetry.json with three sections — what ran, every override by name, and the comparison of what branch protection requires against what the workflows declare.

Two constraints held throughout

• Stays inside the client's tenancy. The workflow reads the repo's own history through the platform's own API and writes into the same repo. No external calls — that is what makes it approvable rather than a negotiation.
• Counts gates, never people. Overrides are reported against merged changes so they read as a rate; there is no per-author breakdown anywhere in the file. GOLD-STANDARD §9 forbids activity metrics, and this measures the rails, not the humans.

Where live branch protection can't be read it falls back to the committed ruleset and records which, because that reading describes intent rather than what the platform enforces. Presenting the two as equivalent would be the same silent-green failure one layer up.

The schema ships fixed at version 1 before the install wave — a schema retrofitted across live repos is a migration nobody schedules. That was the sequencing constraint in #12.

The collector

scripts/collect_rails_telemetry.py is operator tooling — deliberately not part of kit/, never installed into a client repo. Reads the committed files worst-first; exits non-zero on a high finding or an unreadable repo, because silence is not health.

Two things that shaped it:

• Discovery is keyed on .git, not on the report file. Keyed on the report it would only ever find repos already reporting, so one where the workflow was never installed would be silently absent — the exact failure being built against, reproduced inside the tool meant to catch it.
• An empty --root is a finding with its own message, not the usage error argparse gives for a missing flag. Sending someone to re-type a flag they already typed is how a tool earns a reputation for being broken.

Test plan

• python -m pytest scripts/tests -q — 65 passing (19 new). Tests inject real drift and assert it is caught, following test_check_standard.py: a checker that has stopped noticing is worse than no checker.
• The assembling jq was exercised against synthetic data before shipping, including the disarmed-gate case, with assertions on each finding kind
• python scripts/check_standard.py — no drift
• End-to-end run against a fabricated three-repo fleet: ranks the disarmed repo first, lists the silent repo as unknown, exits 1

🤖 Generated with Claude Code