test(scripts): fail loudly when REQUIRED_CONTEXTS drifts from ci.yml's job names - #7819
Conversation
…s job names
check-stuck-required-checks.mjs's REQUIRED_CONTEXTS is hardcoded (branch
protection's required-checks list needs Administration read, which the ephemeral
workflow token can't get), so a workflow-side rename of the required aggregate
check without a matching edit here silently blinds the stuck-check watchdog to it.
- Add a test that parses .github/workflows/ci.yml's job names and asserts every
workflow-sourced entry in REQUIRED_CONTEXTS still names a real job, plus that
the single documented required aggregate ("validate") stays listed. Deriving
which checks are required from YAML alone isn't possible (same permissions
limitation), so this guarantees forward consistency and fails loudly on the
drift it can detect (verified: injecting a bogus required context fails it).
- Move the "which required checks are external / not workflow-declared" knowledge
(currently "Superagent Security Scan", a third-party App check) into the script
as an exported, documented EXTERNAL_REQUIRED_CHECKS constant beside
REQUIRED_CONTEXTS, rather than hiding it in the test — the test imports it and
also asserts it's a subset of REQUIRED_CONTEXTS.
Closes JSONbored#7774
|
Superagent didn't find any vulnerabilities or security issues in this PR. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #7819 +/- ##
=======================================
Coverage 88.65% 88.65%
=======================================
Files 730 730
Lines 76959 76959
Branches 22787 22787
=======================================
Hits 68229 68229
Misses 7684 7684
Partials 1046 1046
Flags with carried forward coverage won't be shown. Click here to find out more. |
|
Tip ✅ LoopOver review result - approve/merge recommendedReview updated: 2026-07-21 13:59:18 UTC
Review summary Nits — 4 non-blocking
Decision drivers
Context & advisory signals — never blocks the verdict
Linked issue satisfactionAddressed Review context
Contributor next steps
Signal definitions
🧪 Chat with LoopOverAsk LoopOver a question about this PR directly in a comment — grounded only in the same cached, public-safe facts shown above, never a new claim.
Full command reference: https://loopover.ai/docs/loopover-commands 🧪 Experimental — new and may change. 🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed 💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →. Checked by LoopOver, a quiet PR intelligence layer for OSS maintainers.
|
What & why
scripts/check-stuck-required-checks.mjs'sREQUIRED_CONTEXTSis hardcoded — branch protection's required-checks list needs Administration read, which the ephemeral workflowGITHUB_TOKENcan't get (the file's own comment documents this). That makes it silent-drift-prone: if the workflow's required aggregate check is renamed without a matching manual edit here, the stuck-check watchdog (built after a real incident) goes blind to it, with no error or CI signal anywhere.Change
describeblock to the existingtest/unit/check-stuck-required-checks-script.test.tsthat parses.github/workflows/ci.yml's job names (via theyamlpackage, ascodecov-policy.test.tsdoes) and asserts every workflow-sourced entry inREQUIRED_CONTEXTSstill names a real ci.yml job, plus that the single documented required aggregate (validate) stays listed. Deriving which checks are required from YAML alone isn't possible (the same permissions limitation), so this guarantees forward consistency and fails loudly on the drift it can detect."Superagent Security Scan", a third-party GitHub App check) into the script as an exported, documentedEXTERNAL_REQUIRED_CHECKSconstant besideREQUIRED_CONTEXTS, rather than hiding it in the test. The test imports it and also asserts it's a subset ofREQUIRED_CONTEXTS.scripts/**is outside the Codecovcoverage.includeset, socodecov/patchdoes not gate this. Verified locally: all 12 tests in the file pass, roottsc --noEmitclean,git diff --checkclean,node --checkon the script passes, and injecting a bogus required context genuinely fails the test (then reverted).Closes #7774
(Supersedes #7814, which the gate auto-closed: a test-only change ran under scoped
--changedselection and produced an emptycoverage/lcov.info. Touching the script here runs the full suite, so coverage is non-empty.)