Skip to content

CI_CD.md

Latest commit

 

History

History
191 lines (152 loc) · 11.5 KB

CI_CD.md

File metadata and controls

191 lines (152 loc) · 11.5 KB
Raw

CI/CD pipeline

This document describes the continuous-integration and continuous-delivery chain of method-kit: what each piece does, how the protections fit together, and — importantly — what travels with a fork and what does not.

It complements CONTRIBUTING.md, which covers the day-to-day contributor workflow. This document is the why; CONTRIBUTING.md is the how.

1. Overview

The chain is a gated pull-request flow. Nothing reaches main except through a pull request that passes every required gate. On top of the gates, detection and maintenance features run continuously, and the release runs on a version tag.

flowchart TD
    PR["Pull request to main"]
    PR --> Q["quality<br/>ruff &middot; compile · skills"]
    PR --> T["tests<br/>auto-discovered module floors"]
    PR --> S["security<br/>Bandit SAST"]
    PR --> DP["docs-privacy<br/>private-marker scan over docs/**"]
    PR --> CQ["Analyze (python)<br/>CodeQL security-extended"]
    Q --> G{"All required checks green?<br/>branch up to date?<br/>no CodeQL alert &ge; High / Errors?"}
    T --> G
    S --> G
    DP --> G
    CQ --> G
    G -- yes --> M(["Merge to main"])
    G -- no --> X(["Merge blocked"])
Loading

The enforcement has two layers:

  • Layer A — files in the repository (travel with a fork): the workflows (including the action-drift watcher), and the tooling configuration in pyproject.toml.
  • Layer B — repository settings (do not travel with a fork): branch protection, the code-scanning ruleset, and the Advanced Security toggles.

Both layers are required for the full posture. A fork inherits Layer A by copying the branch but gets none of Layer B — until it is recreated, the workflows still run, but nothing blocks a merge. Section 6 lists exactly what a fork must reconfigure.

2. Workflows (Layer A — in .github/workflows/)

Workflow Trigger Job (check name) What it enforces
ci.yml PR to main, push to main quality ruff check → ruff format --check → doc-drift (tracked docs — scripts/gates/doc_drift_targets.tsv) → compileall -q skills
tests.yml PR to main, push to main tests the auto-discovered module floors + pytest tests/ (exit 0/1); the first non-zero exit fails the check
security.yml PR to main, push to main security Bandit SAST: bandit -c pyproject.toml -r .
docs-privacy.yml PR to main, push to main docs-privacy private-marker scan over docs/**: bash scripts/gates/docs-privacy.sh
codeql.yml PR + push + weekly (Mon 06:00 UTC) Analyze (python) CodeQL security-extended — the analysis runs (content gate is the Layer-B ruleset, below)
release.yml push of a tag v* release package the .skill deliverable + attach it to a Release; a version guard aborts if the tag ≠ the artifact's metadata.version
action-drift.yml weekly (Mon 06:00 UTC) + manual dispatch drift (detector, not a gate) re-checks the SHA-pinned action versions against upstream tags; opens an issue on drift — never edits a workflow, never blocks a merge

Design notes:

  • Quality, security, and test tools are pip-installed and version-pinned via PEP 735 dependency-groups (ruff==0.15.18, bandit[toml]==1.9.4, pytest==9.1.1, defusedxml==0.7.1), not run via a marketplace action — deterministic, with exact CI↔local parity and no extra action to maintain.
  • CodeQL uses advanced setup (a committed workflow), not GitHub's default setup — the two are mutually exclusive. security-extended is broader than the default suite.
  • Scan scope: the whole repository (.venv excluded by the ruff config). compileall targets skills/ (the source directory). The test gate auto-discovers the module floors and also runs pytest tests/. No cross-repo pinned checkout is used.

3. Tooling configuration (Layer A — pyproject.toml, single source of truth)

  • [tool.ruff] — target-version = "py312", line-length = 100, lint rules E/F/I/UP/B, double-quote format, extend-exclude = [".venv"].
  • [tool.bandit] — read with -c pyproject.toml (the [toml] extra). Skips are deliberate and documented, not blanket silencing — the active set is B101 B404 B406 B603 B607:
    • B404 (import subprocess) — informational; the module is used in safe form.
    • B603 (subprocess without shell=True) — the safe form: a fixed argument list, never a shell string, no user-controlled tokens.
    • B607 (partial path) — processes invoked by name on a trusted dev PATH.
    • B406 (xml.sax.saxutils.escape) — escapes text for safe XML output in gen_diagram; it parses nothing. A Bandit false positive, and the only XML skip that remains.
    • The dangerous case, B602 (shell=True), stays active.
    • The xml.etree parse channel is gone: check_diagram now parses with defusedxml (retire the channel, METHOD §6), so the former B405/B314 skips were removed — there is nothing left to flag. defusedxml is a runtime dependency of the diagram checker (declared in skills/project-pilot/modules/diagram-generation/requirements.txt) and is installed by the tests gate.
  • [tool.pytest.ini_options] — testpaths = ["tests"]; the tests gate runs the auto-discovered module floors and then pytest tests/.
  • Dependency story — the toolchain is declared in PEP 735 dependency-groups and pinned; tzdata is pulled on Windows. There is no top-level requirements.txt; a module may declare its own runtime dependency in a module requirements.txt (defusedxml for the diagram checker, tzdata for filename stamping). None of these is a pinned pip manifest, so this repository ships no Dependabot version-updates configuration at all (§4) — GitHub Actions are SHA-pinned and watched by action-drift, not tracked by Dependabot.

4. Action pinning & drift (Layer A — .github/workflows/, action-drift.yml)

Workflow actions are pinned to full commit SHAs, not floating tags — a version comment records the human-readable tag (e.g. actions/checkout@9c091bb… # v7.0.0). Pinning by SHA closes the mutable-tag supply-chain channel: a floating @v4 lets the upstream rewrite what the runner executes, a SHA freezes it (retire the channel, METHOD §6).

Because a SHA never moves on its own, a pinned action can silently fall behind upstream. The action-drift.yml workflow is the watcher: weekly (Mon 06:00 UTC) and on manual dispatch, it runs scripts/check_action_drift.py over .github/workflows/ and, on drift, opens an issue — it is a detector, not a gate (METHOD §8): it never edits a workflow, never blocks a merge, and its job stays green so the signal surfaces in the Issues tab. Re-pinning is manual and deliberate (resolve the new tag, update the SHA), never an auto-refresh.

This repository ships no Dependabot configuration: action versions are managed by SHA pinning plus the action-drift watcher above, not by automated tag bumps.

5. Repository settings (Layer B — NOT in files, do not travel with a fork)

Configured via gh api (METHOD §7/§10) and recreated in any fork that wants the same protection.

Branch protection on main: require a PR; require status checks

`quality, tests, security, docs-privacy, Analyze (python)`; require branches up to date; block force

pushes; restrict deletions; do not allow bypassing (admins included).

Code-scanning ruleset (targeting main): require code-scanning results, tool CodeQL, threshold security alerts ≥ High or higher, alerts = Errors, mode Active. This is distinct from the Analyze (python) status check: the check verifies the analysis ran; the ruleset blocks the merge when alerts exist above the threshold. The status check gates on the job; the ruleset gates on the finding.

Advanced Security: secret scanning + push protection on; Dependabot alerts + security updates on.

Honest current state of this repository. Layer B is armed in files but not active. The ruleset and branch-protection setup live in .github/rulesets/main.json and scripts/setup-layer-b.sh, ready to apply, but the GitHub API rejects them on a free private repository (HTTP 403 — "upgrade to Pro or make the repository public"). So the workflows run and are green, but no merge is currently blocked — the gated-PR rule is held by discipline. Likewise, CodeQL analyses every PR (nominal), but its SARIF upload is blocked on the free private tier, so the codeql run shows red while the analysis itself is fine. Activating Layer B requires GitHub Pro or making the repository public.

6. Forking your own version

A fork copies all of Layer A and none of Layer B. To get the same posture:

  1. Enable Actions (GitHub disables workflows on new forks).
  2. Public vs private fork. Code scanning, secret scanning, and push protection are free on public repositories; on a private fork they need GitHub's paid code-security features (this is exactly the 403 above).
  3. Enable Advanced Security (secret scanning + push protection; Dependabot alerts + security updates). CodeQL is picked up from codeql.yml on first run — do not also enable default setup.
  4. Recreate branch protection with the required checks and "require branches up to date".
  5. Recreate the code-scanning ruleset (CodeQL, High+/Errors, Active) so alerts block merges. scripts/setup-layer-b.sh does steps 4–5 via gh api.

Until steps 3–5 are done, the workflows still run (you can read their results), but nothing blocks a merge — the blocking comes from Layer B.

7. Security posture summary (defense in depth)

Control Layer Blocks
quality required check B (branch) broken lint / format / compile
tests required check B (branch) a failing skill floor
security required check B (branch) a Bandit finding
docs-privacy required check B (branch) a private-marker leak into docs/**
Analyze (python) required check B (branch) a CodeQL analysis failure (the run, not an alert)
Code-scanning ruleset (CodeQL ≥ High / Errors) B (ruleset) a CodeQL alert (the finding)
Secret scanning + push protection B (Adv. Security) a secret at push time
Dependabot alerts / security updates B (Adv. Security) a vulnerable dependency
action-drift watcher A (action-drift.yml) (detector) stale action SHAs — opens an issue, does not block

Caveats worth knowing:

  • Layer B is not active on this repository's tier (see §5) — the table above describes the intended posture; today only the Layer-A workflows run.
  • release.yml is only exercised on a tag push — a green PR does not prove the release still works. Validate it at the next v* tag.
  • Action pins are full commit SHAs with a version comment (e.g. actions/checkout@9c091bb… # v7.0.0, actions/setup-python@a309ff8b… # v6.2.0, github/codeql-action@8aad20d… # v4.36.2) — frozen execution, not floating tags. The action-drift watcher (§4) opens an issue when a pin falls behind upstream; re-pin by hand.