A directory that was emptied for good is not a directory that might refill #92

Workflow file for this run

	name: CI

	# The last job runs docproof on docproof. That is not a flourish: this tool's whole
	# argument is that a documentation check can be a required gate rather than an advisory
	# report, and a project making that argument while not gating on it would be making it
	# badly. `fetch-depth: 0` is there for the same reason the README says it is — without
	# history the path check cannot tell a deleted file from one that never existed, and it
	# reports that instead of guessing, which would make this job green and meaningless.

	on:
	push:
	branches: [main]
	pull_request:
	# `release.yml` runs this whole file before it publishes, rather than trusting that the
	# last run on main is still true of the tagged commit.
	workflow_call:

	jobs:
	test:
	name: Tests (${{ matrix.os }}, Python ${{ matrix.python-version }})
	runs-on: ${{ matrix.os }}
	strategy:
	fail-fast: false
	matrix:
	os: [ubuntu-latest, macos-latest, windows-latest]
	python-version: ["3.10", "3.11", "3.12", "3.13"]
	steps:
	- uses: actions/checkout@v4
	with:
	fetch-depth: 0
	- uses: actions/setup-python@v5
	with:
	python-version: ${{ matrix.python-version }}
	- run: pip install -e ".[dev]"
	# The fixtures build real git repositories and commit to them, so git needs an
	# identity even though nobody will ever read these commits.
	- name: Give git an identity for the fixtures
	run: \|
	git config --global user.email "ci@example.invalid"
	git config --global user.name "docproof CI"
	- run: python -m pytest

	lint:
	name: Lint and types
	runs-on: ubuntu-latest
	steps:
	- uses: actions/checkout@v4
	- uses: actions/setup-python@v5
	with:
	python-version: "3.12"
	- run: pip install -e ".[dev]"
	- run: python -m ruff check .
	- run: python -m ruff format --check .
	- run: python -m mypy

	package:
	name: The built artifact, installed and run
	runs-on: ubuntu-latest
	# Every other job installs `-e .` and runs out of the source tree, so none of them can
	# see a packaging mistake. The first sdist built for this project was 2.2 MB and 549
	# files: 525 of them were a virtualenv that happened to be sitting in the working
	# directory under a name `.gitignore` did not match. The code was fine and the artifact
	# was junk, and nothing in CI would have said so. This job builds, installs the wheel
	# into an environment that has never seen the source, and runs the console script.
	steps:
	- uses: actions/checkout@v4
	with:
	fetch-depth: 0
	- uses: actions/setup-python@v5
	with:
	python-version: "3.12"
	- run: pip install build
	- run: python -m build

	- name: The sdist contains what it is supposed to and nothing else
	run: \|
	python - <<'PY'
	import tarfile, pathlib
	src = {p.as_posix() for p in pathlib.Path("src").rglob("*.py")}
	names = tarfile.open(next(pathlib.Path("dist").glob("*.tar.gz"))).getnames()
	inner = {n.split("/", 1)[1] for n in names if "/" in n}
	missing = src - inner
	assert not missing, f"sdist is missing source files: {sorted(missing)}"
	stray = {n for n in inner if n.startswith((".venv", "dist/", "build/", ".git/"))}
	assert not stray, f"sdist picked up files that are not the project: {sorted(stray)}"
	print(f"sdist: {len(inner)} entries, all accounted for")
	PY

	- name: Install the wheel where the source is not importable
	run: \|
	python -m venv /tmp/clean
	/tmp/clean/bin/pip install dist/*.whl
	# From `/`, so a stray `src/` on the path cannot be what answers the import.
	cd / && /tmp/clean/bin/python -c "import docproof; print(docproof.__file__)"

	- name: The installed console script runs against a real repository
	run: \|
	git config --global user.email "ci@example.invalid"
	git config --global user.name "docproof CI"
	# Its own checkout: a real project with real history and a known answer.
	/tmp/clean/bin/docproof --show-skips

	self:
	# Through `uses: ./` rather than `pip install -e .`, so this job exercises the exact
	# three lines the README tells a stranger to paste. The README carried
	# `pipx run docproof` for weeks and it never worked once, because nothing was checking
	# that the install instructions ran — which is the same defect class this tool exists
	# to find, in this tool's own shopfront.
	#
	# Three OSes because `action.yml` claims to handle all three (a Windows venv puts its
	# interpreter in `Scripts/`, not `bin/`), and a claim nobody runs is a guess.
	name: docproof on docproof, through the action (${{ matrix.os }})
	runs-on: ${{ matrix.os }}
	strategy:
	fail-fast: false
	matrix:
	os: [ubuntu-latest, macos-latest, windows-latest]
	steps:
	- uses: actions/checkout@v4
	with:
	fetch-depth: 0
	- uses: ./
	with:
	show-skips: 'true'

	action-refuses-a-truncated-clone:
	# The guard is the only reason the action is safe to recommend, so it is tested against
	# a checkout known to be shallow rather than assumed.
	#
	# Measured, on this repository's own corpus: `pallets/click` full clone reports
	# `1 broken` and exits 1; the same repository cloned at `--depth 1` reports "Nothing
	# contradicted" and exits 0. `actions/checkout` defaults to depth 1, so without
	# this guard the DEFAULT experience of adding docproof to CI is a permanently green
	# check that has judged nothing. That is the failure this project keeps meeting from
	# the other side, and it would have shipped it.
	name: The action refuses a truncated clone
	runs-on: ubuntu-latest
	steps:
	- uses: actions/checkout@v4 # depth 1, deliberately: this is the case under test
	- id: shallow
	continue-on-error: true
	uses: ./
	- name: It has to have failed
	run: \|
	if [ "${{ steps.shallow.outcome }}" != "failure" ]; then
	echo "::error title=The shallow guard did not fire::docproof ran on a depth-1"\
	"checkout and reported success. A green run that judged nothing is the"\
	"one outcome this action must never produce."
	exit 1
	fi
	echo "The guard fired, as it must."

	action-fails-a-build-that-has-drift:
	# The gate's entire purpose, and nothing tested it. The `self` job runs the action
	# against this repository, which is clean, so it proves the action RUNS and proves
	# nothing about what it does when it finds something. A regression that made the CLI
	# exit 0 on a broken claim, or a stray `\|\| true` in the run step, would have left every
	# job in this file green while the gate stopped gating.
	#
	# That is the same failure this project exists to catch, one level up: an instrument
	# reporting success while unable to see. The shallow-clone job is its sibling.
	#
	# The fixture is real drift and not a stub: a README that was TRUE when committed, and a
	# LATER commit that deleted the file it names. That ordering is the whole difference
	# between drift and an illustration, and a fixture that gets it wrong tests the wrong rule.
	name: The action fails a build that has drift
	runs-on: ubuntu-latest
	steps:
	- uses: actions/checkout@v4
	with:
	fetch-depth: 0

	- name: A repository whose README names a file a later commit deleted
	run: \|
	set -e
	mkdir -p "${RUNNER_TEMP}/drifted/tests"
	cd "${RUNNER_TEMP}/drifted"
	git init -q
	git config user.email "ci@example.invalid"
	git config user.name "docproof CI"
	echo "def test_thing(): pass" > tests/thing.py
	printf 'See `tests/thing.py` for the suite.\n' > README.md
	git add -A
	git commit -q -m "the suite, and a README that names it"
	git rm -q tests/thing.py
	git commit -q -m "remove the suite, leaving the README claiming it"

	- id: drift
	continue-on-error: true
	uses: ./
	with:
	path: ${{ runner.temp }}/drifted

	- name: It has to have failed
	run: \|
	if [ "${{ steps.drift.outcome }}" != "failure" ]; then
	echo "::error title=The gate did not fail::docproof found drift and the action"\
	"still reported success. An action that cannot fail a build is decorative,"\
	"and every other job here would stay green while it stopped gating."
	exit 1
	fi
	echo "The gate failed the build, as it must."

	action-can-report-without-failing:
	# The adoption mode, through the ACTION rather than the CLI. Three unit tests cover
	# `--exit-zero`; none of them touches the input plumbing in `action.yml`, so a typo in
	# the input name would leave `fail-on-findings: false` silently doing nothing and the
	# first person to adopt this on a drifted repository would get a red build anyway.
	#
	# That is the failure mode this whole project is about, in the one feature whose entire
	# purpose is to not fail.
	name: The action can report without failing
	runs-on: ubuntu-latest
	steps:
	- uses: actions/checkout@v4
	with:
	fetch-depth: 0

	- name: The same drifted repository as the job above
	run: \|
	set -e
	mkdir -p "${RUNNER_TEMP}/adopt/tests"
	cd "${RUNNER_TEMP}/adopt"
	git init -q
	git config user.email "ci@example.invalid"
	git config user.name "docproof CI"
	echo "def test_thing(): pass" > tests/thing.py
	printf 'See `tests/thing.py` for the suite.\n' > README.md
	git add -A
	git commit -q -m "the suite, and a README that names it"
	git rm -q tests/thing.py
	git commit -q -m "remove the suite, leaving the README claiming it"

	- id: adopt
	continue-on-error: true
	uses: ./
	with:
	path: ${{ runner.temp }}/adopt
	fail-on-findings: 'false'

	- name: It has to have SUCCEEDED, on a repository that has drift
	run: \|
	if [ "${{ steps.adopt.outcome }}" != "success" ]; then
	echo "::error title=fail-on-findings did not reach the CLI::The action failed a"\
	"run it was told not to fail. Adoption on a project that already has drift"\
	"is the one thing this input exists for, and it is wired through action.yml,"\
	"which no unit test can see."
	exit 1
	fi
	echo "Reported without failing, as it must."

Navigation Menu

A directory that was emptied for good is not a directory that might refill #92

Workflow file

A directory that was emptied for good is not a directory that might refill #92

Uh oh!

Workflow file for this run