Skip to content

[Tech Debt] CI does not check documentation, so contract docs drift silently #11

Description

@charles-fineman

Context:
Identified during work on #7
PR: #8

Description:

.github/workflows/ci.yml runs formatting, a dependency-free module check, go vet, race-enabled tests, and the fixture self-checks. It does not look at documentation at all. This project treats docs/*.md, README.md, and AGENTS.md as versioned public contracts, so nothing currently defends them.

Three drifts survived multiple merges as a direct result, all found by reading rather than by a gate:

Drift How long
AGENTS.md referenced .github/workflows-staged/, removed in ee1d529 — including naming workflows-staged/ci.yml the authoritative source for the self-check command list, a path that no longer existed until #8
CHANGELOG.md still read ## 0.1.0 - Unreleased after v0.1.0 was tagged, released, and had binaries published — contrary to step 1 of this project's own release process in docs/versioning.md ~3 weeks, until #8
docs/schema.md linked cli.md#matrix-compare-..., a dead anchor left over from the matrixtracedoc rename in fd550ce until #8

The AGENTS.md case is the sharpest: its own tie-breaker rule ("if the two diverge, CI is correct") pointed at a file that had ceased to exist, so following the documented procedure led nowhere.

Rationale:

Every one of these is mechanically detectable and none needs a new dependency. A working link-and-anchor checker was written ad hoc while auditing #8 — roughly 25 lines of Python that parses Markdown headings into GitHub-style slugs, walks every [...](...) target, and reports missing files and dead anchors. It found all remaining link problems in one pass and can be lifted into a CI step or a small Go test.

Suggested checks, cheapest first:

  • Dead internal links and anchors. Every relative Markdown link resolves to an existing file, and every #fragment matches a real heading in the target.
  • Repository paths named in prose exist. Catches the workflows-staged class directly: extract backticked path-looking strings from AGENTS.md/README.md/docs/ and assert each exists.
  • Changelog release state. On a tag build, fail if CHANGELOG.md has no dated section matching the tag. The release workflow already verifies toolVersion against the tag (.github/workflows/release.yml), so this is the same idea applied to the changelog.
  • AGENTS.md self-check commands match CI. AGENTS.md declares the CI step authoritative when the two lists diverge; assert they are actually identical rather than trusting the tie-breaker.

Out of scope:

Prose linting, spelling, or style enforcement. This is about claims that are verifiably false, not about writing quality.

Acceptance Criteria:

  • CI fails on a dead internal link or anchor in any tracked Markdown file
  • CI fails when a repository path named in AGENTS.md, README.md, or docs/ does not exist
  • The check runs without adding a module dependency (go vet ./... and the dependency-free module check must still pass)
  • Each of the three drifts in the table above is reproduced as a failing case before the fix, so the check is known to catch them
  • AGENTS.md documents how to run the check locally, alongside the existing "Validation" commands

Metadata

Metadata

Labels

documentationImprovements or additions to documentation

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions