Skip to content

Correct stale documentation claims in README and CLAUDE.md - #74

Merged
depalmar merged 2 commits into
mainfrom
docs/refresh-stale-claims
Aug 21, 2026
Merged

depalmar merged 2 commits into
mainfrom
docs/refresh-stale-claims

Conversation

@depalmar

Copy link
Copy Markdown
Owner

Two commits, both correcting documentation that had drifted from the repository.

Front page (README.md)

Five claims were wrong. All five sat in hand-written prose — the generated blocks stayed accurate, because readme_counts.py --check gates them in CI and nothing gates the rest.

Claim Was Now
OWASP reference 2025 list 2026 list, with a note that eight of ten IDs changed meaning
Published site unmentioned linked at the top, with what is actually on it
Test suite "7 passes" 15 passes, plus why it hard-exits on missing yara
Exports ForensicArtifacts only all four — ForensicArtifacts, KAPE, Velociraptor, CSV/JSON
Catalog section no scale, no case studies eleven categories, case studies, contested field

Also added docs/ to the Structure tree, which the guide is linked from but was missing.

Counts are deliberately not duplicated into the root README. readme_counts.py is hardwired to artifacts/README.md; any number typed into the root file would be ungated and would drift, which is how two of the five above were created. The section points at the gated table instead.

Project context (CLAUDE.md)

Six figures had drifted. This file loads into every session, so a stale number here misdirects the next session's work rather than sitting quietly in an unopened file.

  • 340 artifacts / 158 credential / 61 MCP → 615 / 164 / 64
  • volatility recomputed over 615 rows, not 559
  • confidence 28-19-4 → 27-20-4; risk 11-24-12-2 → 11-27-12-1
  • aliases 28/51 → 30/51; last_verified 42/51 → 50/51
  • "13 entries are medium" → 20

The provenance paragraph was the costly one: it described nine entries as never verified when eight had since been checked, and the priority list downstream was written against that number. AIRT-0007 Devin is now the sole holdout, and verification_worklist.py annotates it cloud-hosted with no endpoint paths — so that count is structural, not a backlog.

Recorded why the old "could not be checked through a repository API" framing must not return: it presented an unworked backlog as a technical limit, and the entries it covered turned out to be ordinary installable software.

Copied figures are now labelled as convenience copies pointing at readme_counts.py and verification_worklist.py.

Verification

validate.py            51 entries + 12 sigma rules, 0 problems
readme_counts.py       counts are current
build_site.py --check  615 rows, 615 unique anchors, 51 tools, 68 rules, 0 problems
validate_mappings.py   68 on disk = 68 in MAPPINGS, in sync

No catalog data, schema, or detection content touched — documentation only.

🤖 Generated with Claude Code

depalmar and others added 2 commits August 21, 2026 10:34
The generated blocks stayed accurate while the hand-written prose drifted.
readme_counts.py gates artifacts/README.md and reports current; every stale
claim fixed here is in a section no script owns.

- OWASP reference said 2025. The detection content was remapped to the 2026
  list, where eight of the ten IDs changed meaning, so the front page was
  citing a list that names different categories than the rules do.
- The published site was never mentioned. pages.yml has been deploying it on
  every catalog push and only artifacts/README.md linked it, so a visitor
  arriving at the repository root could not find it.
- The test suite claim said 7 passes. validate.sh now carries 15 assertions,
  none skippable. Documented why it hard-exits on a missing yara binary, since
  that design choice is the reason the count can be trusted at all.
- Exports listed ForensicArtifacts only. There are four exporters; KAPE and
  Velociraptor both ship feeds under docs/api/.
- The catalog section carried no scale and omitted case studies.

Counts are deliberately not duplicated here. readme_counts.py is hardwired to
artifacts/README.md, so a number typed into this file would be ungated and
would drift - which is how two of the above were created. This points at the
gated table instead.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
CLAUDE.md loads into every session, so a stale figure here does not sit
quietly in a file nobody opens - it misdirects the next session's work. The
provenance paragraph was the expensive one: it described nine entries as never
verified when eight had since been checked, and the priority list downstream
was written against that wrong number.

- 340 artifacts, 158 credential, 61 MCP -> 615 / 164 / 64
- volatility recomputed over 615 rows, not 559
- confidence 28-19-4 -> 27-20-4, risk 11-24-12-2 -> 11-27-12-1
- aliases 28/51 -> 30/51, last_verified 42/51 -> 50/51
- "13 entries are medium" -> 20

AIRT-0007 Devin is now the sole never-verified entry, and the worklist
annotates it cloud-hosted with no endpoint paths, so that count is structural
rather than a backlog. Recorded that, and recorded why the old "could not be
checked through a repository API" framing must not come back: it dressed up an
unworked backlog as a technical limit, and the entries it covered turned out
to be ordinary installable software.

Marked the copied figures as convenience copies pointing at readme_counts.py
and verification_worklist.py, which are the gated sources.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@depalmar
depalmar merged commit 7379254 into main Aug 21, 2026
4 checks passed
@depalmar
depalmar deleted the docs/refresh-stale-claims branch August 21, 2026 15:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant