docs: add a doc/code audit, fix the one dead reference, document nine nodes - #1718
Open
ptlnextdoor wants to merge 11 commits into
Open
docs: add a doc/code audit, fix the one dead reference, document nine nodes#1718ptlnextdoor wants to merge 11 commits into
ptlnextdoor wants to merge 11 commits into
Conversation
Adds a bidirectional documentation audit so doc cleanup is a review task
rather than an archaeology task after several rounds of renaming.
doc -> code: every path a doc cites is classified with the evidence behind
the verdict, not checked as a boolean. A naive "is this path on disk" test
calls 68% of this repo's citations dead; almost all of that is false, and
acting on it deletes correct docs. Files the reader is told to create,
names the code builds at runtime, changelog entries about deleted files,
and counter-examples ("NOT: `.pipeline.json`") all look identical to rot
under an exists-check. Only ORPHANED -- no path, no basename, no source
literal -- is a deletion candidate, and it is reported, never auto-applied.
code -> doc: nodes shipping Python with no README, READMEs with no
generated params block, and generated blocks that disagree with
services*.json. Profile groupings in `fields` (object/properties rather
than type) are excluded, since nodes:docs-generate omits them and counting
them reports phantom drift on every profile-based node.
Current state of this tree: 381 citations -> 380 accounted for, 1 genuine
dead reference; 14 code->doc gaps.
Every test named test_placeholder_*, test_counter_example_*, or
test_profile_groups_* pins a false positive an earlier revision produced.
Verified: pytest 16 passed; ruff check + format clean.
detect_segment/services.json and video_composer/services.json ship `//` comments. json.loads raises on those, and the except branch treated the node as declaring zero parameters -- so drift in any commented schema could never surface. Strips `//` line comments with quote-state tracking so a `https://` inside a string value survives. Unmasks 14 further drifted nodes (STALE_PARAMS 4 -> 18), including graph_falkordb, whose README still documents the pre-rename `tool_falkordb.*` keys. Verified: pytest 18 passed; ruff check + format clean.
README-n8n.md pointed readers at `.context/n8n-test/` for `run.sh` and a WALKTHROUGH.md. None of that exists in the tracked tree and `.context/` is not gitignored, so no contributor can follow it. Points at the real `examples/n8n/` pipes and the importable dispatch workflow instead. Found by tools/docs_audit -- the only ORPHANED citation of 385.
background_removal, caption, depth_estimate, detect, detect_segment, face_detection, pose_estimation, tool_google_workspace, and video_composer each ship Python but had no README, contrary to the co-located documentation rule in AGENTS.md. Prose is grounded in each node's own services*.json description, lanes, and field set -- not invented. tool_google_workspace additionally spells out that every destructive capability (hard delete, public sharing) is a separate opt-in flag defaulting to false. The ROCKETRIDE:GENERATED:PARAMS block is deliberately NOT hand-written: it is produced by nodes:docs-generate, which needs the engine build. These nine therefore move from MISSING_DOC to MISSING_PARAMS until a maintainer regenerates.
ptlnextdoor
requested review from
Rod-Christensen,
jmaionchi and
stepmikhaylov
as code owners
July 29, 2026 03:40
Contributor
🤖 Internal: Discord sync markerAuto-managed by the Discord notification workflow. Stores the linked Discord message ID and forum thread ID. Do not edit or delete. |
Contributor
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughThe pull request adds a documentation audit tool with citation and schema-coverage checks, tests and CLI reporting, nine node README files, and updated n8n workflow guidance. ChangesDocumentation audit and guides
Estimated code review effort: 3 (Moderate) | ~25 minutes Possibly related issues
Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 7
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@nodes/src/nodes/caption/README.md`:
- Around line 8-9: Update the caption node README’s task documentation so the
declared default aligns with the allowed task values: either change the default
from “caption” to one of the documented granularities or add “caption” to the
allowed options, applying the same correction to both documented occurrences.
In `@nodes/src/nodes/depth_estimate/README.md`:
- Around line 14-16: Update the depth-estimate README wording from “afterwards”
to the American-English “afterward,” without changing the surrounding
explanation.
In `@nodes/src/nodes/tool_google_workspace/README.md`:
- Around line 23-30: Revise the “Destructive operations are opt-in” section to
avoid claiming that every irreversible or externally visible capability is
disabled by default. Limit the wording and examples to deletion and
public/domain-wide sharing, or explicitly document controls for Gmail send,
Drive writes, and Docs/Sheets batchUpdate if those controls exist.
In `@tools/docs_audit/src/docs_audit/cli.py`:
- Around line 17-21: Update _docs to use os.walk instead of root.rglob, pruning
excluded directory names in place before descending while preserving
DOC_SUFFIXES filtering and yielding non-excluded matching files. Follow the
directory-pruning approach used by CodeIndex.build().
- Around line 39-45: Validate the resolved root in main before building the
CodeIndex or running any audits. If root does not exist or is not a directory,
reject it with a nonzero exit status and an appropriate error message; otherwise
preserve the existing audit and fail-on-orphaned behavior.
In `@tools/docs_audit/src/docs_audit/coverage.py`:
- Around line 103-106: Update the schema-loading exception path in audit_node()
so OSError or JSONDecodeError produces an explicit schema-read/parse gap instead
of continuing with no data. Ensure the returned gap is propagated through the
audit result and rendered by the CLI, preserving normal processing for
successfully parsed schemas.
In `@tools/docs_audit/src/docs_audit/index.py`:
- Around line 82-96: Preserve leading dots when constructing repository-relative
paths in the directory scan loop. Update both the
`index.paths`/`index.basenames` entries and the `index._sources` key to use the
unmodified `rel_dir / name` POSIX path, removing the `.lstrip('./')` calls so
hidden paths such as `.github/...` and `.env` remain exact.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 63e0fd3d-78bb-4c45-9f89-fb6895f70539
📒 Files selected for processing (19)
docs/README-n8n.mdnodes/src/nodes/background_removal/README.mdnodes/src/nodes/caption/README.mdnodes/src/nodes/depth_estimate/README.mdnodes/src/nodes/detect/README.mdnodes/src/nodes/detect_segment/README.mdnodes/src/nodes/face_detection/README.mdnodes/src/nodes/pose_estimation/README.mdnodes/src/nodes/tool_google_workspace/README.mdnodes/src/nodes/video_composer/README.mdtools/docs_audit/README.mdtools/docs_audit/cli.pytools/docs_audit/src/docs_audit/__init__.pytools/docs_audit/src/docs_audit/citations.pytools/docs_audit/src/docs_audit/cli.pytools/docs_audit/src/docs_audit/coverage.pytools/docs_audit/src/docs_audit/index.pytools/docs_audit/test/__init__.pytools/docs_audit/test/test_docs_audit.py
…its (rocketride-org#1718 review) The tool_google_workspace README claimed "every irreversible or externally-visible capability is gated behind its own flag and is off by default." That is wrong for the second half. allowHardDelete / allowDelete / allowPublicSharing are indeed opt-in booleans defaulting to false, but the per-service `access` field defaults to `write` on Drive, Calendar, Docs and Sheets (and `modify` on Gmail) — so creating files and calendar events is on by default, and calendar writes send attendee invitations. Reworked the section to separate the two mechanisms and say which one actually bounds the agent day to day. Gmail's `send` really is above the default, and is now called out as the exception rather than the rule. Also from review: - caption: `task` prose said "short / detailed / more detailed" while the default is `"caption"`, leaving no way to map the words to the enum. Now names the actual values: caption, detailed_caption, more_detailed_caption. - depth_estimate: "afterwards" -> "afterward" for American English. Verified: pytest 18 passed; ruff clean.
…ng (rocketride-org#1718 review) Four Major findings from review, three of which share one failure mode: the tool returned success while silently doing no work. - cli: a nonexistent --root produced no docs, no nodes, and exit 0 even under --fail-on-orphaned. A typo in a CI invocation was therefore a green build that never audited anything. Now exits 2 with a message. - coverage: an unparseable services*.json was skipped, leaving zero declared params -- indistinguishable from a node with nothing to document, so real schema drift stayed hidden behind a clean report. Added an UNREADABLE gap kind, reported instead of swallowed; same for an unreadable README. - index: `.lstrip('./')` strips a character SET, not a prefix, so `.env` was indexed as `env` and `.github/workflows/ci.yml` lost its leading dot. Citations to hidden files could be reported orphaned or matched only by basename. `Path('.') / name` is already clean, so the lstrip was both wrong and unnecessary. - cli: _docs used rglob, descending node_modules and vendored trees in full before discarding what they yielded. Now os.walk with in-place pruning, matching CodeIndex.build. Also added the __main__ guard: `python -m docs_audit.cli` previously printed nothing and exited 0, the same silent-success trap as the --root bug. Verified: 21 passed (3 new regression tests), ruff clean, format clean. Full run on this repo: 390 citations, 0 orphaned, 31 coverage gaps, unchanged from before the fix.
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
tools/docs_audit/src/docs_audit/coverage.py (1)
171-174: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick winDetect stale parameters when the schema has no user-facing fields.
After the
documented is Nonebranch,documentedis a set. Ifdeclaredis empty but the generated block contains old rows, this return reports a clean node and hides stale documentation.Remove the early return. Compare both sets so that an empty schema with documented rows produces
STALE_PARAMS, while an empty schema with an empty table remains clean.Proposed fix
- if not declared: - return [] - undocumented = declared - documented phantom = documented - declaredAdd a regression test for an empty schema with one documented parameter.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tools/docs_audit/src/docs_audit/coverage.py` around lines 171 - 174, Remove the early `if not declared` return in the relevant coverage-checking function after the `documented is None` branch, and always compare `declared` and `documented`. Preserve clean results when both are empty, but return `STALE_PARAMS` when an empty schema has documented rows; add a regression test covering that case.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@tools/docs_audit/src/docs_audit/cli.py`:
- Around line 18-28: Update the audit flow connecting _docs(), audit_doc(), and
main() so an OSError while reading a documentation file produces an explicit
unreadable finding or causes the CLI to return a non-zero status. Do not
silently omit failed files or treat them as empty results; ensure the failure is
visible in the collected audit outcome.
In `@tools/docs_audit/test/test_docs_audit.py`:
- Around line 216-227: Extend test_dotfile_paths_keep_their_leading_dot to
create a hidden .py file containing a distinctive literal, then assert
find_literal() returns evidence using its dotted path. Keep the existing
has_path() assertions and ensure the assertion exercises the _sources key used
by literal lookup.
---
Outside diff comments:
In `@tools/docs_audit/src/docs_audit/coverage.py`:
- Around line 171-174: Remove the early `if not declared` return in the relevant
coverage-checking function after the `documented is None` branch, and always
compare `declared` and `documented`. Preserve clean results when both are empty,
but return `STALE_PARAMS` when an empty schema has documented rows; add a
regression test covering that case.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 4f238b24-03f8-49ea-9f51-cc397f1a4ae7
📒 Files selected for processing (4)
tools/docs_audit/src/docs_audit/cli.pytools/docs_audit/src/docs_audit/coverage.pytools/docs_audit/src/docs_audit/index.pytools/docs_audit/test/test_docs_audit.py
Comment on lines
+216
to
+227
| def test_dotfile_paths_keep_their_leading_dot(tmp_path: Path) -> None: | ||
| """Regression: `.lstrip('./')` strips a character SET, not a prefix, so | ||
| `.env` was indexed as `env` and citations to hidden files looked orphaned. | ||
| """ | ||
| (tmp_path / '.env').write_text('K=v\n', encoding='utf-8') | ||
| (tmp_path / '.github' / 'workflows').mkdir(parents=True) | ||
| (tmp_path / '.github' / 'workflows' / 'ci.yml').write_text('on: push\n', encoding='utf-8') | ||
|
|
||
| index = CodeIndex.build(tmp_path) | ||
| assert index.has_path('.env') | ||
| assert index.has_path('.github/workflows/ci.yml') | ||
| assert not index.has_path('env') |
Contributor
There was a problem hiding this comment.
🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win
Cover hidden paths in the source-text index.
This test checks has_path() only. The change also updates the _sources key at Line 99. A regression there would pass this test and break find_literal() evidence for hidden source files.
Add a hidden .py file with a literal and assert that find_literal() returns the dotted path.
Proposed test extension
(tmp_path / '.github' / 'workflows' / 'ci.yml').write_text('on: push\n', encoding='utf-8')
+ (tmp_path / '.github' / 'engine.py').write_text('version.docker.json\n', encoding='utf-8')
index = CodeIndex.build(tmp_path)
assert index.has_path('.env')
assert index.has_path('.github/workflows/ci.yml')
assert not index.has_path('env')
+ assert index.find_literal('version.docker.json') == ('.github/engine.py', 1)📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| def test_dotfile_paths_keep_their_leading_dot(tmp_path: Path) -> None: | |
| """Regression: `.lstrip('./')` strips a character SET, not a prefix, so | |
| `.env` was indexed as `env` and citations to hidden files looked orphaned. | |
| """ | |
| (tmp_path / '.env').write_text('K=v\n', encoding='utf-8') | |
| (tmp_path / '.github' / 'workflows').mkdir(parents=True) | |
| (tmp_path / '.github' / 'workflows' / 'ci.yml').write_text('on: push\n', encoding='utf-8') | |
| index = CodeIndex.build(tmp_path) | |
| assert index.has_path('.env') | |
| assert index.has_path('.github/workflows/ci.yml') | |
| assert not index.has_path('env') | |
| def test_dotfile_paths_keep_their_leading_dot(tmp_path: Path) -> None: | |
| """Regression: `.lstrip('./')` strips a character SET, not a prefix, so | |
| `.env` was indexed as `env` and citations to hidden files looked orphaned. | |
| """ | |
| (tmp_path / '.env').write_text('K=v\n', encoding='utf-8') | |
| (tmp_path / '.github' / 'workflows').mkdir(parents=True) | |
| (tmp_path / '.github' / 'workflows' / 'ci.yml').write_text('on: push\n', encoding='utf-8') | |
| (tmp_path / '.github' / 'engine.py').write_text('version.docker.json\n', encoding='utf-8') | |
| index = CodeIndex.build(tmp_path) | |
| assert index.has_path('.env') | |
| assert index.has_path('.github/workflows/ci.yml') | |
| assert not index.has_path('env') | |
| assert index.find_literal('version.docker.json') == ('.github/engine.py', 1) |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@tools/docs_audit/test/test_docs_audit.py` around lines 216 - 227, Extend
test_dotfile_paths_keep_their_leading_dot to create a hidden .py file containing
a distinctive literal, then assert find_literal() returns evidence using its
dotted path. Keep the existing has_path() assertions and ensure the assertion
exercises the _sources key used by literal lookup.
…-drift class
_is_user_facing_param required a "type" key in addition to excluding
"object" entries. nodes:docs-generate only does the latter:
if (field && field.object !== undefined) continue; // Skip profile definitions
So a field with no "type" -- which the generator emits with an empty Type
cell -- was rendered into the table but refused by the audit as declared,
and surfaced as "in docs but not in schema".
The generator decides what the table contains, so it is the only correct
oracle for this comparison; any stricter rule invents drift.
Measured on develop @ b8068d7 after regenerating the tables: 10 of the
remaining STALE_PARAMS findings were this false positive. With the rule
aligned, STALE_PARAMS goes 10 -> 0 and the surviving 13 findings are all
genuine (9 MISSING_DOC, 4 MISSING_PARAMS).
Verified: 22 passed (1 new regression test), ruff clean, format clean.
ptlnextdoor
added a commit
to ptlnextdoor/rocketride-server
that referenced
this pull request
Aug 1, 2026
cloud_tts, landing_ai, tool_mem0 and tool_n8n were the only 4 of 112 node READMEs without ROCKETRIDE:GENERATED:PARAMS markers, so nodes:docs-generate skipped them by design and their schemas were never machine-checked. Markers appended at the end, matching where the block sits in the other 108, then generated. The generator only writes between the markers, so each file's hand-authored prose is untouched -- including the conceptual Configuration tables (e.g. cloud_tts's profile/field/env-fallback table), which explain where a setting comes from and are not something the generated field dump replaces. Audit on this branch is now: 399 citations, 0 orphaned, 9 gaps -- and all 9 are the MISSING_DOC nodes whose READMEs are written in rocketride-org#1718.
…ol on sibling repos
Ran the audit against rocketride-workshops and rocketride-benchmark. The
benchmark repo came back clean (114 citations, 0 orphaned). Workshops
reported 8 orphaned citations -- and all 8 were the tool's fault, in two
distinct ways:
- `.rocketride/docs/*` (4). The VS Code extension installs that directory
into a *user's* workspace (apps/vscode/src/agents/agent-manager.ts). A doc
telling a reader to open a file under it is correct precisely because the
repo does not contain it. Now classified RUNTIME.
- `ARCHITECTURE.md` / `OWNERSHIP.md` (4). The citing line reads "Emits
`ARCHITECTURE.md` and `OWNERSHIP.md` as inline content blocks" -- artifacts
the workshop's agent produces at run time. "emit" was missing from the
create-verb set, along with produce/output and the -s/-ed forms.
Third, found while fixing the above: classify() built its sibling path with
`.lstrip('./')`, the same character-set-vs-prefix bug already fixed in
index.py, so a citation to a dotfile was looked up with its leading dot
stripped.
Also reordered classify(): the runtime-literal lookup now runs before the
prose heuristics. Finding the name as a literal in source is hard evidence;
a create-verb nearby is a guess about intent. Both verdicts are protected so
nothing changes about what survives a cleanup, but the evidence a human
reads is strictly better. This is why "Writes `built_at_runtime.json`" is
RUNTIME again rather than PLACEHOLDER -- the existing test caught the
regression when the broader verb list shadowed it.
Verified: 24 passed (2 new regression tests), ruff clean, format clean.
rocketride-workshops orphans 8 -> 0; rocketride-server unchanged at 0.
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@tools/docs_audit/src/docs_audit/citations.py`:
- Around line 173-176: Update the citation matching in the surrounding audit
logic to pass the complete normalized token, rather than basename, to
index.find_literal before assigning RUNTIME. Add a regression case covering two
distinct paths with the same basename, ensuring a literal for one path does not
satisfy the other.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 6b56ef34-921b-4a79-8380-4e94df41598b
📒 Files selected for processing (2)
tools/docs_audit/src/docs_audit/citations.pytools/docs_audit/test/test_docs_audit.py
…— the keys were wrong The 9 node READMEs added in this PR documented their config with hand-maintained tables. Every field name in them was missing its node prefix, so they named configuration that does not exist: documented actual maxEdge background_removal.maxEdge task caption.task threshold detect.threshold crf / fps composer.crf / composer.fps video_composer is the worst case: its schema prefix is `composer.`, not the node's directory name, so a reader could not have guessed it. Anyone copying these into a pipe config would get a silently broken node. AGENTS.md already says schema tables are generated and must not be hand-edited; these were the only node docs not following that, which is also why a hand-written table could drift this far in the first place. Replaced with ROCKETRIDE:GENERATED:PARAMS blocks from nodes:docs-generate, and prefixed the same keys where they appear in surrounding prose. Enum *values* (`caption`, `detailed_caption`, `more_detailed_caption`) are left bare -- they are values, not keys. The generated descriptions are also better than what they replace: e.g. depth_estimate.maxEdge now documents the upsample-back behavior and the VRAM/sharpness tradeoff, which the hand-written row omitted. MISSING_PARAMS on this branch: 13 -> 4. The remaining 4 (cloud_tts, landing_ai, tool_mem0, tool_n8n) are fixed in rocketride-org#1773. Verified: 24 passed, ruff clean. Generator itself untouched on this branch — the --force flag used to run it here lives in rocketride-org#1773.
…UNTIME From review on rocketride-org#1718: matching find_literal() on the basename alone let a source occurrence of a common final segment protect an unrelated path. The reviewer's suggested fix — pass the complete token — over-corrects. Measured against develop, full-token-only turns 5 correct citations into ORPHANED: .workspace/global.json built by packages/shell-api/versions/v0.d.ts:3455 build/state.json built by scripts/lib/download.js:42 (x4 citations) Both are real runtime artifacts that code refers to by filename while the doc supplies the directory. So: try the whole token first, and fall back to the basename only when it carries a file extension. That drops the meaningless protections (a citation ending in a bare English word) and keeps the real ones. Repo verdicts are unchanged at 32 RUNTIME / 0 ORPHANED, but the *evidence* is now sound. Before, `tools/list` was protected because the word "list" appears somewhere in source; it now matches the literal "tools/list" in nodes/test/tool_mcp_client/stub_mcp_server.py. Two regression tests added, per the review: one that an extensionless segment is not evidence, one that the extension-bearing fallback still protects. Known limit, now documented in the code: find_literal scans raw source text, so a path merely mentioned in a comment counts as evidence. I hit this while writing the fix — naming real paths in my own explanatory comment made the tool cite that comment as proof. The comment no longer names any real path. Properly narrowing this needs per-language comment stripping, which is a bigger change than this fix. Verified: 26 passed, ruff clean, format clean.
…tations from it From review on rocketride-org#1718. audit_doc() caught OSError and returned [], so a doc that could not be read dropped out of the run silently: no verdict, no gap, no error, exit 0. A permission or I/O failure was indistinguishable from a file containing no citations, and the audit reported success while having skipped it. This is the third instance of one shape in this tool, and I had already fixed the other two without noticing the pattern: a mistyped --root produced an empty audit that passed, and an unparseable services*.json produced zero declared params that looked clean. Same failure mode each time — the tool reporting success for work it did not do. Now yields a single UNREADABLE_DOC verdict carrying the file and the errno type, printed above the verdict table and failing the run unconditionally — not only under --fail-on-orphaned, since being unable to read a doc is a failure to audit rather than a finding about citations. Named UNREADABLE_DOC rather than UNREADABLE because coverage.py already exports UNREADABLE for a node's schema/README; importing both into cli.py collided (ruff F811). Distinct concepts, distinct names. Verified end to end: tree with an unreadable doc -> exit 1, file listed same tree readable -> exit 0 this repo -> exit 0, unchanged 27 passed (1 new regression test, skipped when running as root since mode bits do not apply), ruff clean, format clean.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes part of #1717.
AGENTS.mdco-locates docs with the code they document, but nothing checks that they still agree after a rename. This adds that check, then applies the fixes it can apply safely.1.
tools/docs_audit/— the checkerdoc → code. Every path a doc cites gets a class and the evidence behind it, not a boolean:
VERIFIEDPLACEHOLDERHISTORICALRUNTIMEORPHANEDOnly
ORPHANEDis a deletion candidate, and it is reported, never auto-applied.The classification isn't ceremony. The naive version of this tool — flag every cited path that isn't on disk — calls 68% of this repo's citations dead. Almost all of that is false, and acting on it deletes correct docs:
Save this as `extract.pipe`:apps/vscode/src/engine/docker/engine-docker.tscode → doc.
STALE_PARAMS(generated table disagrees withservices*.json),MISSING_PARAMS,MISSING_DOC. Two subtleties are handled, both of which silently corrupt results otherwise:fields(carryingobject/propertiesrather thantype) are not parameters.nodes:docs-generateomits them; counting them reported phantom drift on 8 clean nodes.detect_segmentandvideo_composership JSONCservices.json.json.loadsthrows, and swallowing that made them look like they declared zero params. Fixing it tookSTALE_PARAMSfrom 4 → 18 — 14 nodes were hidden.2.
docs/README-n8n.mdThe only
ORPHANEDcitation of 385: it pointed readers at.context/n8n-test/forrun.shand aWALKTHROUGH.md, none of which is tracked (and.context/isn't gitignored). Now points at the realexamples/n8n/pipes.3. READMEs for nine undocumented nodes
background_removal,caption,depth_estimate,detect,detect_segment,face_detection,pose_estimation,tool_google_workspace,video_composer.Prose is grounded in each node's own
services*.jsondescription, lanes, and fields — not invented.tool_google_workspaceadditionally documents that every destructive capability (hard delete, public sharing) is a separate opt-in flag defaulting tofalse.Deliberately not fixed here
The 18
STALE_PARAMSand theMISSING_PARAMSblocks regenerate vianodes:docs-generate, andAGENTS.mdsays never to hand-edit between the generated markers. That needs the engine build, so it needs a maintainer. The nine new READMEs therefore moveMISSING_DOC9 → 0 while showing up underMISSING_PARAMSuntil someone regenerates.graph_falkordbis the one I'd look at first — its README documents the pre-renametool_falkordb.*keys.Verification
python3 -m pytest tools/docs_audit/test/ -q→ 18 passedruff check+ruff format --check→ cleanMISSING_DOC9 → 0Every test named
test_placeholder_*,test_counter_example_*,test_profile_groups_*, ortest_jsonc_*pins a false positive an earlier revision of this tool actually produced. They're the reason the tool is trustworthy enough to act on — worth keeping.I don't have the engine build locally, so
builder docs:buildhasn't been run against this.Summary by CodeRabbit
Documentation
New Features