diff --git a/docs/README-n8n.md b/docs/README-n8n.md index b73246585..735dd5639 100644 --- a/docs/README-n8n.md +++ b/docs/README-n8n.md @@ -14,9 +14,9 @@ Combining the two gives **round-trips**: RocketRide → n8n → RocketRide. > (n8n side, importable). > > **Runnable test pipes** that exercise every mode (sync / async / sequential / agent / round-trip) -> live in [`examples/n8n/`](../examples/n8n/) — open them in the IDE. They pair with the local -> test harness in `.context/n8n-test/` (`run.sh --keep` seeds the `rr-echo` / `rr-slow` / `rr-upper` -> / `rr-callback` workflows); see that folder's `WALKTHROUGH.md` for the step-by-step. +> live in [`examples/n8n/`](../examples/n8n/) — open them in the IDE. Import +> [`n8n-dispatch.workflow.json`](../examples/n8n/n8n-dispatch.workflow.json) on the n8n side and +> point each pipe's webhook URL at your own instance. --- diff --git a/nodes/src/nodes/background_removal/README.md b/nodes/src/nodes/background_removal/README.md new file mode 100644 index 000000000..382b1caa9 --- /dev/null +++ b/nodes/src/nodes/background_removal/README.md @@ -0,0 +1,52 @@ +# background_removal + +A RocketRide image-filter node that separates foreground from background and emits an RGBA cutout. + +## What it does + +Receives an image stream and runs **BiRefNet** (MIT) to produce an alpha matte, then +composites an RGBA cutout with a **straight (non-premultiplied) alpha** channel, so +downstream nodes can re-composite over any background without dark fringes. + +Per frame the node emits on two lanes: + +- `image` — the RGBA cutout as PNG +- `text` — JSON alpha statistics (`mean_alpha`, `alpha_coverage_pct`) + +Before inference the source is downscaled so its long edge is at most `background_removal.maxEdge`, which +bounds memory use; the alpha matte is then restored to the original resolution for +compositing. `background_removal.maxEdge` is clamped to 256–4096 (default 1024) regardless of what is +configured. + +Two profiles ship: the default 1K BiRefNet, and a 2K high-resolution variant for fine +hair and detailed edges. The model runs on CPU, Apple Silicon (MPS), or CUDA. Local +inference serializes GPU access behind a shared device lock; when the engine is started +with `--modelserver`, inference is dispatched to the model server instead. + +--- + +## Configuration + +### Lanes + +| Lane | Direction | Description | +|------|-----------|-------------| +| `image` | input | Source image (streamed) | +| `image` | output | RGBA cutout PNG, straight alpha | +| `text` | output | JSON alpha stats: `mean_alpha`, `alpha_coverage_pct` | + + + + +## Schema + +| Field | Type | Description | Default | +|---|---|---|---| +| `background_removal.maxEdge` | `number` | **Max input edge (px)**
Downscale source so long edge <= this value before inference; alpha is upsampled back to the (capped) source size for compositing. Lower = faster + less VRAM; higher = sharper edges. | `1024` | +| `background_removal.model` | `string` | **Model**
HuggingFace model identifier for background removal (overrides the profile default) | | +| `background_removal.profile` | `string` | **Model**
BiRefNet variant — default is 1K, HR is 2K for finer edges. | `"birefnet-default"` | + +## Source + +[ View source](https://github.com/rocketride-org/rocketride-server/tree/develop/nodes/src/nodes/background_removal) + diff --git a/nodes/src/nodes/caption/README.md b/nodes/src/nodes/caption/README.md new file mode 100644 index 000000000..67642d819 --- /dev/null +++ b/nodes/src/nodes/caption/README.md @@ -0,0 +1,42 @@ +# caption + +A RocketRide image-filter node that generates a natural-language caption for an image. + +## What it does + +Receives an image and runs **Florence-2 Base** (MIT) locally to produce a descriptive +caption on the text lane. Three granularities are exposed via `caption.task`, from shortest to +longest: `caption` (the default), `detailed_caption`, and `more_detailed_caption`. + +Runs on CPU, Apple Silicon (MPS), and CUDA with **no API key required** — inference is +local, so images never leave the host. + +For object detection use the **Object Detection** (`detect`) node; for reading text in +an image use the **OCR** node. This node describes a scene, it does not localize or +transcribe. + +--- + +## Configuration + +### Lanes + +| Lane | Direction | Description | +|------|-----------|-------------| +| `image` | input | Source image (streamed) | +| `text` | output | The generated caption | + + + + +## Schema + +| Field | Type | Description | Default | +|---|---|---|---| +| `caption.profile` | `string` | **Model** | `"florence-base"` | +| `caption.task` | `string` | **Granularity**
How detailed the caption should be. | `"caption"` | + +## Source + +[ View source](https://github.com/rocketride-org/rocketride-server/tree/develop/nodes/src/nodes/caption) + diff --git a/nodes/src/nodes/depth_estimate/README.md b/nodes/src/nodes/depth_estimate/README.md new file mode 100644 index 000000000..785bd5a6b --- /dev/null +++ b/nodes/src/nodes/depth_estimate/README.md @@ -0,0 +1,43 @@ +# depth_estimate + +A RocketRide image-filter node that estimates per-pixel depth from a single image. + +## What it does + +Runs **Depth Anything V2 Small** (Apache-2.0) for monocular depth estimation and emits +a colorized depth map on the image lane, where **red is near and blue is far**. Depth +statistics (min, max, mean) are emitted as JSON on the text lane. + +Pair this with the **Object Detection** (`detect`) node to get a rough distance to each +detected object. + +Before inference the input is downscaled so its long edge is at most `depth_estimate.maxEdge`, which +bounds memory use; the dense output is restored to the original resolution afterward. +Runs on CPU, Apple Silicon (MPS), and CUDA. + +--- + +## Configuration + +### Lanes + +| Lane | Direction | Description | +|------|-----------|-------------| +| `image` | input | Source image (streamed) | +| `image` | output | Colorized depth map (red = near, blue = far) | +| `text` | output | JSON depth statistics: min, max, mean | + + + + +## Schema + +| Field | Type | Description | Default | +|---|---|---|---| +| `depth_estimate.maxEdge` | `number` | **Max input edge (px)**
Downscale input so the long edge <= this value before inference; dense output is upsampled back to original. Lower = faster + less VRAM, higher = sharper depth. | `1024` | +| `depth_estimate.profile` | `string` | **Model** | `"v2-small"` | + +## Source + +[ View source](https://github.com/rocketride-org/rocketride-server/tree/develop/nodes/src/nodes/depth_estimate) + diff --git a/nodes/src/nodes/detect/README.md b/nodes/src/nodes/detect/README.md new file mode 100644 index 000000000..d980ce99c --- /dev/null +++ b/nodes/src/nodes/detect/README.md @@ -0,0 +1,50 @@ +# detect + +A RocketRide image-filter node that finds objects in a frame and emits bounding boxes. + +## What it does + +Runs per-frame object detection and emits bounding boxes, labels, and centroids on the +text lane alongside an annotated frame on the image lane. + +Two engines are available via `detect.profile`: + +- **RF-DETR** (Apache-2.0, default) — a fast **closed-set** detector over the 80 COCO + classes (person, car, dog, and so on). +- **MM-Grounding-DINO** (Apache-2.0 / BSD-3) — the **open-vocabulary** option. Set + `detect.prompt` to detect anything you can name. + +`detect.prompt` accepts either a period- or comma-separated class list (`person . car . dog`) +or a described object (`red car`, `person in a hat`), and returns every matching region. +It matches objects and attributes, not spatial relationships. + +Useful as a cheap per-frame gate in front of heavier models. For pixel-level masks use +the **Segmentation** (`detect_segment`) node instead. + +--- + +## Configuration + +### Lanes + +| Lane | Direction | Description | +|------|-----------|-------------| +| `image` | input | Source frame (streamed) | +| `image` | output | Annotated frame with boxes drawn | +| `text` | output | JSON detections: bounding boxes, labels, centroids | + + + + +## Schema + +| Field | Type | Description | Default | +|---|---|---|---| +| `detect.profile` | `string` | **Model** | `"rfdetr"` | +| `detect.prompt` | `string` | **Detection prompt**
Example: "person . car . dog" (period or comma-separated list) or "red car" / "person in a hat" (described object). Returns all matching regions. Matches objects and attributes — not spatial relationships, so "the car on the left" returns all cars, not just the left one. | | +| `detect.threshold` | `number` | **Confidence threshold**
Minimum confidence score (0.0–1.0) to include a detection | `0.3` | + +## Source + +[ View source](https://github.com/rocketride-org/rocketride-server/tree/develop/nodes/src/nodes/detect) + diff --git a/nodes/src/nodes/detect_segment/README.md b/nodes/src/nodes/detect_segment/README.md new file mode 100644 index 000000000..fa73ac9e2 --- /dev/null +++ b/nodes/src/nodes/detect_segment/README.md @@ -0,0 +1,54 @@ +# detect_segment + +A RocketRide image-filter node that produces pixel-level segmentation masks. + +## What it does + +Runs pixel-level segmentation with HuggingFace-native engines and emits an annotated +overlay on the image lane plus a Masks JSON payload on the text lane. + +Two modes are available: + +- **Mask2Former-instance** (MIT, default) — closed-set **instance** masks, one mask per + detected object. +- **Mask2Former-semantic** (MIT) — a per-pixel **class map** over the whole frame. + +Accepts a single frame or multiple frames (via `frame_grabber` documents). Input is +downscaled so its long edge is at most `detect_segment.maxEdge` before inference. + +For bounding boxes only — which is considerably cheaper — use the **Object Detection** +(`detect`) node. + +--- + +## Configuration + +### Lanes + +| Lane | Direction | Description | +|------|-----------|-------------| +| `image` | input | Source frame, or multi-frame documents | +| `image` | output | Annotated overlay | +| `text` | output | Masks JSON | + + + + +## Schema + +| Field | Type | Description | Default | +|---|---|---|---| +| `detect_segment.engine` | `string` | **Engine**
Backing engine. Gated by mode: instance -> mask2former-instance; semantic -> mask2former-semantic. | `"mask2former-instance"` | +| `detect_segment.maxEdge` | `number` | **Max input edge (px)**
Downscale source so long edge <= this value before inference; masks are upsampled back to the source size. Lower = faster + less VRAM. | `1024` | +| `detect_segment.mode` | `string` | **Mode**
Segmentation mode. instance: per-instance masks (default). semantic: per-pixel class map. Both use Mask2Former under the hood. | `"instance"` | +| `detect_segment.profile` | `string` | **Profile**
Segmentation preset. Runs on CPU/MPS/CUDA via transformers. | `"mask2former-instance"` | +| `detect_segment.threshold` | `number` | **Confidence threshold**
Minimum score (0.0-1.0) to include a mask | `0.3` | + +## Dependencies + +- `pycocotools` + +## Source + +[ View source](https://github.com/rocketride-org/rocketride-server/tree/develop/nodes/src/nodes/detect_segment) + diff --git a/nodes/src/nodes/face_detection/README.md b/nodes/src/nodes/face_detection/README.md new file mode 100644 index 000000000..1cea2d759 --- /dev/null +++ b/nodes/src/nodes/face_detection/README.md @@ -0,0 +1,50 @@ +# face_detection + +A RocketRide image-filter node that detects faces and optional alignment keypoints. + +## What it does + +Runs per-frame face detection using **MediaPipe BlazeFace** (Apache-2.0) and emits +axis-aligned bounding boxes for every detected face. + +When `face_detection.emit_landmarks` is on (the default) each face also carries 6 coarse, +alignment-grade keypoints: `right_eye`, `left_eye`, `nose_tip`, `mouth_center`, +`right_ear_tragion`, `left_ear_tragion`. + +Fast enough to use as a face-presence gate ahead of heavier models, or to drive +face-aware framing and cropping. These are coarse alignment keypoints — this is not a +dense facial-landmark or face-recognition node. + +--- + +## Configuration + +### Lanes + +| Lane | Direction | Description | +|------|-----------|-------------| +| `image` | input | Source frame (streamed) | +| `image` | output | Annotated frame | +| `text` | output | JSON faces: bounding boxes and, optionally, 6 keypoints each | + + + + +## Schema + +| Field | Type | Description | Default | +|---|---|---|---| +| `face_detection.emit_landmarks` | `boolean` | **Emit 6 alignment keypoints**
Include coarse 6-point keypoints per face (eyes, nose, mouth, ear tragions) for face-aware framing and alignment. | `true` | +| `face_detection.profile` | `string` | **Model** | `"short"` | +| `face_detection.threshold` | `number` | **Confidence threshold**
Minimum detection confidence (0.0-1.0). Default 0.5 - higher than object detect to suppress false faces on textured backgrounds. | `0.5` | + +## Dependencies + +- `mediapipe` `>=0.10.35` +- `Pillow` +- `numpy` + +## Source + +[ View source](https://github.com/rocketride-org/rocketride-server/tree/develop/nodes/src/nodes/face_detection) + diff --git a/nodes/src/nodes/pose_estimation/README.md b/nodes/src/nodes/pose_estimation/README.md new file mode 100644 index 000000000..839c9d1e1 --- /dev/null +++ b/nodes/src/nodes/pose_estimation/README.md @@ -0,0 +1,42 @@ +# pose_estimation + +A RocketRide image-filter node that estimates human body pose per frame. + +## What it does + +Runs top-down human pose estimation using **RTMPose** (Apache-2.0) through the `rtmlib` +ONNX wrapper. **RTMDet-nano** performs person detection first, then RTMPose predicts +**17 COCO keypoints** for each person crop. + +Accepts an image or a document and emits an annotated frame, with the per-person +keypoint array attached to the document's metadata. + +Top-down means cost scales with the number of people in frame; `pose_estimation.max_persons` bounds +that work. + +--- + +## Configuration + +### Lanes + +| Lane | Direction | Description | +|------|-----------|-------------| +| `image` | input | Source frame or document | +| `image` | output | Annotated frame; keypoint array attached to document metadata | + + + + +## Schema + +| Field | Type | Description | Default | +|---|---|---|---| +| `pose_estimation.max_persons` | `number` | **Max persons per frame**
Cap on persons retained per frame (sorted by detection score). Limits memory + compute on crowd scenes. | `20` | +| `pose_estimation.profile` | `string` | **Model** | `"rtmpose-medium"` | +| `pose_estimation.threshold` | `number` | **Keypoint score threshold**
Minimum per-keypoint confidence (0.0–1.0). Keypoints below this score are skipped when drawing the skeleton. | `0.3` | + +## Source + +[ View source](https://github.com/rocketride-org/rocketride-server/tree/develop/nodes/src/nodes/pose_estimation) + diff --git a/nodes/src/nodes/tool_google_workspace/README.md b/nodes/src/nodes/tool_google_workspace/README.md new file mode 100644 index 000000000..b1d55c299 --- /dev/null +++ b/nodes/src/nodes/tool_google_workspace/README.md @@ -0,0 +1,98 @@ +# tool_google_workspace + +A RocketRide tool node that exposes Google Workspace operations to an AI agent. + +## What it does + +Registers five separate tool surfaces an agent can call. Each ships its own service +definition and its own access controls, so a pipeline can enable only what it needs: + +| Service | File | What the agent can do | +|---|---|---| +| **Gmail** | `services.gmail.json` | Read, search, label, draft, send, and organize mail | +| **Drive** | `services.drive.json` | List and search files, read metadata, download binaries, export native Docs/Sheets/Slides, create/update/copy/move files and folders, manage sharing, trash and untrash, track changes. Supports My Drive and shared drives | +| **Calendar** | `services.calendar.json` | List, get, create, update, move, and delete events (including recurring-series instances and natural-language quick-add); query free/busy; manage calendars and ACL rules. Supports incremental sync via `syncToken` | +| **Docs** | `services.docs.json` | Read document text; create documents; append and replace text; insert images and tables; run arbitrary `batchUpdate` requests | +| **Sheets** | `services.sheets.json` | Read, write, append, and clear cell values; create spreadsheets; add, delete, duplicate, and copy sheets; run arbitrary `batchUpdate` requests | + +Authenticates via a **Google service account** or **user OAuth**. + +This is a tool node, not a filter: it has no image or text lanes. It is invoked by an +agent rather than placed in a streaming path. + +### What is gated, and what is not + +Two different mechanisms, and only one of them defaults to safe. + +**Irreversible and public-facing operations are opt-in.** Permanent deletion +(`allowHardDelete` on Gmail and Drive, `allowDelete` on Calendar) and public or +domain-wide sharing (`allowPublicSharing` on Drive and Calendar) are separate booleans, +each defaulting to `false`. They must be turned on deliberately. + +**Ordinary writes are not.** The per-service `access` field defaults to `write` on +Drive, Calendar, Docs, and Sheets, and to `modify` (read + organize) on Gmail. So an +agent can create, update, and move files, edit documents, and create calendar events +without any flag being enabled — and calendar writes send invitations to attendees, +which is externally visible. Gmail is the exception in one direction: `send` is a +higher level than the `modify` default, so sending mail does require raising `access`. + +Set `access` to `readonly` for any service the pipeline only needs to read from. That +field, not the boolean flags, is what bounds the agent's day-to-day reach. + +--- + +## Configuration + + + + +## Schema + +### Google Calendar (`services.calendar.json`) + +| Field | Type | Description | Default | +|---|---|---|---| +| `calendar.access` | `string` | **Access level**
Calendar scopes to request. readonly: read events, calendars, ACLs, and free/busy only. write: full read/write (create, update, move, quick-add events; manage calendars and ACLs). Deletion additionally requires the allowDelete flag; public/domain-wide ACL sharing requires allowPublicSharing. | `"write"` | +| `calendar.allowDelete` | `boolean` | **Allow event / calendar deletion**
When off (the default), event_delete and calendar_delete are refused even at the write tier. Enable only if the agent should be able to permanently delete events and calendars — this is irreversible. | `false` | +| `calendar.allowPublicSharing` | `boolean` | **Allow public / domain-wide calendar sharing**
Off by default. When off, acl_insert refuses rules that expose the calendar beyond individual grantees (scopeType 'default' = anyone on the internet, and 'domain' = everyone in a domain). Turn on to allow public or domain-wide sharing. Grants to individual users/groups are not gated. | `false` | + +### Google Docs (`services.docs.json`) + +| Field | Type | Description | Default | +|---|---|---|---| +| `docs.access` | `string` | **Access level**
Docs scopes to request. readonly: read document text and metadata only. write: full read/write (create documents, append and replace text, insert images and tables, and run arbitrary batchUpdate requests). | `"write"` | + +### Google Drive (`services.drive.json`) + +| Field | Type | Description | Default | +|---|---|---|---| +| `drive.access` | `string` | **Access level**
Drive scopes to request. readonly: list, read metadata, download, and export only. write: full read/write (create, update, copy, move, trash, folders, and sharing). | `"write"` | +| `drive.allowHardDelete` | `boolean` | **Allow permanent delete**
Off by default. When off, file_delete (which permanently deletes a file, bypassing Trash and irreversibly) is refused. Turn on to allow permanent deletion; file_trash is the recoverable alternative. | `false` | +| `drive.allowPublicSharing` | `boolean` | **Allow public / external sharing**
Off by default. When off, permission_create refuses anyone-with-link grants and grants to a domain or user outside the account's own domain. Turn on to allow sharing files publicly or with external parties. | `false` | + +### Gmail (`services.gmail.json`) + +| Field | Type | Description | Default | +|---|---|---|---| +| `gmail.access` | `string` | **Access level**
Gmail scopes to request. readonly: read only. modify: read + label/organize. send: modify + send mail. settings: modify + filters/IMAP/POP/vacation/forwarding. settings_sharing: settings + sendAs/delegation/S⁠MIME. full: complete mailbox access, required for permanent delete. | `"modify"` | +| `gmail.allowHardDelete` | `boolean` | **Allow permanent delete**
Enable permanent message/thread deletion (requires full access tier). Disabled by default to protect against accidental data loss. | `false` | + +### Google Sheets (`services.sheets.json`) + +| Field | Type | Description | Default | +|---|---|---|---| +| `sheets.access` | `string` | **Access level**
Sheets scopes to request. readonly: read values and metadata only. write: full read/write (create, update, append, clear, and structure changes such as add/delete/duplicate sheet). | `"write"` | + +## Dependencies + +- `google-api-python-client` +- `google-auth` +- `google-auth-oauthlib` +- `google-auth-httplib2` +- `idna` `>=3.15` +- `protobuf` `>=5.29.6` + +## Source + +[ View source](https://github.com/rocketride-org/rocketride-server/tree/develop/nodes/src/nodes/tool_google_workspace) + diff --git a/nodes/src/nodes/video_composer/README.md b/nodes/src/nodes/video_composer/README.md new file mode 100644 index 000000000..bf4bce3c2 --- /dev/null +++ b/nodes/src/nodes/video_composer/README.md @@ -0,0 +1,42 @@ +# video_composer + +A RocketRide node that stitches a sequence of image frames into an MP4. + +## What it does + +Collects the image frames flowing through it and re-encodes them into a playable MP4 +clip using **FFmpeg**. + +Place it after any image-producing filter — for example `detect`, `pose_estimation`, or +`background_removal` — to turn that filter's annotated frames back into a video. + +Output frame rate is set by `composer.fps`, and quality by `composer.crf` (lower is higher quality and a +larger file; 23 is FFmpeg's default). + +Requires an FFmpeg binary available to the engine. + +--- + +## Configuration + +### Lanes + +| Lane | Direction | Description | +|------|-----------|-------------| +| `image` | input | Frames to stitch, in arrival order | + + + + +## Schema + +| Field | Type | Description | Default | +|---|---|---|---| +| `composer.crf` | `number` | **Quality (CRF)**
Constant Rate Factor for H.264. Lower = better quality, larger file. Range 0-51. | `23` | +| `composer.fps` | `number` | **Output frame rate (fps)**
Playback speed of the output video. Should match the upstream frame rate. | `1` | +| `composer.profile` | `string` | **Output quality**
Video encoding quality preset | `"standard"` | + +## Source + +[ View source](https://github.com/rocketride-org/rocketride-server/tree/develop/nodes/src/nodes/video_composer) + diff --git a/tools/docs_audit/README.md b/tools/docs_audit/README.md new file mode 100644 index 000000000..1f37f59b1 --- /dev/null +++ b/tools/docs_audit/README.md @@ -0,0 +1,69 @@ +# docs-audit: verify documentation against the code it describes + +Answers two questions mechanically, so doc cleanup is a review task instead of +an archaeology task: + +1. **Does every path a doc cites actually exist?** (delete/fix candidates) +2. **Does every node that ships code actually have docs?** (write candidates) + +## Why it is not a `grep -c` + +The obvious version of this tool — "flag every cited path that isn't on disk" — +reports **68% of this repo's doc citations as dead**. Nearly all of that is +wrong, and acting on it deletes correct documentation. Three ways a citation +looks dead while being right: + +| Doc says | On disk | Actually | +| --- | --- | --- | +| ``Save this as `extract.pipe`:`` | absent | a file the **reader** creates | +| ``Writes `version.docker.json``` | absent | built at runtime by `apps/vscode/src/engine/docker/engine-docker.ts` | +| ``**NOT:** `.pipeline.json``` | absent | a **counter-example** — deleting it reintroduces the mistake the doc prevents | + +So every citation gets a **class plus the evidence behind it**, and only one +class is ever a deletion candidate: + +- `VERIFIED` — resolves to a real path, or some file in the tree has that basename +- `PLACEHOLDER` — create-verb prose, a scaffolding tree, or an illustrative example +- `HISTORICAL` — a changelog naming a deleted file is correct by definition +- `RUNTIME` — no file at rest, but source code constructs the name +- `ORPHANED` — no path, no basename, no source literal → **review it** + +`ORPHANED` is never auto-deleted. The tool reports; a human decides. + +## Code → doc + +Ordered by how loudly the gap misleads a reader: + +- `STALE_PARAMS` — the generated schema table disagrees with `services*.json`. + Confidently wrong, which is worse than absent. Fix by re-running + `nodes:docs-generate` — never by hand-editing the generated block. +- `MISSING_PARAMS` — a node README with no generated block at all. +- `MISSING_DOC` — a node ships Python and has no README. + +Profile groupings in `fields` (entries carrying `object`/`properties` rather +than `type`) are **not** parameters; `nodes:docs-generate` omits them from the +table, so counting them reports phantom drift on every profile-based node. + +## Run it + +```sh +python3 tools/docs_audit/cli.py --root . +python3 tools/docs_audit/cli.py --root . --json # machine-readable +python3 tools/docs_audit/cli.py --root . --fail-on-orphaned # CI gate +``` + +Tests: + +```sh +python3 -m pytest tools/docs_audit/test/ -q +``` + +Every test named `test_placeholder_*`, `test_counter_example_*`, or +`test_profile_groups_*` pins a false positive an earlier version of this tool +actually produced. Keep them. + +## Scope + +Path-level citations only. Symbol-level checking (does this doc's +`session.display.render()` still match the signature?) is a natural extension +and is not implemented. diff --git a/tools/docs_audit/cli.py b/tools/docs_audit/cli.py new file mode 100644 index 000000000..4ac8f7df3 --- /dev/null +++ b/tools/docs_audit/cli.py @@ -0,0 +1,22 @@ +""" +Thin CLI entry point for local debugging. + +Invoke as: python tools/docs_audit/cli.py [--root=...] [--json] + +Adjusts sys.path so the ``docs_audit`` package is importable, then delegates +to :mod:`docs_audit.cli`. +""" + +from __future__ import annotations + +import sys +from pathlib import Path + +_SRC = Path(__file__).resolve().parent / 'src' +if str(_SRC) not in sys.path: + sys.path.insert(0, str(_SRC)) + +from docs_audit.cli import main # noqa: E402 + +if __name__ == '__main__': + sys.exit(main()) diff --git a/tools/docs_audit/src/docs_audit/__init__.py b/tools/docs_audit/src/docs_audit/__init__.py new file mode 100644 index 000000000..5119fad24 --- /dev/null +++ b/tools/docs_audit/src/docs_audit/__init__.py @@ -0,0 +1 @@ +"""Documentation audit: verify docs against the code they claim to describe.""" diff --git a/tools/docs_audit/src/docs_audit/citations.py b/tools/docs_audit/src/docs_audit/citations.py new file mode 100644 index 000000000..300e91a4b --- /dev/null +++ b/tools/docs_audit/src/docs_audit/citations.py @@ -0,0 +1,232 @@ +"""Doc -> code direction: every path a doc cites, classified with evidence. + +A naive "does this path exist?" check reports ~68% of this repo's doc citations +as dead. Nearly all of that is false: docs legitimately name files the *reader* +creates, files that only ever exist at runtime, and files that were deleted on +purpose (changelogs). Deleting on that signal destroys correct documentation. + +So a citation is not boolean, it is classified: + +``VERIFIED`` resolves to a real path in the tree +``PLACEHOLDER`` prose tells the reader to create it -- protected +``HISTORICAL`` changelog/release note describing the past -- protected +``RUNTIME`` no file at rest, but source code builds the name -- protected +``ORPHANED`` no referent found anywhere -- the only deletion candidate + +Every verdict carries evidence so a human can check the tool's work. +""" + +from __future__ import annotations + +import re +from dataclasses import dataclass +from pathlib import Path + +from .index import CodeIndex + +VERIFIED = 'VERIFIED' +PLACEHOLDER = 'PLACEHOLDER' +HISTORICAL = 'HISTORICAL' +RUNTIME = 'RUNTIME' +ORPHANED = 'ORPHANED' +#: The doc itself could not be read. Not a citation verdict -- a failure to +#: audit. Named apart from coverage.UNREADABLE, which is about a node's schema. +UNREADABLE_DOC = 'UNREADABLE_DOC' + +PROTECTED = frozenset({VERIFIED, PLACEHOLDER, HISTORICAL, RUNTIME}) + +_EXT = r'\.(?:py|ts|tsx|js|mjs|cjs|json|cpp|cc|h|hpp|cmake|toml|yaml|yml|sh|cmd|pipe|mdx?|env|tsv|csv)' +# An inline-code span that looks like a file or directory reference. +_PATH_SPAN = re.compile(r'`([A-Za-z0-9_.][A-Za-z0-9_.\-/]*' + _EXT + r')`') +_DIR_SPAN = re.compile(r'`((?:nodes|packages|apps|tools|docs|scripts|examples|deploy|docker)/[A-Za-z0-9_.\-/]+)`') + +# Prose that introduces a file the reader -- or the code being described -- is +# about to make. ``emit``/``produce``/``output`` cover docs that describe what a +# pipeline or agent writes at run time, which is absent at rest by definition. +_CREATE_VERB = re.compile( + r'\b(?:create|creating|add|adding|new|name\s+it|call\s+it|save\s+(?:this|it|that)?\s*as|scaffold|' + r'generate|generates|generated|emit|emits|emitted|produce|produces|output|outputs|' + r'write|writes|make|touch|rename\s+to|copy\s+to|place\s+in|put\s+in)\b', + re.IGNORECASE, +) + +# Directories that tooling installs into a *user's* workspace, never checked in. +# `.rocketride/` is written by the VS Code extension's installer +# (apps/vscode/src/agents/agent-manager.ts), so docs telling a reader to open a +# file under it are correct precisely because the repo does not contain it. +_INSTALLED_DIR = re.compile(r'(?:^|/)\.rocketride/') +# Prose citing a name to ILLUSTRATE a convention rather than to point at a file. +# Includes counter-examples ("NOT: `.pipeline.json`"), which are the most +# dangerous thing a cleanup pass can delete: removing them reintroduces exactly +# the mistake the doc exists to prevent. +_ILLUSTRATION = re.compile( + r'(?:\bexamples?\s*:|\be\.g\.|\bfor\s+example\b|\bsuch\s+as\b|\blike\b\s*:|' + r'\bNOT\s*:|\bnot\b\s*:|\bavoid\b|\binstead\s+of\b|\buse\s+descriptive\s+names?\b|' + r'\binclude\s+purpose\b|\bnaming\b)', + re.IGNORECASE, +) +# ASCII tree drawings in scaffolding docs. +_TREE_GLYPH = re.compile(r'[├└│]|^\s*[-*]?\s*\|--') +# Template-ish stems that are obviously stand-ins, not real repo files. +_TEMPLATE_STEM = re.compile(r'^(?:my|your|example|sample|foo|bar|placeholder|some|test)[-_A-Z]', re.IGNORECASE) + +# Docs whose entire job is to describe the past. +_HISTORICAL_DOCS = re.compile( + r'(?:^|/)(?:CHANGELOG|RELEASE|RELEASES|HISTORY|MIGRATION|UPGRADING)[^/]*\.mdx?$', re.IGNORECASE +) + + +@dataclass(frozen=True) +class Citation: + """One path-like token cited by one doc at one line.""" + + token: str + doc: str + line: int + + +@dataclass(frozen=True) +class Verdict: + citation: Citation + verdict: str + evidence: str + + @property + def is_protected(self) -> bool: + return self.verdict in PROTECTED + + +def _strip_fenced_blocks(text: str) -> list[str]: + """Return lines with fenced code-block bodies blanked out. + + Citations inside a fence are usually sample output or config the reader + pastes, not claims about this repo's layout. We keep the line count stable + so reported line numbers still point at the real file. + """ + lines = text.splitlines() + out: list[str] = [] + in_fence = False + for line in lines: + if re.match(r'^\s*(?:```|~~~)', line): + in_fence = not in_fence + out.append('') + continue + out.append('' if in_fence else line) + return out + + +def extract(text: str, doc: str) -> list[Citation]: + """Every distinct path-like citation in ``text``, with line numbers.""" + seen: set[tuple[str, int]] = set() + found: list[Citation] = [] + for number, line in enumerate(_strip_fenced_blocks(text), start=1): + for pattern in (_PATH_SPAN, _DIR_SPAN): + for token in pattern.findall(line): + token = token.rstrip('/') + key = (token, number) + if key in seen: + continue + seen.add(key) + found.append(Citation(token=token, doc=doc, line=number)) + return found + + +def _context(lines: list[str], line: int, before: int = 2) -> str: + """The cited line plus a little preceding prose, for intent detection.""" + start = max(0, line - 1 - before) + return '\n'.join(lines[start:line]) + + +def classify(citation: Citation, index: CodeIndex, doc_lines: list[str]) -> Verdict: + """Classify one citation, attaching the evidence behind the verdict.""" + token = citation.token + doc_dir = Path(citation.doc).parent + + # 1. Resolves relative to the citing doc, or to the repo root. + # No lstrip('./') -- it strips a character SET, not a prefix, so a citation + # to `.env` would be looked up as `env`. Same bug as index.py had. + sibling = (doc_dir / token).as_posix() + if index.has_path(sibling): + return Verdict(citation, VERIFIED, f'path exists: {sibling}') + if index.has_path(token): + return Verdict(citation, VERIFIED, f'path exists: {token}') + + # 2. Some file in the tree has this basename -- loosely worded, not wrong. + basename = Path(token).name + matches = index.find_basename(basename) + if matches: + return Verdict(citation, VERIFIED, f'basename matches {len(matches)} path(s), e.g. {matches[0]}') + + # 3. A changelog naming a deleted file is correct by definition. + if _HISTORICAL_DOCS.search(citation.doc): + return Verdict(citation, HISTORICAL, f'{citation.doc} documents past state') + + # 4. Installed into the reader's workspace by tooling, not stored here. + if _INSTALLED_DIR.search(token): + return Verdict(citation, RUNTIME, 'installed into the workspace by tooling, not checked in') + + # 5. No file at rest, but the code constructs the name at runtime. + # + # This runs before the prose heuristics below deliberately: finding the name + # as a literal in source is hard evidence, while a create-verb nearby is a + # guess about intent. Both verdicts are protected, so the ordering does not + # change what survives a cleanup -- it changes the evidence a human reads, + # and "source builds this name at pkg/real.py:3" is worth more than + # "create-verb in context". (Prose like "Writes `x.json`" matches both.) + # Prefer the whole citation. Falling straight back to the basename let any + # source occurrence of a common final segment protect an unrelated path: a + # citation ending in a bare English word was held RUNTIME because that word + # appears somewhere in source, which is not evidence of anything. + # + # The fallback survives only for a basename carrying a file extension, which + # is the real case: a build artifact referred to by filename in code while + # the doc supplies its directory. Requiring the full token reports those as + # dead. + # + # NB: no real repository path is named in this comment on purpose -- + # find_literal scans raw source text, so a path written here would index as + # its own evidence. That cuts both ways and is a known limit: a path merely + # mentioned in a comment anywhere in the tree counts as "source builds this + # name". Narrowing that needs comment-stripping per language, which is a + # bigger change than this fix. + literal = index.find_literal(token) + if literal is None and '.' in basename and basename != token: + found = index.find_literal(basename) + if found is not None: + where, where_line = found + return Verdict(citation, RUNTIME, f'source builds this filename: {where}:{where_line}') + if literal is not None: + where, where_line = literal + return Verdict(citation, RUNTIME, f'source builds this name: {where}:{where_line}') + + # 6. Prose tells the reader to create it, or it is a template stand-in. + context = _context(doc_lines, citation.line) + if _CREATE_VERB.search(context): + return Verdict(citation, PLACEHOLDER, f'create-verb in context at line {citation.line}') + if _ILLUSTRATION.search(doc_lines[citation.line - 1] if citation.line <= len(doc_lines) else ''): + return Verdict(citation, PLACEHOLDER, f'illustrative naming example at line {citation.line}') + if _TREE_GLYPH.search(doc_lines[citation.line - 1] if citation.line <= len(doc_lines) else ''): + return Verdict(citation, PLACEHOLDER, f'inside a directory-tree diagram at line {citation.line}') + if _TEMPLATE_STEM.search(basename): + return Verdict(citation, PLACEHOLDER, f'template stem: {basename}') + + return Verdict(citation, ORPHANED, 'no path, basename, or source literal found') + + +def audit_doc(path: Path, root: Path, index: CodeIndex) -> list[Verdict]: + """Classify every citation in a single doc. + + A doc that cannot be read yields a single ``UNREADABLE`` verdict rather than + an empty list. Returning nothing made an I/O or permission failure + indistinguishable from a doc containing no citations, so the file dropped + out of the audit silently and the run still reported success -- the same + failure shape as a mistyped ``--root``. + """ + doc = path.relative_to(root).as_posix() + try: + text = path.read_text(encoding='utf-8', errors='replace') + except OSError as exc: + citation = Citation(token=doc, doc=doc, line=0) + return [Verdict(citation, UNREADABLE_DOC, f'could not be read ({type(exc).__name__})')] + lines = text.splitlines() + return [classify(citation, index, lines) for citation in extract(text, doc)] diff --git a/tools/docs_audit/src/docs_audit/cli.py b/tools/docs_audit/src/docs_audit/cli.py new file mode 100644 index 000000000..d0a676683 --- /dev/null +++ b/tools/docs_audit/src/docs_audit/cli.py @@ -0,0 +1,124 @@ +"""Command-line entry point for the documentation audit.""" + +from __future__ import annotations + +import argparse +import json +import os +import sys +from pathlib import Path + +from .citations import ORPHANED, UNREADABLE_DOC, audit_doc +from .coverage import MISSING_DOC, MISSING_PARAMS, STALE_PARAMS, UNREADABLE, audit_nodes +from .index import EXCLUDED_PARTS, CodeIndex, is_excluded + +DOC_SUFFIXES = ('.md', '.mdx') + + +def _docs(root: Path): + # os.walk with in-place pruning, not rglob: rglob descends node_modules and + # every vendored tree in full before the filter discards what it yielded. + for dirpath, dirnames, filenames in os.walk(root): + rel_dir = Path(dirpath).relative_to(root) + dirnames[:] = [d for d in dirnames if d not in EXCLUDED_PARTS] + if is_excluded(rel_dir): + continue + for name in filenames: + if name.endswith(DOC_SUFFIXES): + yield Path(dirpath) / name + + +def build_parser() -> argparse.ArgumentParser: + parser = argparse.ArgumentParser( + prog='docs-audit', + description='Audit documentation against the code it claims to describe.', + ) + parser.add_argument('--root', default='.', help='Repository root (default: cwd)') + parser.add_argument('--json', action='store_true', help='Emit machine-readable JSON') + parser.add_argument( + '--fail-on-orphaned', + action='store_true', + help='Exit non-zero if any ORPHANED citation is found (for CI)', + ) + return parser + + +def main(argv: list[str] | None = None) -> int: + args = build_parser().parse_args(argv) + root = Path(args.root).resolve() + + # A typo in --root would otherwise audit nothing, find nothing, and exit 0 -- + # green CI that never ran. Fail loudly instead. + if not root.is_dir(): + print(f'docs-audit: --root is not a directory: {root}', file=sys.stderr) + return 2 + + index = CodeIndex.build(root) + verdicts = [verdict for path in _docs(root) for verdict in audit_doc(path, root, index)] + gaps = audit_nodes(root) + + orphaned = [v for v in verdicts if v.verdict == ORPHANED] + # A doc that could not be read is a failure to audit, not a clean audit, + # so it fails the run unconditionally rather than only under --fail-on-orphaned. + unreadable = [v for v in verdicts if v.verdict == UNREADABLE_DOC] + + if args.json: + payload = { + 'citations': { + 'total': len(verdicts), + 'by_verdict': { + verdict: sum(1 for v in verdicts if v.verdict == verdict) + for verdict in sorted({v.verdict for v in verdicts}) + }, + 'orphaned': [ + {'doc': v.citation.doc, 'line': v.citation.line, 'token': v.citation.token, 'evidence': v.evidence} + for v in orphaned + ], + }, + 'coverage': [{'kind': g.kind, 'node': g.node, 'detail': g.detail} for g in gaps], + } + json.dump(payload, sys.stdout, indent=2) + sys.stdout.write('\n') + return 1 if (unreadable or (args.fail_on_orphaned and orphaned)) else 0 + + if unreadable: + print(f'UNREADABLE docs ({len(unreadable)}) -- these were NOT audited:\n') + for v in unreadable: + print(f' {v.citation.doc}: {v.evidence}') + print() + + print(f'Scanned {len(verdicts)} doc->code citations across the tree.\n') + print(' verdict count meaning') + print(' --------------- ----- -------') + labels = { + 'VERIFIED': 'resolves to real code', + 'PLACEHOLDER': 'reader creates it (protected)', + 'HISTORICAL': 'describes the past (protected)', + 'RUNTIME': 'built at runtime (protected)', + 'ORPHANED': 'no referent -> review for deletion', + } + for verdict in ('VERIFIED', 'PLACEHOLDER', 'HISTORICAL', 'RUNTIME', 'ORPHANED'): + count = sum(1 for v in verdicts if v.verdict == verdict) + print(f' {verdict:<15} {count:>5} {labels[verdict]}') + + if orphaned: + print(f'\nORPHANED citations ({len(orphaned)}) -- each needs a human decision:\n') + for verdict in sorted(orphaned, key=lambda v: (v.citation.doc, v.citation.line)): + citation = verdict.citation + print(f' {citation.doc}:{citation.line} `{citation.token}`') + + if gaps: + print(f'\nUndocumented / drifted code ({len(gaps)}):\n') + for kind in (UNREADABLE, STALE_PARAMS, MISSING_PARAMS, MISSING_DOC): + matching = [g for g in gaps if g.kind == kind] + if not matching: + continue + print(f' [{kind}] {len(matching)}') + for gap in matching: + print(f' {gap.node}: {gap.detail}') + + return 1 if (unreadable or (args.fail_on_orphaned and orphaned)) else 0 + + +if __name__ == '__main__': # `python -m docs_audit.cli` printed nothing without this + raise SystemExit(main()) diff --git a/tools/docs_audit/src/docs_audit/coverage.py b/tools/docs_audit/src/docs_audit/coverage.py new file mode 100644 index 000000000..e106599bf --- /dev/null +++ b/tools/docs_audit/src/docs_audit/coverage.py @@ -0,0 +1,207 @@ +"""Code -> doc direction: public surface that documentation fails to cover. + +Three findings, ordered by how loudly they mislead a reader: + +``STALE_PARAMS`` the generated schema table disagrees with ``services*.json``. + Worst kind: confidently wrong. Happens when someone edits a + node's schema and never re-runs ``nodes:docs-generate``. +``MISSING_PARAMS`` a node README with no generated block at all, contrary to + the co-located documentation rule in AGENTS.md. +``MISSING_DOC`` a node that ships Python but no README whatsoever. +``UNREADABLE`` a schema or README that could not be read or parsed. Reported + rather than skipped: a malformed ``services*.json`` yields no + declared fields, which would otherwise look identical to a + node with nothing to document and hide real drift. +""" + +from __future__ import annotations + +import json +import re +from dataclasses import dataclass +from pathlib import Path + +STALE_PARAMS = 'STALE_PARAMS' +MISSING_PARAMS = 'MISSING_PARAMS' +MISSING_DOC = 'MISSING_DOC' +UNREADABLE = 'UNREADABLE' + +_GENERATED_BLOCK = re.compile( + r'(.*?)', + re.DOTALL, +) +# A row of the generated schema table: | `key` | `type` | ... | +_PARAM_ROW = re.compile(r'^\|\s*`([^`]+)`\s*\|', re.MULTILINE) + + +@dataclass(frozen=True) +class Gap: + """One documentation gap, with the evidence that proves it.""" + + kind: str + node: str + detail: str + + +def documented_params(readme_text: str) -> set[str] | None: + """Param keys listed in the README's generated block, or None if absent.""" + match = _GENERATED_BLOCK.search(readme_text) + if match is None: + return None + return set(_PARAM_ROW.findall(match.group(1))) + + +def strip_jsonc(text: str) -> str: + """Drop ``//`` line comments so JSONC ``services.json`` files parse. + + Several nodes ship commented schemas. A plain ``json.loads`` raises on + those, and swallowing the error silently reports the node as having no + parameters at all -- so real drift would never surface. Quote state is + tracked so ``https://`` inside a string value survives. + """ + out: list[str] = [] + in_string = False + escaped = False + index = 0 + while index < len(text): + char = text[index] + if in_string: + out.append(char) + if escaped: + escaped = False + elif char == '\\': + escaped = True + elif char == '"': + in_string = False + index += 1 + continue + if char == '"': + in_string = True + out.append(char) + index += 1 + continue + if char == '/' and index + 1 < len(text) and text[index + 1] == '/': + while index < len(text) and text[index] != '\n': + index += 1 + continue + out.append(char) + index += 1 + return ''.join(out) + + +def _is_user_facing_param(value: object) -> bool: + """True for a real settable parameter, false for a profile grouping. + + This mirrors ``nodes:docs-generate`` exactly, because that generator decides + what the table contains and is therefore the only correct oracle:: + + if (field && field.object !== undefined) continue; // Skip profile definitions + + (``nodes/scripts/gen-node-tables.mjs``). A profile group carries ``object``/ + ``properties`` and merely bundles other keys under a preset, so it is + excluded from the table and from this comparison. + + Requiring a ``type`` key here as well -- which this did originally -- is + stricter than the generator, and produced phantom STALE_PARAMS on every + node with a typeless field: the generator emits such a field with an empty + Type cell, the audit refused to count it as declared, and the diff surfaced + as "in docs but not in schema". Ten of the repository's findings were this + false positive rather than real drift. + """ + return isinstance(value, dict) and 'object' not in value + + +def schema_params(node_dir: Path) -> set[str]: + """User-facing param keys declared across every ``services*.json``.""" + keys: set[str] = set() + for services in sorted(node_dir.glob('services*.json')): + try: + data = json.loads(strip_jsonc(services.read_text(encoding='utf-8', errors='replace'))) + except (OSError, json.JSONDecodeError): + continue + fields = data.get('fields') + if isinstance(fields, dict): + keys.update(key for key, value in fields.items() if _is_user_facing_param(value)) + return keys + + +def unreadable_schemas(node_dir: Path) -> list[str]: + """Names of ``services*.json`` files that could not be read or parsed. + + Separate from :func:`schema_params` so that a broken schema is reported as + a finding instead of quietly contributing zero declared params -- which + reads exactly like a node that has nothing to document. + """ + broken = [] + for services in sorted(node_dir.glob('services*.json')): + try: + json.loads(strip_jsonc(services.read_text(encoding='utf-8', errors='replace'))) + except (OSError, json.JSONDecodeError) as exc: + broken.append(f'{services.name} ({type(exc).__name__})') + return broken + + +def audit_node(node_dir: Path, root: Path) -> list[Gap]: + """Every documentation gap for a single node directory.""" + name = node_dir.name + has_python = any(node_dir.glob('*.py')) + if not has_python: + return [] + + broken = unreadable_schemas(node_dir) + if broken: + # Stop here: declared params are unknowable, so any STALE/MISSING + # verdict computed from them would be noise on top of a real problem. + return [Gap(UNREADABLE, name, 'unparseable schema: ' + '; '.join(broken))] + + readme = node_dir / 'README.md' + if not readme.exists(): + count = len(list(node_dir.glob('*.py'))) + return [Gap(MISSING_DOC, name, f'{count} Python file(s), no README.md')] + + try: + text = readme.read_text(encoding='utf-8', errors='replace') + except OSError as exc: + return [Gap(UNREADABLE, name, f'README.md could not be read ({type(exc).__name__})')] + + declared = schema_params(node_dir) + documented = documented_params(text) + + if documented is None: + if declared: + return [ + Gap( + MISSING_PARAMS, + name, + f'{len(declared)} param(s) in services*.json, no ROCKETRIDE:GENERATED:PARAMS block', + ) + ] + return [] + + # Only meaningful when the node actually declares a schema; a node with no + # fields legitimately generates an empty table. + if not declared: + return [] + + undocumented = declared - documented + phantom = documented - declared + if not undocumented and not phantom: + return [] + + parts = [] + if undocumented: + parts.append(f'in schema but not in docs: {", ".join(sorted(undocumented)[:6])}') + if phantom: + parts.append(f'in docs but not in schema: {", ".join(sorted(phantom)[:6])}') + return [Gap(STALE_PARAMS, name, '; '.join(parts) + ' (re-run nodes:docs-generate)')] + + +def audit_nodes(root: Path) -> list[Gap]: + """Every documentation gap across every node.""" + nodes_root = root / 'nodes' / 'src' / 'nodes' + if not nodes_root.is_dir(): + return [] + gaps: list[Gap] = [] + for node_dir in sorted(p for p in nodes_root.iterdir() if p.is_dir()): + gaps.extend(audit_node(node_dir, root)) + return gaps diff --git a/tools/docs_audit/src/docs_audit/index.py b/tools/docs_audit/src/docs_audit/index.py new file mode 100644 index 000000000..1c65a9d10 --- /dev/null +++ b/tools/docs_audit/src/docs_audit/index.py @@ -0,0 +1,121 @@ +"""Read-only index of the repository's source tree. + +Built once per run and shared by every check. Everything here is a lookup the +classifier needs in order to attach *evidence* to a verdict: not just "this +citation is dead" but "no file, basename, or source literal named X exists". +""" + +from __future__ import annotations + +import os +from dataclasses import dataclass, field +from pathlib import Path + +# Trees that are vendored, generated, or otherwise not ours to audit. +EXCLUDED_PARTS = frozenset( + { + '.git', + 'node_modules', + 'site-packages', + 'engine-lib', + 'dist', + 'build', + '__pycache__', + '.venv', + 'venv', + '.eggs', + } +) + +# Source extensions we search for citation referents. +SOURCE_SUFFIXES = frozenset( + { + '.py', + '.ts', + '.tsx', + '.js', + '.mjs', + '.cjs', + '.cpp', + '.cc', + '.h', + '.hpp', + '.cmake', + '.json', + '.toml', + '.yaml', + '.yml', + '.sh', + '.cmd', + } +) + +# Skip files larger than this when scanning for string literals. Lockfiles and +# generated blobs blow up the scan and never contain meaningful referents. +MAX_SCAN_BYTES = 512 * 1024 + + +def is_excluded(relpath: Path) -> bool: + """True if any path segment is in an excluded tree.""" + return any(part in EXCLUDED_PARTS for part in relpath.parts) + + +@dataclass +class CodeIndex: + """Paths and source text of the auditable tree.""" + + root: Path + paths: set[str] = field(default_factory=set) + basenames: dict[str, list[str]] = field(default_factory=dict) + _sources: dict[str, str] = field(default_factory=dict) + + @classmethod + def build(cls, root: Path) -> CodeIndex: + root = root.resolve() + index = cls(root=root) + for dirpath, dirnames, filenames in os.walk(root): + rel_dir = Path(dirpath).relative_to(root) + # Prune excluded directories in place so os.walk never descends them. + dirnames[:] = [d for d in dirnames if d not in EXCLUDED_PARTS] + if is_excluded(rel_dir): + continue + for name in list(dirnames) + filenames: + # No lstrip('./') here: it strips a character SET, not a prefix, + # so `.env` becomes `env` and `.github/...` loses its dot. + # `Path('.') / name` already yields a clean relative path. + rel = (rel_dir / name).as_posix() + index.paths.add(rel) + index.basenames.setdefault(name, []).append(rel) + for name in filenames: + path = Path(dirpath) / name + if path.suffix.lower() not in SOURCE_SUFFIXES: + continue + try: + if path.stat().st_size > MAX_SCAN_BYTES: + continue + text = path.read_text(encoding='utf-8', errors='replace') + except OSError: + continue + index._sources[(rel_dir / name).as_posix()] = text + return index + + def has_path(self, relpath: str) -> bool: + return relpath.strip('/') in self.paths + + def find_basename(self, basename: str) -> list[str]: + """Every indexed path whose final segment is ``basename``.""" + return self.basenames.get(basename, []) + + def find_literal(self, token: str) -> tuple[str, int] | None: + """First ``(relpath, line_number)`` where ``token`` appears in source text. + + This is what separates a genuinely dead reference from one naming a + path the code builds at runtime (a doc citing ``version.docker.json`` + is correct even though no such file exists at rest, because + ``engine-docker.ts`` constructs it). + """ + for relpath, text in self._sources.items(): + position = text.find(token) + if position != -1: + return relpath, text.count('\n', 0, position) + 1 + return None diff --git a/tools/docs_audit/test/__init__.py b/tools/docs_audit/test/__init__.py new file mode 100644 index 000000000..e69de29bb diff --git a/tools/docs_audit/test/test_docs_audit.py b/tools/docs_audit/test/test_docs_audit.py new file mode 100644 index 000000000..37e63a71c --- /dev/null +++ b/tools/docs_audit/test/test_docs_audit.py @@ -0,0 +1,320 @@ +"""Tests for the documentation audit. + +The two regression tests that matter most are ``test_placeholder_*`` and +``test_profile_groups_*``: each pins a false positive that an earlier version +of this tool produced, and each would have caused correct documentation to be +deleted or a clean node to be reported as drifted. +""" + +from __future__ import annotations + +import json +import os +import sys +from pathlib import Path + +import pytest + +_SRC = Path(__file__).resolve().parents[1] / 'src' +if str(_SRC) not in sys.path: + sys.path.insert(0, str(_SRC)) + +from docs_audit.citations import ( # noqa: E402 + HISTORICAL, + ORPHANED, + PLACEHOLDER, + RUNTIME, + VERIFIED, + classify, + extract, +) +from docs_audit.citations import UNREADABLE_DOC, audit_doc # noqa: E402 +from docs_audit.cli import main # noqa: E402 +from docs_audit.coverage import ( # noqa: E402 + MISSING_DOC, + STALE_PARAMS, + UNREADABLE, + audit_node, + schema_params, + strip_jsonc, +) +from docs_audit.index import CodeIndex # noqa: E402 + + +@pytest.fixture +def repo(tmp_path: Path) -> Path: + (tmp_path / 'pkg').mkdir() + (tmp_path / 'pkg' / 'real.py').write_text("PATH = 'built_at_runtime.json'\n", encoding='utf-8') + return tmp_path + + +def _classify(text: str, repo: Path, doc: str = 'docs/guide.md') -> list: + index = CodeIndex.build(repo) + lines = text.splitlines() + return [classify(c, index, lines) for c in extract(text, doc)] + + +def test_extract_finds_path_citations() -> None: + found = extract('See `pkg/real.py` for details.', 'docs/guide.md') + assert [c.token for c in found] == ['pkg/real.py'] + + +def test_extract_ignores_fenced_blocks() -> None: + text = '```\n`pkg/inside_fence.py`\n```\n`pkg/outside.py`\n' + assert [c.token for c in extract(text, 'd.md')] == ['pkg/outside.py'] + + +def test_verified_when_path_exists(repo: Path) -> None: + (verdict,) = _classify('See `pkg/real.py`.', repo) + assert verdict.verdict == VERIFIED + + +def test_verified_by_basename_when_path_is_loose(repo: Path) -> None: + """`real.py` alone is loosely worded, not wrong -- it must not be deleted.""" + (verdict,) = _classify('See `real.py`.', repo) + assert verdict.verdict == VERIFIED + + +def test_orphaned_when_nothing_matches(repo: Path) -> None: + (verdict,) = _classify('See `pkg/ghost_module.py`.', repo) + assert verdict.verdict == ORPHANED + + +def test_placeholder_when_prose_says_create(repo: Path) -> None: + """Regression: docs naming a file the READER creates are not stale.""" + (verdict,) = _classify('Create the entry point (`chat.pipe`).', repo) + assert verdict.verdict == PLACEHOLDER + + +def test_placeholder_inside_tree_diagram(repo: Path) -> None: + """Regression: scaffolding trees name template files, not repo files.""" + text = 'Layout:\n └── `src/MyApp.tsx` # client area\n' + verdicts = [v for v in _classify(text, repo) if v.citation.token == 'src/MyApp.tsx'] + assert verdicts and verdicts[0].verdict == PLACEHOLDER + + +def test_placeholder_save_this_as(repo: Path) -> None: + """Regression: 'Save this as X' is a create instruction, not a claim.""" + (verdict,) = _classify('Save this as `extract.pipe`:', repo) + assert verdict.verdict == PLACEHOLDER + + +def test_placeholder_naming_illustration(repo: Path) -> None: + """Regression: 'Examples: `a.pipe`' illustrates a convention.""" + (verdict,) = _classify('**Examples:** `document_processor.pipe`', repo) + assert verdict.verdict == PLACEHOLDER + + +def test_counter_example_is_never_orphaned(repo: Path) -> None: + """Regression: deleting a 'NOT: `x`' line reintroduces the very mistake + the doc exists to prevent. This is the highest-cost false positive. + """ + (verdict,) = _classify('- **NOT:** `.json` or `.pipeline.json`', repo) + assert verdict.verdict == PLACEHOLDER + + +def test_historical_doc_is_protected(repo: Path) -> None: + """A changelog naming a deleted file is correct by definition.""" + (verdict,) = _classify('Removed `pkg/deleted_thing.py`.', repo, doc='CHANGELOG.md') + assert verdict.verdict == HISTORICAL + + +def test_runtime_path_built_by_code_is_protected(repo: Path) -> None: + """Regression: a file that only exists at runtime is still documented correctly.""" + (verdict,) = _classify('Writes `built_at_runtime.json`.', repo) + assert verdict.verdict == RUNTIME + assert 'pkg/real.py' in verdict.evidence + + +def test_extensionless_segment_is_not_runtime_evidence(repo: Path) -> None: + """Regression: matching only the basename let a citation ending in a common + word be protected by that word appearing anywhere in source. `pkg/real.py` + contains "json", which must not make `tools/json` look like a built path. + (`tools/` prefix so the extractor picks the token up as a directory span.) + """ + (verdict,) = _classify('Call `tools/json` to fetch it.', repo) + assert verdict.verdict == ORPHANED + + +def test_basename_fallback_needs_an_extension(repo: Path) -> None: + """The complement: a doc supplying the directory for a file the code refers + to by name is still protected, because the basename looks like a filename. + """ + (verdict,) = _classify('State lives in `build/built_at_runtime.json`.', repo) + assert verdict.verdict == RUNTIME + assert 'pkg/real.py' in verdict.evidence + + +def test_tool_installed_workspace_path_is_protected(repo: Path) -> None: + """Regression: `.rocketride/` is written into a *user's* workspace by the + VS Code installer, so a doc telling a reader to open a file under it is + correct precisely because this repo does not contain it. Treating those as + orphaned produced 8 false positives on rocketride-workshops alone. + """ + (verdict,) = _classify('Read `.rocketride/docs/ROCKETRIDE_README.md` first.', repo) + assert verdict.verdict == RUNTIME + assert 'installed into the workspace' in verdict.evidence + + +def test_emitted_artifact_is_not_orphaned(repo: Path) -> None: + """Regression: prose describing what a pipeline emits at run time names a + file that is absent at rest by definition. "emits" was missing from the + create-verb set, so those citations were reported for deletion. + """ + (verdict,) = _classify('Emits `ARCHITECTURE.md` as an inline content block.', repo) + assert verdict.is_protected + + +def _node(root: Path, name: str, fields: dict, readme: str | None) -> Path: + node = root / 'nodes' / 'src' / 'nodes' / name + node.mkdir(parents=True) + (node / 'impl.py').write_text('x = 1\n', encoding='utf-8') + (node / 'services.json').write_text(json.dumps({'fields': fields}), encoding='utf-8') + if readme is not None: + (node / 'README.md').write_text(readme, encoding='utf-8') + return node + + +def _block(*keys: str) -> str: + rows = '\n'.join(f'| `{k}` | `string` | desc | |' for k in keys) + return f'\n{rows}\n\n' + + +def test_profile_groups_are_not_params(tmp_path: Path) -> None: + """Regression: `object`/`properties` entries are groupings, not settable params. + + Counting them made 8 clean nodes report phantom drift. + """ + node = _node( + tmp_path, + 'grouped', + { + 'model': {'type': 'string', 'title': 'Model'}, + 'grouped.fast': {'object': 'fast', 'properties': ['model']}, + }, + _block('model'), + ) + assert schema_params(node) == {'model'} + assert audit_node(node, tmp_path) == [] + + +def test_stale_params_detected_when_block_misses_a_real_param(tmp_path: Path) -> None: + node = _node( + tmp_path, + 'drifted', + {'a': {'type': 'string'}, 'b': {'type': 'boolean'}}, + _block('a'), + ) + (gap,) = audit_node(node, tmp_path) + assert gap.kind == STALE_PARAMS + assert 'b' in gap.detail + + +def test_missing_doc_for_node_with_code_and_no_readme(tmp_path: Path) -> None: + node = _node(tmp_path, 'undocumented', {'a': {'type': 'string'}}, readme=None) + (gap,) = audit_node(node, tmp_path) + assert gap.kind == MISSING_DOC + + +def test_node_without_python_is_not_a_gap(tmp_path: Path) -> None: + node = tmp_path / 'nodes' / 'src' / 'nodes' / 'assets_only' + node.mkdir(parents=True) + (node / 'icon.svg').write_text('', encoding='utf-8') + assert audit_node(node, tmp_path) == [] + + +def test_jsonc_services_file_is_parsed(tmp_path: Path) -> None: + """Regression: several nodes ship `//`-commented schemas. Failing to parse + them silently reported the node as having zero params, so real drift could + never surface. + """ + node = tmp_path / 'nodes' / 'src' / 'nodes' / 'commented' + node.mkdir(parents=True) + (node / 'impl.py').write_text('x = 1\n', encoding='utf-8') + (node / 'services.json').write_text( + '{\n\t//\n\t// Required:\n\t//\n\t"fields": {"a": {"type": "string"}}\n}\n', + encoding='utf-8', + ) + assert schema_params(node) == {'a'} + + +def test_typeless_field_counts_as_declared(tmp_path: Path) -> None: + """Regression: the audit required a ``type`` key, but nodes:docs-generate + only skips entries carrying ``object``. A typeless field (rendered by the + generator with an empty Type cell) was therefore counted as documented but + not declared, reporting phantom drift on ten real nodes. + """ + node = _node( + tmp_path, + 'typeless', + { + 'vector.local.host': {'default': 'localhost'}, # no "type" key + 'vector.profile': {'type': 'string'}, + 'vector.group': {'object': 'grp', 'properties': ['vector.profile']}, + }, + _block('vector.local.host', 'vector.profile'), + ) + assert schema_params(node) == {'vector.local.host', 'vector.profile'} + assert audit_node(node, tmp_path) == [] + + +def test_malformed_schema_is_reported_not_skipped(tmp_path: Path) -> None: + """Regression: a broken services.json yielded zero declared params, which is + indistinguishable from a node with nothing to document -- so the audit + reported the node as clean while hiding whatever the schema really said. + """ + node = tmp_path / 'nodes' / 'src' / 'nodes' / 'broken' + node.mkdir(parents=True) + (node / 'impl.py').write_text('x = 1\n', encoding='utf-8') + (node / 'services.json').write_text('{"fields": {', encoding='utf-8') + (node / 'README.md').write_text(_block('a'), encoding='utf-8') + + (gap,) = audit_node(node, tmp_path) + assert gap.kind == UNREADABLE + assert 'services.json' in gap.detail + + +def test_dotfile_paths_keep_their_leading_dot(tmp_path: Path) -> None: + """Regression: `.lstrip('./')` strips a character SET, not a prefix, so + `.env` was indexed as `env` and citations to hidden files looked orphaned. + """ + (tmp_path / '.env').write_text('K=v\n', encoding='utf-8') + (tmp_path / '.github' / 'workflows').mkdir(parents=True) + (tmp_path / '.github' / 'workflows' / 'ci.yml').write_text('on: push\n', encoding='utf-8') + + index = CodeIndex.build(tmp_path) + assert index.has_path('.env') + assert index.has_path('.github/workflows/ci.yml') + assert not index.has_path('env') + + +def test_unreadable_doc_is_reported_and_fails(tmp_path: Path) -> None: + """Regression: audit_doc returned [] on OSError, so a doc that could not be + read vanished from the audit and the run still reported success -- the same + silent-green shape as a mistyped --root. + """ + (tmp_path / 'docs').mkdir() + doc = tmp_path / 'docs' / 'unreadable.md' + doc.write_text('See `pkg/real.py`.\n', encoding='utf-8') + doc.chmod(0o000) + try: + if os.access(doc, os.R_OK): # running as root ignores the mode bits + pytest.skip('cannot make a file unreadable as this user') + index = CodeIndex.build(tmp_path) + (verdict,) = audit_doc(doc, tmp_path, index) + assert verdict.verdict == UNREADABLE_DOC + assert main(['--root', str(tmp_path)]) == 1 + finally: + doc.chmod(0o644) + + +def test_nonexistent_root_fails_instead_of_passing_green(tmp_path: Path) -> None: + """A typo in --root must not look like a clean audit.""" + assert main(['--root', str(tmp_path / 'nope'), '--fail-on-orphaned']) == 2 + + +def test_strip_jsonc_keeps_urls_inside_strings() -> None: + """A `//` inside a quoted value is data, not a comment.""" + kept = strip_jsonc('{"url": "https://example.com/x"} // trailing') + assert 'https://example.com/x' in kept + assert 'trailing' not in kept