Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions docs/README-n8n.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,9 +14,9 @@ Combining the two gives **round-trips**: RocketRide → n8n → RocketRide.
> (n8n side, importable).
>
> **Runnable test pipes** that exercise every mode (sync / async / sequential / agent / round-trip)
> live in [`examples/n8n/`](../examples/n8n/) — open them in the IDE. They pair with the local
> test harness in `.context/n8n-test/` (`run.sh --keep` seeds the `rr-echo` / `rr-slow` / `rr-upper`
> / `rr-callback` workflows); see that folder's `WALKTHROUGH.md` for the step-by-step.
> live in [`examples/n8n/`](../examples/n8n/) — open them in the IDE. Import
> [`n8n-dispatch.workflow.json`](../examples/n8n/n8n-dispatch.workflow.json) on the n8n side and
> point each pipe's webhook URL at your own instance.

---

Expand Down
52 changes: 52 additions & 0 deletions nodes/src/nodes/background_removal/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
# background_removal

A RocketRide image-filter node that separates foreground from background and emits an RGBA cutout.

## What it does

Receives an image stream and runs **BiRefNet** (MIT) to produce an alpha matte, then
composites an RGBA cutout with a **straight (non-premultiplied) alpha** channel, so
downstream nodes can re-composite over any background without dark fringes.

Per frame the node emits on two lanes:

- `image` — the RGBA cutout as PNG
- `text` — JSON alpha statistics (`mean_alpha`, `alpha_coverage_pct`)

Before inference the source is downscaled so its long edge is at most `background_removal.maxEdge`, which
bounds memory use; the alpha matte is then restored to the original resolution for
compositing. `background_removal.maxEdge` is clamped to 256–4096 (default 1024) regardless of what is
configured.

Two profiles ship: the default 1K BiRefNet, and a 2K high-resolution variant for fine
hair and detailed edges. The model runs on CPU, Apple Silicon (MPS), or CUDA. Local
inference serializes GPU access behind a shared device lock; when the engine is started
with `--modelserver`, inference is dispatched to the model server instead.

---

## Configuration

### Lanes

| Lane | Direction | Description |
|------|-----------|-------------|
| `image` | input | Source image (streamed) |
| `image` | output | RGBA cutout PNG, straight alpha |
| `text` | output | JSON alpha stats: `mean_alpha`, `alpha_coverage_pct` |

<!-- ROCKETRIDE:GENERATED:PARAMS START -->
<!-- Generated by nodes:docs-generate. Do not edit by hand. -->

## Schema

| Field | Type | Description | Default |
|---|---|---|---|
| `background_removal.maxEdge` | `number` | **Max input edge (px)**<br/>Downscale source so long edge <= this value before inference; alpha is upsampled back to the (capped) source size for compositing. Lower = faster + less VRAM; higher = sharper edges. | `1024` |
| `background_removal.model` | `string` | **Model**<br/>HuggingFace model identifier for background removal (overrides the profile default) | |
| `background_removal.profile` | `string` | **Model**<br/>BiRefNet variant — default is 1K, HR is 2K for finer edges. | `"birefnet-default"` |

## Source

[<svg viewBox="0 0 16 16" width="15" height="15" fill="currentColor" aria-hidden="true" style="vertical-align:-0.15em;margin-right:0.35em"><path d="M8 0C3.58 0 0 3.58 0 8c0 3.54 2.29 6.53 5.47 7.59.4.07.55-.17.55-.38 0-.19-.01-.82-.01-1.49-2.01.37-2.53-.49-2.69-.94-.09-.23-.48-.94-.82-1.13-.28-.15-.68-.52-.01-.53.63-.01 1.08.58 1.23.82.72 1.21 1.87.87 2.33.66.07-.52.28-.87.51-1.07-1.78-.2-3.64-.89-3.64-3.95 0-.87.31-1.59.82-2.15-.08-.2-.36-1.02.08-2.12 0 0 .67-.21 2.2.82.64-.18 1.32-.27 2-.27.68 0 1.36.09 2 .27 1.53-1.04 2.2-.82 2.2-.82.44 1.1.16 1.92.08 2.12.51.56.82 1.27.82 2.15 0 3.07-1.87 3.75-3.65 3.95.29.25.54.73.54 1.48 0 1.07-.01 1.93-.01 2.2 0 .21.15.46.55.38A8.013 8.013 0 0016 8c0-4.42-3.58-8-8-8z"/></svg> View source](https://github.com/rocketride-org/rocketride-server/tree/develop/nodes/src/nodes/background_removal)
<!-- ROCKETRIDE:GENERATED:PARAMS END -->
42 changes: 42 additions & 0 deletions nodes/src/nodes/caption/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
# caption

A RocketRide image-filter node that generates a natural-language caption for an image.

## What it does

Receives an image and runs **Florence-2 Base** (MIT) locally to produce a descriptive
caption on the text lane. Three granularities are exposed via `caption.task`, from shortest to
longest: `caption` (the default), `detailed_caption`, and `more_detailed_caption`.

Runs on CPU, Apple Silicon (MPS), and CUDA with **no API key required** — inference is
local, so images never leave the host.

For object detection use the **Object Detection** (`detect`) node; for reading text in
an image use the **OCR** node. This node describes a scene, it does not localize or
transcribe.

---

## Configuration

### Lanes

| Lane | Direction | Description |
|------|-----------|-------------|
| `image` | input | Source image (streamed) |
| `text` | output | The generated caption |

<!-- ROCKETRIDE:GENERATED:PARAMS START -->
<!-- Generated by nodes:docs-generate. Do not edit by hand. -->

## Schema

| Field | Type | Description | Default |
|---|---|---|---|
| `caption.profile` | `string` | **Model** | `"florence-base"` |
| `caption.task` | `string` | **Granularity**<br/>How detailed the caption should be. | `"caption"` |

## Source

[<svg viewBox="0 0 16 16" width="15" height="15" fill="currentColor" aria-hidden="true" style="vertical-align:-0.15em;margin-right:0.35em"><path d="M8 0C3.58 0 0 3.58 0 8c0 3.54 2.29 6.53 5.47 7.59.4.07.55-.17.55-.38 0-.19-.01-.82-.01-1.49-2.01.37-2.53-.49-2.69-.94-.09-.23-.48-.94-.82-1.13-.28-.15-.68-.52-.01-.53.63-.01 1.08.58 1.23.82.72 1.21 1.87.87 2.33.66.07-.52.28-.87.51-1.07-1.78-.2-3.64-.89-3.64-3.95 0-.87.31-1.59.82-2.15-.08-.2-.36-1.02.08-2.12 0 0 .67-.21 2.2.82.64-.18 1.32-.27 2-.27.68 0 1.36.09 2 .27 1.53-1.04 2.2-.82 2.2-.82.44 1.1.16 1.92.08 2.12.51.56.82 1.27.82 2.15 0 3.07-1.87 3.75-3.65 3.95.29.25.54.73.54 1.48 0 1.07-.01 1.93-.01 2.2 0 .21.15.46.55.38A8.013 8.013 0 0016 8c0-4.42-3.58-8-8-8z"/></svg> View source](https://github.com/rocketride-org/rocketride-server/tree/develop/nodes/src/nodes/caption)
<!-- ROCKETRIDE:GENERATED:PARAMS END -->
43 changes: 43 additions & 0 deletions nodes/src/nodes/depth_estimate/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
# depth_estimate

A RocketRide image-filter node that estimates per-pixel depth from a single image.

## What it does

Runs **Depth Anything V2 Small** (Apache-2.0) for monocular depth estimation and emits
a colorized depth map on the image lane, where **red is near and blue is far**. Depth
statistics (min, max, mean) are emitted as JSON on the text lane.

Pair this with the **Object Detection** (`detect`) node to get a rough distance to each
detected object.

Before inference the input is downscaled so its long edge is at most `depth_estimate.maxEdge`, which
bounds memory use; the dense output is restored to the original resolution afterward.
Runs on CPU, Apple Silicon (MPS), and CUDA.

---

## Configuration

### Lanes

| Lane | Direction | Description |
|------|-----------|-------------|
| `image` | input | Source image (streamed) |
| `image` | output | Colorized depth map (red = near, blue = far) |
| `text` | output | JSON depth statistics: min, max, mean |

<!-- ROCKETRIDE:GENERATED:PARAMS START -->
<!-- Generated by nodes:docs-generate. Do not edit by hand. -->

## Schema

| Field | Type | Description | Default |
|---|---|---|---|
| `depth_estimate.maxEdge` | `number` | **Max input edge (px)**<br/>Downscale input so the long edge <= this value before inference; dense output is upsampled back to original. Lower = faster + less VRAM, higher = sharper depth. | `1024` |
| `depth_estimate.profile` | `string` | **Model** | `"v2-small"` |

## Source

[<svg viewBox="0 0 16 16" width="15" height="15" fill="currentColor" aria-hidden="true" style="vertical-align:-0.15em;margin-right:0.35em"><path d="M8 0C3.58 0 0 3.58 0 8c0 3.54 2.29 6.53 5.47 7.59.4.07.55-.17.55-.38 0-.19-.01-.82-.01-1.49-2.01.37-2.53-.49-2.69-.94-.09-.23-.48-.94-.82-1.13-.28-.15-.68-.52-.01-.53.63-.01 1.08.58 1.23.82.72 1.21 1.87.87 2.33.66.07-.52.28-.87.51-1.07-1.78-.2-3.64-.89-3.64-3.95 0-.87.31-1.59.82-2.15-.08-.2-.36-1.02.08-2.12 0 0 .67-.21 2.2.82.64-.18 1.32-.27 2-.27.68 0 1.36.09 2 .27 1.53-1.04 2.2-.82 2.2-.82.44 1.1.16 1.92.08 2.12.51.56.82 1.27.82 2.15 0 3.07-1.87 3.75-3.65 3.95.29.25.54.73.54 1.48 0 1.07-.01 1.93-.01 2.2 0 .21.15.46.55.38A8.013 8.013 0 0016 8c0-4.42-3.58-8-8-8z"/></svg> View source](https://github.com/rocketride-org/rocketride-server/tree/develop/nodes/src/nodes/depth_estimate)
<!-- ROCKETRIDE:GENERATED:PARAMS END -->
50 changes: 50 additions & 0 deletions nodes/src/nodes/detect/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
# detect

A RocketRide image-filter node that finds objects in a frame and emits bounding boxes.

## What it does

Runs per-frame object detection and emits bounding boxes, labels, and centroids on the
text lane alongside an annotated frame on the image lane.

Two engines are available via `detect.profile`:

- **RF-DETR** (Apache-2.0, default) — a fast **closed-set** detector over the 80 COCO
classes (person, car, dog, and so on).
- **MM-Grounding-DINO** (Apache-2.0 / BSD-3) — the **open-vocabulary** option. Set
`detect.prompt` to detect anything you can name.

`detect.prompt` accepts either a period- or comma-separated class list (`person . car . dog`)
or a described object (`red car`, `person in a hat`), and returns every matching region.
It matches objects and attributes, not spatial relationships.

Useful as a cheap per-frame gate in front of heavier models. For pixel-level masks use
the **Segmentation** (`detect_segment`) node instead.

---

## Configuration

### Lanes

| Lane | Direction | Description |
|------|-----------|-------------|
| `image` | input | Source frame (streamed) |
| `image` | output | Annotated frame with boxes drawn |
| `text` | output | JSON detections: bounding boxes, labels, centroids |

<!-- ROCKETRIDE:GENERATED:PARAMS START -->
<!-- Generated by nodes:docs-generate. Do not edit by hand. -->

## Schema

| Field | Type | Description | Default |
|---|---|---|---|
| `detect.profile` | `string` | **Model** | `"rfdetr"` |
| `detect.prompt` | `string` | **Detection prompt**<br/>Example: "person . car . dog" (period or comma-separated list) or "red car" / "person in a hat" (described object). Returns all matching regions. Matches objects and attributes — not spatial relationships, so "the car on the left" returns all cars, not just the left one. | |
| `detect.threshold` | `number` | **Confidence threshold**<br/>Minimum confidence score (0.0–1.0) to include a detection | `0.3` |

## Source

[<svg viewBox="0 0 16 16" width="15" height="15" fill="currentColor" aria-hidden="true" style="vertical-align:-0.15em;margin-right:0.35em"><path d="M8 0C3.58 0 0 3.58 0 8c0 3.54 2.29 6.53 5.47 7.59.4.07.55-.17.55-.38 0-.19-.01-.82-.01-1.49-2.01.37-2.53-.49-2.69-.94-.09-.23-.48-.94-.82-1.13-.28-.15-.68-.52-.01-.53.63-.01 1.08.58 1.23.82.72 1.21 1.87.87 2.33.66.07-.52.28-.87.51-1.07-1.78-.2-3.64-.89-3.64-3.95 0-.87.31-1.59.82-2.15-.08-.2-.36-1.02.08-2.12 0 0 .67-.21 2.2.82.64-.18 1.32-.27 2-.27.68 0 1.36.09 2 .27 1.53-1.04 2.2-.82 2.2-.82.44 1.1.16 1.92.08 2.12.51.56.82 1.27.82 2.15 0 3.07-1.87 3.75-3.65 3.95.29.25.54.73.54 1.48 0 1.07-.01 1.93-.01 2.2 0 .21.15.46.55.38A8.013 8.013 0 0016 8c0-4.42-3.58-8-8-8z"/></svg> View source](https://github.com/rocketride-org/rocketride-server/tree/develop/nodes/src/nodes/detect)
<!-- ROCKETRIDE:GENERATED:PARAMS END -->
54 changes: 54 additions & 0 deletions nodes/src/nodes/detect_segment/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
# detect_segment

A RocketRide image-filter node that produces pixel-level segmentation masks.

## What it does

Runs pixel-level segmentation with HuggingFace-native engines and emits an annotated
overlay on the image lane plus a Masks JSON payload on the text lane.

Two modes are available:

- **Mask2Former-instance** (MIT, default) — closed-set **instance** masks, one mask per
detected object.
- **Mask2Former-semantic** (MIT) — a per-pixel **class map** over the whole frame.

Accepts a single frame or multiple frames (via `frame_grabber` documents). Input is
downscaled so its long edge is at most `detect_segment.maxEdge` before inference.

For bounding boxes only — which is considerably cheaper — use the **Object Detection**
(`detect`) node.

---

## Configuration

### Lanes

| Lane | Direction | Description |
|------|-----------|-------------|
| `image` | input | Source frame, or multi-frame documents |
| `image` | output | Annotated overlay |
| `text` | output | Masks JSON |

<!-- ROCKETRIDE:GENERATED:PARAMS START -->
<!-- Generated by nodes:docs-generate. Do not edit by hand. -->

## Schema

| Field | Type | Description | Default |
|---|---|---|---|
| `detect_segment.engine` | `string` | **Engine**<br/>Backing engine. Gated by mode: instance -> mask2former-instance; semantic -> mask2former-semantic. | `"mask2former-instance"` |
| `detect_segment.maxEdge` | `number` | **Max input edge (px)**<br/>Downscale source so long edge <= this value before inference; masks are upsampled back to the source size. Lower = faster + less VRAM. | `1024` |
| `detect_segment.mode` | `string` | **Mode**<br/>Segmentation mode. instance: per-instance masks (default). semantic: per-pixel class map. Both use Mask2Former under the hood. | `"instance"` |
| `detect_segment.profile` | `string` | **Profile**<br/>Segmentation preset. Runs on CPU/MPS/CUDA via transformers. | `"mask2former-instance"` |
| `detect_segment.threshold` | `number` | **Confidence threshold**<br/>Minimum score (0.0-1.0) to include a mask | `0.3` |

## Dependencies

- `pycocotools`

## Source

[<svg viewBox="0 0 16 16" width="15" height="15" fill="currentColor" aria-hidden="true" style="vertical-align:-0.15em;margin-right:0.35em"><path d="M8 0C3.58 0 0 3.58 0 8c0 3.54 2.29 6.53 5.47 7.59.4.07.55-.17.55-.38 0-.19-.01-.82-.01-1.49-2.01.37-2.53-.49-2.69-.94-.09-.23-.48-.94-.82-1.13-.28-.15-.68-.52-.01-.53.63-.01 1.08.58 1.23.82.72 1.21 1.87.87 2.33.66.07-.52.28-.87.51-1.07-1.78-.2-3.64-.89-3.64-3.95 0-.87.31-1.59.82-2.15-.08-.2-.36-1.02.08-2.12 0 0 .67-.21 2.2.82.64-.18 1.32-.27 2-.27.68 0 1.36.09 2 .27 1.53-1.04 2.2-.82 2.2-.82.44 1.1.16 1.92.08 2.12.51.56.82 1.27.82 2.15 0 3.07-1.87 3.75-3.65 3.95.29.25.54.73.54 1.48 0 1.07-.01 1.93-.01 2.2 0 .21.15.46.55.38A8.013 8.013 0 0016 8c0-4.42-3.58-8-8-8z"/></svg> View source](https://github.com/rocketride-org/rocketride-server/tree/develop/nodes/src/nodes/detect_segment)
<!-- ROCKETRIDE:GENERATED:PARAMS END -->
50 changes: 50 additions & 0 deletions nodes/src/nodes/face_detection/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
# face_detection

A RocketRide image-filter node that detects faces and optional alignment keypoints.

## What it does

Runs per-frame face detection using **MediaPipe BlazeFace** (Apache-2.0) and emits
axis-aligned bounding boxes for every detected face.

When `face_detection.emit_landmarks` is on (the default) each face also carries 6 coarse,
alignment-grade keypoints: `right_eye`, `left_eye`, `nose_tip`, `mouth_center`,
`right_ear_tragion`, `left_ear_tragion`.

Fast enough to use as a face-presence gate ahead of heavier models, or to drive
face-aware framing and cropping. These are coarse alignment keypoints — this is not a
dense facial-landmark or face-recognition node.

---

## Configuration

### Lanes

| Lane | Direction | Description |
|------|-----------|-------------|
| `image` | input | Source frame (streamed) |
| `image` | output | Annotated frame |
| `text` | output | JSON faces: bounding boxes and, optionally, 6 keypoints each |

<!-- ROCKETRIDE:GENERATED:PARAMS START -->
<!-- Generated by nodes:docs-generate. Do not edit by hand. -->

## Schema

| Field | Type | Description | Default |
|---|---|---|---|
| `face_detection.emit_landmarks` | `boolean` | **Emit 6 alignment keypoints**<br/>Include coarse 6-point keypoints per face (eyes, nose, mouth, ear tragions) for face-aware framing and alignment. | `true` |
| `face_detection.profile` | `string` | **Model** | `"short"` |
| `face_detection.threshold` | `number` | **Confidence threshold**<br/>Minimum detection confidence (0.0-1.0). Default 0.5 - higher than object detect to suppress false faces on textured backgrounds. | `0.5` |

## Dependencies

- `mediapipe` `>=0.10.35`
- `Pillow`
- `numpy`

## Source

[<svg viewBox="0 0 16 16" width="15" height="15" fill="currentColor" aria-hidden="true" style="vertical-align:-0.15em;margin-right:0.35em"><path d="M8 0C3.58 0 0 3.58 0 8c0 3.54 2.29 6.53 5.47 7.59.4.07.55-.17.55-.38 0-.19-.01-.82-.01-1.49-2.01.37-2.53-.49-2.69-.94-.09-.23-.48-.94-.82-1.13-.28-.15-.68-.52-.01-.53.63-.01 1.08.58 1.23.82.72 1.21 1.87.87 2.33.66.07-.52.28-.87.51-1.07-1.78-.2-3.64-.89-3.64-3.95 0-.87.31-1.59.82-2.15-.08-.2-.36-1.02.08-2.12 0 0 .67-.21 2.2.82.64-.18 1.32-.27 2-.27.68 0 1.36.09 2 .27 1.53-1.04 2.2-.82 2.2-.82.44 1.1.16 1.92.08 2.12.51.56.82 1.27.82 2.15 0 3.07-1.87 3.75-3.65 3.95.29.25.54.73.54 1.48 0 1.07-.01 1.93-.01 2.2 0 .21.15.46.55.38A8.013 8.013 0 0016 8c0-4.42-3.58-8-8-8z"/></svg> View source](https://github.com/rocketride-org/rocketride-server/tree/develop/nodes/src/nodes/face_detection)
<!-- ROCKETRIDE:GENERATED:PARAMS END -->
42 changes: 42 additions & 0 deletions nodes/src/nodes/pose_estimation/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
# pose_estimation

A RocketRide image-filter node that estimates human body pose per frame.

## What it does

Runs top-down human pose estimation using **RTMPose** (Apache-2.0) through the `rtmlib`
ONNX wrapper. **RTMDet-nano** performs person detection first, then RTMPose predicts
**17 COCO keypoints** for each person crop.

Accepts an image or a document and emits an annotated frame, with the per-person
keypoint array attached to the document's metadata.

Top-down means cost scales with the number of people in frame; `pose_estimation.max_persons` bounds
that work.

---

## Configuration

### Lanes

| Lane | Direction | Description |
|------|-----------|-------------|
| `image` | input | Source frame or document |
| `image` | output | Annotated frame; keypoint array attached to document metadata |

<!-- ROCKETRIDE:GENERATED:PARAMS START -->
<!-- Generated by nodes:docs-generate. Do not edit by hand. -->

## Schema

| Field | Type | Description | Default |
|---|---|---|---|
| `pose_estimation.max_persons` | `number` | **Max persons per frame**<br/>Cap on persons retained per frame (sorted by detection score). Limits memory + compute on crowd scenes. | `20` |
| `pose_estimation.profile` | `string` | **Model** | `"rtmpose-medium"` |
| `pose_estimation.threshold` | `number` | **Keypoint score threshold**<br/>Minimum per-keypoint confidence (0.0–1.0). Keypoints below this score are skipped when drawing the skeleton. | `0.3` |

## Source

[<svg viewBox="0 0 16 16" width="15" height="15" fill="currentColor" aria-hidden="true" style="vertical-align:-0.15em;margin-right:0.35em"><path d="M8 0C3.58 0 0 3.58 0 8c0 3.54 2.29 6.53 5.47 7.59.4.07.55-.17.55-.38 0-.19-.01-.82-.01-1.49-2.01.37-2.53-.49-2.69-.94-.09-.23-.48-.94-.82-1.13-.28-.15-.68-.52-.01-.53.63-.01 1.08.58 1.23.82.72 1.21 1.87.87 2.33.66.07-.52.28-.87.51-1.07-1.78-.2-3.64-.89-3.64-3.95 0-.87.31-1.59.82-2.15-.08-.2-.36-1.02.08-2.12 0 0 .67-.21 2.2.82.64-.18 1.32-.27 2-.27.68 0 1.36.09 2 .27 1.53-1.04 2.2-.82 2.2-.82.44 1.1.16 1.92.08 2.12.51.56.82 1.27.82 2.15 0 3.07-1.87 3.75-3.65 3.95.29.25.54.73.54 1.48 0 1.07-.01 1.93-.01 2.2 0 .21.15.46.55.38A8.013 8.013 0 0016 8c0-4.42-3.58-8-8-8z"/></svg> View source](https://github.com/rocketride-org/rocketride-server/tree/develop/nodes/src/nodes/pose_estimation)
<!-- ROCKETRIDE:GENERATED:PARAMS END -->
Loading
Loading