Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,9 @@ HocusPocus exists for creators who do not want a one-off prompt machine. It is a
2. **Director (sidebar)** — select Music Video, Short Film, Trailer or a story-driven workflow and describe the outcome. The LLM turns the brief into reviewable shots, prompts and references. Choose manual review for control or automatic mode for a complete recoverable pipeline.
3. **Gallery: All, Images, Videos, Audio, Videoclips, Trailers and Chapters** — browse results by kind. Open an item to inspect it; use it as a reference, send it to an editor, or keep it in the active workspace.
4. **3D** — create a mesh from text, one image or the four front/left/right/back views. You can upload references or select existing HocusPocus images. Export GLB for later animation or 3D-video composition.
5. **3D Video** — place GLBs, images and effects in a controllable scene and render an MP4/WebM composition. Use it for camera moves that must be exact rather than invented by a video model.
5. **3D Video** — place GLBs, images, Character Kits and effects in a controllable scene and render an MP4/WebM composition. Use it for camera moves that must be exact rather than invented by a video model, and for 2D cutout speech (Face Rig + dialogue beats).
6. **Animate** — rig a compatible static GLB and apply procedural or AI-assisted motion. Export the rigged model or bring it back to 3D Video.
7. **Character Creator** — upload one subject image. H3 makes a centered 360° turntable; select/re-take the best front, left, back and right frames, then build a Hunyuan multi-view mesh. It is the quickest route from character art to a 3D asset.
7. **Character Creator** — upload one subject image. H3 makes a centered 360° turntable; select/re-take the best front, left, back and right frames, then build a Hunyuan multi-view mesh. **Create / open CharacterKit Face Rig** hands a saved view to **3D Video** as a 2D puppet (not a mesh). Operator guide: [Character Kits / Face Rig](docs/character-kits/HOWUSEIT.md).
8. **Story Lab** — write the production bible: premise, world rules, locations, cast, relationships and beats. Review each field, then hand the approved canon to Comic Studio, Director, trailers or music-video production.
9. **Series Lab** — turn approved canon into seasons, episodes, scenes and shots. Use it when continuity has to survive several episodes and you need attempts and references tracked per shot.
10. **Workspaces** — separate client work, worlds and experiments. The workspace selector changes the active output folder; this tab is also the place to inspect and resume Director production threads.
Expand Down Expand Up @@ -200,7 +200,7 @@ Included geometry variants:
- **Hunyuan3D 2 Multi-view**: Turbo, Fast, and full-step models using front/left/right/back references
- **Hunyuan3D 2.1**: high-fidelity 3.3B geometry with optional PBR materials

The **Characters** tab builds a Hunyuan multi-view mesh from one subject or object photo: MiniMax-M3 vision writes the A Prompt (`POST /api/v1/characters/describe-refs`), H3 Ref2VA records a frozen 360° orbit (`768x1344`, 124 frames), then four stills at frames 2 / 21 / 42 / 63 become front / left / back / right for `hunyuan3d-2mv-turbo`. Details: **[3D Video compositor](docs/3d-video-compositor/HOWUSEIT.md)** §5.4.
The **Characters** tab builds a Hunyuan multi-view mesh from one subject or object photo: MiniMax-M3 vision writes the A Prompt (`POST /api/v1/characters/describe-refs`), H3 Ref2VA records a frozen 360° orbit (`768x1344`, 124 frames), then four stills at frames 2 / 21 / 42 / 63 become front / left / back / right for `hunyuan3d-2mv-turbo`. Details: **[3D Video compositor](docs/3d-video-compositor/HOWUSEIT.md)** §5.4. The same tab can hand a saved view to **3D Video** as a Character Kit Face Rig (2D cutout, not a mesh): **[Character Kits](docs/character-kits/HOWUSEIT.md)**.

The advanced panel exposes inference steps, guidance, octree resolution, processing chunks, seed, texture model/resolution, CPU offload, FlashVDM, Torch compilation, DMC/Marching Cubes, mesh simplification, face target, and GLB/OBJ/PLY/STL export. Four presets provide sensible Low VRAM, Balanced, Quality/PBR, and Multi-view configurations.

Expand Down Expand Up @@ -439,7 +439,7 @@ After clicking **Start**, the launcher shows an **Open Web UI** button once the
- **Activity footer** — persistent live job progress and access to current or past **Productions**
- **Settings drawer** (gear icon) — model visibility, performance auto-tune, services (LLM, API keys, NSFW, theme)
- **Pinokio menu** — Update, Reset, Install Inpaint Support, LoRA folder shortcuts
- **Operator guides** — [HOWUSEIT index](docs/HOWUSEIT.md) (Video Editor, Workspaces tab, 3D compositor)
- **Operator guides** — [HOWUSEIT index](docs/HOWUSEIT.md) (Video Editor, Workspaces tab, 3D compositor, Character Kits)

## Sharing on the local network

Expand Down
34 changes: 32 additions & 2 deletions app/docs/API.md
Original file line number Diff line number Diff line change
Expand Up @@ -717,13 +717,43 @@ curl -X POST "$MAESTRO_URL/api/v1/characters/describe-refs" \

`kind` is `character` or `object` (default `character`). `roles` are `subject`, `face`, `outfit`, `extra`, or `accessory`; missing/invalid roles become `subject` for index 0 and `extra` otherwise. Response: `{ "a_prompt", "kind" }`. `400` if `image_paths` is empty, a file is missing, or the API key is unset.

## Character Kits

Reusable 2D cutout puppets live in `{workspace}/.character-kit-library-v1.json` (max 100 kits, 20 MB). Writes are compare-and-swap on `revision`. Operator guide: `docs/character-kits/HOWUSEIT.md`.

- `GET /api/v1/character-kits/library?workspace=default` — load or return an empty `{ version: 1, revision: 0, activeId: "", kits: {} }`.
- `PATCH /api/v1/character-kits/library/kits/{kit_id}` — `{ workspace, baseRevision, kit, makeActive? }`. `makeActive` defaults true. `400` invalid kit. `409` `{ "code": "character_kit_revision_conflict", "expectedRevision", "currentRevision" }`.
- `DELETE /api/v1/character-kits/library/kits/{kit_id}` — `{ workspace, baseRevision }`. `404` if missing. Source files are not deleted.
- `POST /api/v1/character-kits/face-rig/cleanup` — `{ workspace, source, padding? }` (`padding` 0–64, default 8). rembg U2Net + crop-to-alpha; writes a new PNG and never overwrites `source`. `400` / `404` if the path is outside uploads/workspace.

Workspace token: `default` or `[A-Za-z0-9][A-Za-z0-9_-]*`. Mouth keys are only `closed|small|wide|round`. `blob:` sources are rejected. `lookNotes` is UI-only and is stripped on save.

```bash
curl -X PATCH "$MAESTRO_URL/api/v1/character-kits/library/kits/luma" \
-H "Content-Type: application/json" \
-d '{
"workspace": "default",
"baseRevision": 0,
"kit": {
"version": 1, "id": "luma", "name": "Luma", "style": "cutout",
"base": {
"id": "luma-base", "name": "Luma base", "source": "luma-base.png",
"kind": "image", "alphaStatus": "transparent", "reviewState": "approved"
},
"poses": {}, "mouth": {}, "eyes": {},
"anchors": { "base": { "mouth": { "offsetX": 0, "offsetY": -18, "scale": 0.05, "rotation": 0 } } },
"provenance": []
}
}'
```

## Director pipeline threads

These routes always use the server active workspace. They do not accept `?workspace=`.

- `GET /api/v1/director/pipelines` / `GET /api/v1/director/pipelines/active` / `GET /api/v1/director/pipelines/{pid}` — list or load. `{pid}` hydrates an empty `clips` array from `clip_plans` or `planned_clips` and sets `queue_source` to `clips`, `clip_plans`, or `planned`.
- `GET /api/v1/director/pipelines` / `GET /api/v1/director/pipelines/active` / `GET /api/v1/director/pipelines/{pid}` — list or load. `{pid}` hydrates an empty `clips` array from `clip_plans` or `planned_clips` and sets `queue_source` to `clips`, `clip_plans`, or `planned`. List accepts `limit` and `offset` (newest first). `limit=0` (the default) returns the full list; the Workspaces tab pages 8 at a time and reads `total` for “load more”.
- `PUT /api/v1/director/pipelines/{pid}/clips/{clip_index}/prompt` — optional `video_prompt`, `image_prompt`, `soundtrack_drive`. `true` writes an `audio_driven` / `lip_sync_critical` plan; `false` writes `music_driven` and clears `_director_dialogue_beats`. `409` while the pipeline is active.
- `POST /api/v1/director/pipeline/{pid}/resume` and `POST /api/v1/director/pipeline/{pid}/continue` use the singular `pipeline` path.
- Batch prompt rewrite is UI-only: loop `POST /api/v1/llm/generate` (local LLM) then PUT the chosen prompts.

Operator notes: `docs/video-editor/HOWUSEIT.md` and `docs/workspaces/HOWUSEIT.md`.
Operator notes: `docs/video-editor/HOWUSEIT.md`, `docs/workspaces/HOWUSEIT.md`, and `docs/character-kits/HOWUSEIT.md`.
13 changes: 12 additions & 1 deletion docs/3d-video-compositor/HOWUSEIT.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,9 @@ A **layered compositor**, not MiniMax H3.
| Scene Animator | **3D Video** | Stack images, videos, GLBs, camera, rain/fog/etc. Animate them. Record a clip |
| MiniMax H3 | Studio / Story Lab | Native video + stereo audio (acting, dialogue, locations) |
| Video Editor | **Video Editor** | Join compositor clips with H3 clips |
| Character Kits | **3D Video** sidebar | Reusable 2D cutout puppets + Face Rig mouths. Operator guide: [Character Kits](../character-kits/HOWUSEIT.md) |

Use the compositor when you need **controllable motion of a known object** over plates: a ship crossing stars, a UFO rising behind mountains, a logo flying in, rain over a still. Use H3 when you need **performance, speech, or a living location**. Mix them: H3 for people/places, compositor for the vehicle insert, Video Editor to cut them together.
Use the compositor when you need **controllable motion of a known object** over plates: a ship crossing stars, a UFO rising behind mountains, a logo flying in, rain over a still. Use H3 when you need **performance, speech, or a living location**. Mix them: H3 for people/places, compositor for the vehicle insert, Video Editor to cut them together. Use a **Character Kit** when the “known object” is a graphic puppet that must speak with four mouth sprites — not a Hunyuan mesh and not H3 lipsync.

Do **not** ask H3 to “keep this exact GLB flying on a perfect path.” H3 will invent a new ship. The compositor keeps the mesh.

Expand Down Expand Up @@ -232,6 +233,14 @@ Import H3 MP4s + recorded WebM. Multi-select in **From Loreframe Lab**. Export M

Director / Series **auto-joins** (not the editor) use a 0.5 s last-frame freeze + ~0.4 s crossfade when there is no driving audio. Full probe/export/`result_kind` contract: [Video Editor / mixes](../video-editor/HOWUSEIT.md).

### 5.9 Character Kits (2D puppets)

Workspace library: `GET/PATCH/DELETE /api/v1/character-kits/library…`. Overlay matte: `POST /api/v1/character-kits/face-rig/cleanup`. Character Creator can hand a saved view into this editor (`hocuspocus:character-kit-face-rig-handoff`) but does not generate visemes itself.

Only **approved** poses/mouths mount into the scene or appear in the Recipe inventory (`APPROVED_CHARACTER_KIT`). Spoken cutout dialogue is `scene.dialogueBeats` compiled to hold/snap opacity keyframes — not phoneme-perfect lipsync.

Full contract, CAS revisions, mouth packs, and pitfalls: [Character Kits / Face Rig](../character-kits/HOWUSEIT.md).

---

## 6. Layer cookbook (what to stack)
Expand Down Expand Up @@ -421,6 +430,7 @@ Do this **in the browser tab**, not as a Python overnight script. UI: `SceneReci
4. **Auto**: **Generate + compose** creates missing plates/meshes. One `identity` per object — a UFO series uses **one** GLB and several `shots[]`. Static environments use image plates; inherently moving scenery can use an H3 video plate. Rain, fog, snow and particles use procedural effects instead of redundant generated overlays. Default `record`/`save` are false so you preview first.
5. GPU jobs and Hunyuan poll with timeouts; **Cancel** aborts the run. If Lab dies (segfault), the runner errors instead of spinning forever.
6. After compose, switch shots in the recipe panel without regenerating the mesh. A recipe rig `clip` is mounted into the Scene Animator and disables unintended turntable spin.
7. Approved Character Kits enter inventory as `APPROVED_CHARACTER_KIT` rows. Spoken cutout shots need both a `speech` audio entry and a `dialogueBeats` row whose `mouthLayerIds` name the overlays (for `cutout-talking-head`: `["mouth-open", "mouth-closed"]`). Never mix one kit’s mouth with another kit’s body.

Keep the 3D Video tab visible while it records. WebM is the recorded format; import it in Video Editor to join with H3 MP4s.

Expand All @@ -440,6 +450,7 @@ Keep the 3D Video tab visible while it records. WebM is the recorded format; imp
| `app/services/rig_service.py` | Rig profiles and clips |
| `docs/minimax-h3-prompting.md` | H3 prompt dialect |
| `docs/video-editor/HOWUSEIT.md` | Cut compositor WebM with H3 MP4s; mix kinds |
| `docs/character-kits/HOWUSEIT.md` | Character Kit library, Face Rig, cutout dialogue |
| `ui/src/features/characters/orbitPrompt.ts` | Orbit A/B prompts and still-frame indices |

When in doubt: **one identity per mesh, one path per compositor shot, H3 never draws that mesh.**
1 change: 1 addition & 0 deletions docs/HOWUSEIT.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,3 +5,4 @@ Operator guides for Loreframe Lab subsystems. Prefer these over inventing a seco
- **Video Editor** (import Lab clips, timeline, export) and **assembled mixes** (`result_kind` gallery tabs): [`docs/video-editor/HOWUSEIT.md`](video-editor/HOWUSEIT.md)
- **Workspaces tab** (Director generation threads — not the output-directory switcher): [`docs/workspaces/HOWUSEIT.md`](workspaces/HOWUSEIT.md)
- **3D Video compositor** (Hunyuan meshes + plates + camera + rain/fog, mixed with MiniMax H3): [`docs/3d-video-compositor/HOWUSEIT.md`](3d-video-compositor/HOWUSEIT.md)
- **Character Kits / Face Rig** (workspace 2D puppets, viseme overlays, cutout dialogue): [`docs/character-kits/HOWUSEIT.md`](character-kits/HOWUSEIT.md)
Loading
Loading