Skip to content

feat(video3d): add a real 3D stage beside the 2.5D compositor - #198

Open
IAnMove wants to merge 8 commits into
developmentfrom
feat/video3d-real-scene-mode
Open

feat(video3d): add a real 3D stage beside the 2.5D compositor#198
IAnMove wants to merge 8 commits into
developmentfrom
feat/video3d-real-scene-mode

Conversation

@IAnMove

@IAnMove IAnMove commented Sep 6, 2026

Copy link
Copy Markdown
Owner

What

Adds a sibling 3D stage inside Video 3D without replacing the 2.5D compositor.

  • Default remains the layered model-viewer compositor (stageMode=compositor).
  • Toggle data-testid="video3d-stage-mode" mounts ui/src/features/scene3d/ (three.js + GLTFLoader).
  • World units are metres, Y-up. Clip identity is (index, exact glTF name) — no idle/run/dance inference.
  • Local GLB files are object URLs in the browser only. They are not uploaded and are not in git.
  • G2 resolve_glb_asset stays unmounted. _launch_runtime.py is untouched.

RAM

The previous session OOM’d (full npm install replacing a node_modules symlink, plus tsc -b / eslint of the 3k-line panel). This follow-up:

  • does not reinstall packages
  • disposes WebGL geometries/materials/textures and calls forceContextLoss on unmount
  • caps the 3D canvas at 1280×720 and pixel ratio 1.25
  • revokes blob URLs when replacing a GLB or leaving the stage
  • pauses compositor RAF when entering 3D so both engines are not alive at once
  • reuses a loaded GLB when only the clip binding changes

Tests (bounded, no full UI suite)

cd ui && NODE_OPTIONS=--max-old-space-size=384 ./node_modules/.bin/tsx --test tests/scene3dStage.test.mjs

6/6 pass: compositor frame clock, unknown duration not zero, clip index+exact name, per-slot catalogs, save/reopen, software-frame hashes.

Also: eslint src/features/scene3d --max-warnings=0 (complexity cap 25), node scripts/check-i18n-catalogs.mjs (19 namespaces).

Not run here, to avoid another OOM: tsc -b, full eslint ., validate_local.sh, Playwright, Cursor review. Cursor quota was already exhausted in this line of work.

Out of scope

  • Private Meshy GLB is a local read fixture only (not in this PR).
  • Preview servers 36591/36592 are not the editor and were not restarted.
  • CharacterSpeech / Character Creator / Face Rig / recipes / _launch_runtime.py / ports 42003–42004 are untouched.

Note

Medium Risk
Adds a new WebGL rendering path and refactors asset selection across large compositor panels; impact is mostly client-side but regressions in studio navigation or 2.5D picking are plausible.

Overview
Introduces a separate studio tab (world3d / Video 3D) for a real metres-scale 3D stage, while the existing compositor stays on scene3d (Video 2.5D). Navigation, wizard context, and the procedural 3D spec are updated to describe that split instead of one overloaded panel.

The new ui/src/features/scene3d/ stack adds a three.js editor: cinematic templates (e.g. two-shot, run-loop), camera families, slot transforms, GLB/image assignment from the app or local files (blob URLs only), skeletal clips bound by (index, exact glTF name), and optional infinite cylinder image backdrops for treadmill-style shots. A hocuspocus:world3d-workflow-request hook lets the wizard mount the same template ids later.

AssetExplorerDialog (search, preview, pagination) replaces inline dropdown/thumbnail pickers in the 2.5D SceneAnimatorPanel, SceneRecipePanel, and the 3D workspace. three is added as a direct dependency. Unit tests cover the asset explorer and scene3d clock, clips, templates, and software-render hashes.

Reviewed by Cursor Bugbot for commit d84c835. Configure here.

@cursor

cursor Bot commented Sep 6, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_a9564ae1-c49a-47a6-94af-a40e8091d7d9)

@github-actions

github-actions Bot commented Sep 6, 2026

Copy link
Copy Markdown

PR Review — Loreframe Studio

Risk: medium
Scope: 39 file(s); +2940/-96; React UI, docs

Automated review from scripts/analyze_pr.py. This is a heuristic pass (no LLM) so humans still own the merge decision.

Findings

  • medium — Large pull request
    2940 additions / 96 deletions. Reviewers will have an easier time with smaller, focused PRs.
  • low — UI changed — rebuild before merge
    Run cd ui && npm run build (CI already does this). Pinokio Update rebuilds for end users; keep ui/dist untracked.

Changed files

  • added: ui/src/components/Sidebar/SceneAnimatorExplorer.tsx, ui/src/components/common/AssetExplorerDialog.tsx, ui/src/components/common/assetExplorer.ts, ui/src/features/scene3d/Scene3DEditorPanel.tsx, ui/src/features/scene3d/Scene3DStage.tsx, ui/src/features/scene3d/Scene3DWorkspace.tsx, ui/src/features/scene3d/backdrop.ts, ui/src/features/scene3d/camera.ts, ui/src/features/scene3d/clips.ts, ui/src/features/scene3d/clock.ts, ui/src/features/scene3d/document.ts, ui/src/features/scene3d/exportFlow.ts, ui/src/features/scene3d/exportMp4.ts, ui/src/features/scene3d/gpu.ts, ui/src/features/scene3d/index.ts, ui/src/features/scene3d/publish.ts, ui/src/features/scene3d/softwareRender.ts, ui/src/features/scene3d/templates.ts, ui/src/features/scene3d/types.ts, ui/src/features/scene3d/world3dAgent.ts
    … and 3 more
  • modified: docs/development/PROCEDURAL_3D_SCENE_SPEC.md, ui/package-lock.json, ui/package.json, ui/src/components/MainContent/MainContent.tsx, ui/src/components/MainContent/TabFilter.tsx, ui/src/components/Sidebar/SceneAnimatorPanel.tsx, ui/src/components/Sidebar/SceneRecipePanel.tsx, ui/src/features/agent/wizardContext.ts, ui/src/i18n/locales/en/common.json, ui/src/i18n/locales/en/navigation.json, ui/src/i18n/locales/en/scene3d.json, ui/src/i18n/locales/es/common.json, ui/src/i18n/locales/es/navigation.json, ui/src/i18n/locales/es/scene3d.json, ui/src/lib/navigationCategories.ts, ui/src/types/index.ts

CONTRIBUTING checklist

  • python scripts/verify_clean_repo.py
  • python -m compileall -q app/services app/launch.py scripts
  • cd ui && npm run build if the UI changed
  • No weights, CivitAI sidecars, or generated guides
  • Stays local-first (no required accounts / telemetry)

Posted by the repo PR review workflow. Re-runs on each push to the PR.

@github-actions

github-actions Bot commented Sep 6, 2026

Copy link
Copy Markdown

Code health

Quality score: 53.7/100

Higher is better. The score is a trend dashboard; the independent ratchet below remains the CI gate.

Component Weight Current Change
Cyclomatic health 45% 54.1 +0.4
File concentration 25% 60.3 +0.7
Oversized-file debt 20% 35.7 +0.8
Modularity 10% 71.3 +1.6

Change vs PR base: +0.7 points.

Metric Value
Production LOC 260,049
Production files 650
Test LOC 89,262
Functions measured 16,641
Functions complexity ≥ 15 821
Maximum complexity 670
Policy code-health-policy-v1
HEAD 9fd6c48fb1cd26cf2a980a6b320b472c4e13d6e7
Base dad1b0b0ba0b54b60610a4fab6f791a03d8ab79f
UI measurement complete

Markdown, JSON catalogs and tests are out of this table. Only app/ runtime + ui/src TS/JS count.

Most complex functions

Complexity Where
670 app/wgp.py:7164 generate_video
376 ui/src/stores/useStore.ts:4029 Async method 'startGeneration'
355 app/_launch_runtime.py:23505 _run_generation
308 app/wgp.py:12298 generate_video_tab
271 ui/src/stores/useStore.ts:8572 Async method 'loadSettingsFromOutput'
258 app/services/director/planners/short_film.py:3433 ShortFilmPlanner._plan_story_driven
257 ui/src/components/Sidebar/SceneAnimatorPanel.tsx:478 Function 'SceneAnimatorPanel'
248 app/services/director_pipeline.py:13607 _run_video_generation
245 app/services/director_pipeline.py:7712 _run_pipeline
244 ui/src/features/agent/agentActions.ts:1145 Function 'parseAction'
226 app/services/director_pipeline.py:6541 update_comic_preview
225 ui/src/features/agent/agentActions.ts:2892 Async function 'executeAgentActions'

Trend vs baseline

Metric Δ
Production LOC +2,296
Test LOC +314
Functions ≥ 15 +3
Maximum complexity +0

Warnings

  • production LOC increased by +2,296
  • functions at complexity >= 15 increased by +3
  • hotspot ui/src/components/Sidebar/SceneAnimatorPanel.tsx increased by +25 lines
  • hotspot ui/src/features/agent/wizardContext.ts increased by +1 lines

Ratchet passed.

@IAnMove
IAnMove force-pushed the feat/video3d-real-scene-mode branch from bdd8b9d to 42a559b Compare September 6, 2026 22:19
Video 3D keeps the layered compositor as the default. A sibling three.js
stage (metres, Y-up) plays glTF clips only when index and exact name match.
Local GLB files stay in the browser. WebGL is disposed on unmount, canvas
size is capped, and blob URLs are revoked so the stage does not keep a
second renderer alive after switching back to 2.5D.
Replace filename-only resource dropdowns in 3D Video with a modal grid
(preview, name, creation date, choose/cancel), matching the scene library
overlay. The 2.5D | 3D stage switch sits at the top of the editor so the
mode is visible without hunting 9px buttons.
The 2.5D compositor stays Video 2.5D. A separate Video 3D editor mounts
cinematic templates, object/background pickers, move/scale/yaw, and
predirected camera families. Wizard can later mount the same template ids
via hocuspocus:world3d-workflow-request without sharing the compositor panel.
@IAnMove
IAnMove force-pushed the feat/video3d-real-scene-mode branch from 42a559b to aeacda0 Compare September 6, 2026 22:57
@cursor

cursor Bot commented Sep 6, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_b3bd2b8b-9883-4a78-8ba9-19ba9b24e612)

…drop

The character stays on the spot; the world is an inside-out cylinder whose
UVs scroll. Clip identity stays (index, exact glTF name) — Running is not inferred.
Extract the asset-explorer apply path so SceneAnimatorPanel no longer
grows past the hotspot cap. Type template ids, add the missing musical
camera family copy, and annotate Three.js callbacks so tsc can build
the E2E preview server.
@cursor

cursor Bot commented Sep 7, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_4c31a2c9-87bf-408d-8be7-2867105ac554)

…itor

Paint three.js frames (run-loop stays put, cylinder world scrolls) and encode
H.264 in the editor. Clip identity stays index plus exact glTF name.
Move asset-explorer routing and the narrative setup box out of
SceneAnimatorPanel so the file stays inside the complexity and line
ratchets. Helpers live next to AssetExplorerDialog; the panel only
wires handlers.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant