Skip to content

feat(studio): share native audio and upscale commands - #280

Draft
IAnMove wants to merge 61 commits into
developmentfrom
feat/shared-tool-commands
Draft

feat(studio): share native audio and upscale commands#280
IAnMove wants to merge 61 commits into
developmentfrom
feat/shared-tool-commands

Conversation

@IAnMove

@IAnMove IAnMove commented Sep 9, 2026

Copy link
Copy Markdown
Owner

What changes

Studio speech, music, SFX and Tools upscale use the same durable commands as external MCP clients. Wizard shows the exact request before submission; retrying a transport failure recovers the existing receipt and native task. A deliberate new intention creates another job.

Music preserves lyrics, caption and description, validates native controls and applies seed, steps, guidance and output count visibly. Explicitly bounded authored text in the user request takes precedence over LLM rewrites. References retain source identity and workspace. Executable server contracts supply the catalogs. Audio parsing uses the registered capability without a duplicate fallback.

This draft incorporates development d88c85d5, rechecked by fetch. Current published HEAD: 0220c80c.

SFX adds text-only MMAudio and canonical video-guided sound replacement. Server preparation checks all seven installed dependencies without downloads, records guide identity and inspected duration, and rechecks the admitted task/parameters/guide before native work. Wizard preserves literal descriptions and exposes seed/guidance/text weight; omitted guide preserves selection and explicit null removes it. Corrupt recovery hints no longer hide unrelated pending commands.

Validation

  • Music candidate complete local UI check on 60019e54: 1,381 tests pass, i18n, ESLint, TypeScript, build and bundle budget pass (323,245 / 327,680 B gzip). Code-health ratchet against development d88 passes. No baseline or budget increases. Required CI and both Cursor checks on60019 passed. The earlier0578 module graph failure was corrected without changing its allowlist.
  • SFX: full UI on95819 passes 1,396 tests, translations, lint, types, build and budget (323,516/327,680 B gzip). Final74d29 passes 80 directed Python tests, 12 directed UI tests, inventory5, catalog verification, TypeScript, ESLint and ratchet vsd88. Refactoring split receipt identity, guide timing and content validation; no baseline/budget changes. Required CI on0220 passes; Find critical bugs passes; InternalBugBot pending.
  • Real Music on 60019e54: one Wizard turn, one V2 POST, zero legacy POSTs, task task-generation-fde42ec3. ACE-Step XL generated a fully decoded 20 s stereo WAV at 48 kHz in 81.13 s (59.65 s loading). The official MCP client replayed the identical intention with the same receipt/task/file; one matching task and zero active tasks. Native/MCP/public SHA-256 f09cf7adf288edf273e2ee4d87fed566569dbb2046d6466bb8d692b9a0b3caab.
  • The Music observer lost its first status read during model loading. The admitted job continued and was recovered using GET only. Its original FAIL evidence remains alongside recovered completion and MCP evidence; this is not an uninterrupted runner PASS. Earlier attempts exposed routing, inactive model defaults and LLM newline loss, now covered by directed regressions.
  • Earlier real Tools Wizard/MCP on d62cd30: one Lanczos2 job, decoded 1024×1024 PNG, identical native/UI/MCP/public hashes. Earlier KugelAudio speech: decoded 3.6 s WAV and identical replay/hash. Image evidence from feat(studio): share durable image commands with Wizard and MCP #275 preserved. Earlier inference is not attributed to a later SHA.
  • Python Music contracts: 64 passed; inventory/matrix: 12 passed on 0578 (unchanged backend). Current final corrections reviewed by root; Luna workers exhausted their usage quota, so independent review of this HEAD remains pending.

Remaining work

Real SFX is validated: text-only on74d and video-guided on0220, each through one Wizard admission and official MCP receipt replay. Text WAV3.02s,42.71s process; guided MP4 preserves54 original frames, uses1.8s source duration and replaces audio,40.34s process. Guided output fully decoded, audible in Chromium, and downloaded from the application; native/UI/MCP/public hashes match (ffa64ea26b74e01c82c20b63776d698b970893584e31d4986a740977cf4694c1). Earlier rejected attempt retained; stale H3 options and guide instructions fixed. Local0220 UI1398 tests, lint/types/build/budget and ratchet PASS. P5 remaining modalities and P6–P10 are unfinished. H3 processor reference arrays remain limited by the native scalar settings validator. The observation runner now preserves transient GET errors and continues observing the same admitted job; its original failed Music observation is retained.

Evidence: outputs/wizard-mcp-execution-20260909/. LAN samples: http://192.168.1.87:4192/wizard-mcp-20260909/ . Music: http://192.168.1.87:4192/wizard-mcp-20260909/music/ .

…red-audio-tool-commands

# Conflicts:
#	app/services/image_generation_commands.py
# Conflicts:
#	app/services/image_generation_commands.py
@github-actions

github-actions Bot commented Sep 9, 2026

Copy link
Copy Markdown

PR Review — Loreframe Studio

Risk: medium
Scope: 133 file(s); +21118/-1044; React UI, backend services, docs, repo scripts / CI

Automated review from scripts/analyze_pr.py. This is a heuristic pass (no LLM) so humans still own the merge decision.

Findings

  • medium — Dangerous dynamic execution (tests/test_studio_sfx_native_worker.py)
    tests/test_studio_sfx_native_worker.py adds exec(). Confirm the input is trusted and sandboxed.
  • medium — Dangerous dynamic execution (tests/test_tools_command_runtime.py)
    tests/test_tools_command_runtime.py adds exec(). Confirm the input is trusted and sandboxed.
  • medium — Large pull request
    21118 additions / 1044 deletions. Reviewers will have an easier time with smaller, focused PRs.
  • medium — Very large file change (app/services/studio_music_spec.py)
    app/services/studio_music_spec.py adds 503 lines. Consider splitting the PR.
  • medium — Very large file change (app/services/studio_sfx_spec.py)
    app/services/studio_sfx_spec.py adds 458 lines. Consider splitting the PR.
  • medium — Very large file change (app/services/studio_speech_preparation.py)
    app/services/studio_speech_preparation.py adds 510 lines. Consider splitting the PR.
  • medium — Very large file change (app/services/studio_speech_spec.py)
    app/services/studio_speech_spec.py adds 637 lines. Consider splitting the PR.
  • medium — Very large file change (app/services/tools_upscale_preparation.py)
    app/services/tools_upscale_preparation.py adds 412 lines. Consider splitting the PR.
  • medium — Very large file change (tests/test_tools_upscale_preparation.py)
    tests/test_tools_upscale_preparation.py adds 404 lines. Consider splitting the PR.
  • medium — Very large file change (ui/src/api/generationCommandClient.ts)
    ui/src/api/generationCommandClient.ts adds 727 lines. Consider splitting the PR.
  • medium — Very large file change (ui/src/api/musicCommandCatalog.json)
    ui/src/api/musicCommandCatalog.json adds 1395 lines. Consider splitting the PR.
  • medium — Very large file change (ui/src/api/sfxCommandCatalog.json)
    ui/src/api/sfxCommandCatalog.json adds 535 lines. Consider splitting the PR.
  • medium — Very large file change (ui/src/api/speechCommandCatalog.json)
    ui/src/api/speechCommandCatalog.json adds 1973 lines. Consider splitting the PR.
  • medium — Very large file change (ui/src/api/toolsCommandCatalog.json)
    ui/src/api/toolsCommandCatalog.json adds 491 lines. Consider splitting the PR.
  • medium — Very large file change (ui/src/features/studio/musicGenerationSpec.ts)
    ui/src/features/studio/musicGenerationSpec.ts adds 426 lines. Consider splitting the PR.
  • medium — Very large file change (ui/tests/studioMusicCommandPresentation.test.tsx)
    ui/tests/studioMusicCommandPresentation.test.tsx adds 435 lines. Consider splitting the PR.
  • low — UI changed — rebuild before merge
    Run cd ui && npm run build (CI already does this). Pinokio Update rebuilds for end users; keep ui/dist untracked.

Changed files

  • added: app/routers/studio_music_commands.py, app/routers/studio_sfx_commands.py, app/routers/studio_speech_commands.py, app/routers/tools_upscale_commands.py, app/services/native_generation_operation.py, app/services/studio_music_preparation.py, app/services/studio_music_spec.py, app/services/studio_sfx_commands.py, app/services/studio_sfx_execution.py, app/services/studio_sfx_preparation.py, app/services/studio_sfx_resources.py, app/services/studio_sfx_spec.py, app/services/studio_speech_preparation.py, app/services/studio_speech_resources.py, app/services/studio_speech_spec.py, app/services/tools_upscale_commands.py, app/services/tools_upscale_preparation.py, app/services/tools_upscale_spec.py, docs/development/MUSIC_COMMANDS.md, docs/development/SFX_COMMANDS.md
    … and 71 more
  • modified: app/_launch_runtime.py, app/routers/image_generation_commands.py, app/services/image_generation_commands.py, app/services/image_generation_runtime.py, app/services/studio_image_resources.py, app/services/wangp_submission.py, docs/development/LABS_WIZARD_ACTION_MATRIX.md, tests/fixtures/architecture_wire_inventory.json, tests/fixtures/labs_wizard_action_matrix.json, tests/test_h3_preplan_job_contract.py, ui/e2e/helpers/apiRoutes.ts, ui/e2e/specs/tools-background-removal.spec.ts, ui/src/api/imageCommandCatalog.json, ui/src/api/imageGenerationCommands.ts, ui/src/components/Sidebar/MusicControls.tsx, ui/src/components/Sidebar/SfxControls.tsx, ui/src/components/Sidebar/Sidebar.tsx, ui/src/components/Sidebar/ToolsPanel.tsx, ui/src/features/agent/agentActionTypes.ts, ui/src/features/agent/agentActions.ts
    … and 22 more

CONTRIBUTING checklist

  • python scripts/verify_clean_repo.py
  • python -m compileall -q app/services app/launch.py scripts
  • cd ui && npm run build if the UI changed
  • No weights, CivitAI sidecars, or generated guides
  • Stays local-first (no required accounts / telemetry)

Posted by the repo PR review workflow. Re-runs on each push to the PR.

@github-actions

github-actions Bot commented Sep 9, 2026

Copy link
Copy Markdown

Code health

Quality score: 58.6/100

Higher is better. The score is a trend dashboard; the independent ratchet below remains the CI gate.

Component Weight Current Change
Cyclomatic health 45% 56.1 +0.7
File concentration 25% 66.3 +2.1
Oversized-file debt 20% 42.3 +2.2
Modularity 10% 83.0 +2.0

Change vs PR base: +1.5 points.

Metric Value
Production LOC 282,342
Production files 849
Test LOC 111,026
Functions measured 18,616
Functions complexity ≥ 15 865
Maximum complexity 674
Policy code-health-policy-v1
HEAD 471c3b02f979a88e8c5cd03318437b65c29b7705
Base d88c85d543100dad3469014ff0d4c7bd7b242222
UI measurement complete

Markdown, JSON catalogs and tests are out of this table. Only app/ runtime + ui/src TS/JS count.

Most complex functions

Complexity Where
674 app/wgp.py:7221 generate_video
379 ui/src/stores/useStore.ts:4099 Async method 'startGeneration'
355 app/_launch_runtime.py:23653 _run_generation
308 app/wgp.py:12363 generate_video_tab
268 ui/src/stores/useStore.ts:8641 Async method 'loadSettingsFromOutput'
262 ui/src/components/Sidebar/SceneAnimatorPanel.tsx:496 Function 'SceneAnimatorPanel'
258 app/services/director/planners/short_film.py:3433 ShortFilmPlanner._plan_story_driven
248 app/services/director_pipeline.py:13607 _run_video_generation
245 app/services/director_pipeline.py:7712 _run_pipeline
233 ui/src/features/agent/agentActions.ts:1151 Function 'parseAction'
226 app/services/director_pipeline.py:6541 update_comic_preview
225 ui/src/features/agent/agentActions.ts:2902 Async function 'executeAgentActions'

Trend vs baseline

Metric Δ
Production LOC +8,035
Test LOC +6,627
Functions ≥ 15 +4
Maximum complexity +0

Warnings

  • production LOC increased by +8,035
  • functions at complexity >= 15 increased by +4
  • complexity hotspot ui/src/components/Sidebar/ToolsPanel.tsx rose 21 -> 24
  • complexity hotspot ui/src/features/agent/capabilityRunner.ts rose 23 -> 24
  • hotspot app/_launch_runtime.py increased by +36 lines
  • hotspot ui/src/features/agent/capabilityRegistry.ts increased by +3 lines
  • hotspot ui/src/features/agent/applicationAdapters.ts increased by +20 lines

Ratchet passed.

@IAnMove IAnMove changed the title feat(studio): share native speech, music and upscale commands feat(studio): share native audio and upscale commands Sep 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants