Skip to content

Add reusable character voices from audio imports and microphone recordings - #417

Open
IAnMove wants to merge 3 commits into
developmentfrom
feat/custom-character-voices
Open

IAnMove wants to merge 3 commits into
developmentfrom
feat/custom-character-voices

Conversation

@IAnMove

@IAnMove IAnMove commented Sep 14, 2026

Copy link
Copy Markdown
Owner

Character Creator could only store Qwen3 presets and hid their original language and timbre. It now supports reusable voices from imported audio or microphone recordings, plus Spanish/English auditions before configuring mouths or saving a character.

  • Import a clean 3–30 second sample (up to 20 MB), or record it, add a name and the exact transcript, and save the character. Other characters can select the saved voice; new or regenerated native 2D/3D dialogue uses Qwen3 Base with that reference. Microphone takes retain the first 30 seconds if recorder timing overruns; imported recordings are never silently cropped.
  • Show the nine presets’ original language/profile and timbre, retaining their IDs and saved directions. Auditions use the existing generation queue and only run on an explicit click.
  • Store recordings as persistent local uploads. Resolve the declared root and source workspace exactly before generation, including colliding filenames. Reusable metadata contains public references, not machine-specific paths.
  • Block saving/auditioning incomplete voices and changing the character during capture. Replacing a recording clears its old transcript; cancellation releases microphone tracks, including late permission grants, and cancels only the owned audition.

Recording requires HTTPS or localhost; importing also works over LAN HTTP.

Validation: the full 1,825-test UI suite, 93 focused Python tests, production build, catalogs, bundle budget, changed-source lint and the exact development code-health comparison passed. Follow-up coverage includes 89 CI inventory/architecture/voice contracts and six import/capture/timing regression tests; latest TypeScript and lint pass. The new Python test is registered in CI and the architecture reader inventory. GitHub CI is rerunning the final capture-timing adjustment.

A real browser imported the bundled speech sample, stored it through the existing upload API and played the resulting 4.27-second WAV. Real TTS audition was blocked by CUDA out of memory while another process occupied the GPU. Microphone permission/recording/cancellation and persistence contracts are covered by automated tests.

Integration with the independent release fix in #418 was checked in an isolated checkout: #417 at 8cdc68c4 and #418 at 5a5a069e combine without conflicts and pass TypeScript plus 49 focused tests. Independent review identified the capture/switching race, which is fixed and covered by a regression test.

@github-actions

github-actions Bot commented Sep 14, 2026

Copy link
Copy Markdown

PR Review — Loreframe Studio

Risk: medium
Scope: 25 file(s); +1074/-34; React UI, backend services, repo scripts / CI

Automated review from scripts/analyze_pr.py. This is a heuristic pass (no LLM) so humans still own the merge decision.

Findings

  • medium — Dangerous dynamic execution (tests/test_custom_character_voice.py)
    tests/test_custom_character_voice.py adds exec(). Confirm the input is trusted and sandboxed.
  • medium — Dangerous dynamic execution (ui/src/lib/characterVoice.ts)
    ui/src/lib/characterVoice.ts adds exec(). Confirm the input is trusted and sandboxed.
  • low — UI changed — rebuild before merge
    Run cd ui && npm run build (CI already does this). Pinokio Update rebuilds for end users; keep ui/dist untracked.

Changed files

  • added: tests/test_custom_character_voice.py, ui/src/features/characters/CharacterVoiceAudition.tsx, ui/src/features/characters/CustomCharacterVoiceFields.tsx, ui/src/features/characters/characterVoiceReference.ts, ui/src/features/characters/useVoiceReferenceCapture.ts, ui/src/lib/characterVoiceCatalog.ts, ui/tests/characterVoiceAudition.test.tsx, ui/tests/customCharacterVoice.test.ts, ui/tests/customCharacterVoiceCaptureScope.test.tsx, ui/tests/customCharacterVoiceFields.test.tsx
  • modified: README.md, app/_launch_runtime.py, app/services/character_speech_definition.py, app/services/media_paths.py, scripts/ci_test_groups.json, tests/fixtures/architecture_wire_inventory.json, ui/src/features/characters/CharacterDefinitionEditor.tsx, ui/src/features/characters/CharacterKitSummary.tsx, ui/src/features/characters/CharacterVoiceFields.tsx, ui/src/features/scene3d/speech/microphone.ts, ui/src/i18n/locales/en/scene3dEditor.json, ui/src/i18n/locales/es/scene3dEditor.json, ui/src/lib/characterKit.ts, ui/src/lib/characterVoice.ts, ui/src/lib/sceneSpeech.ts

CONTRIBUTING checklist

  • python scripts/verify_clean_repo.py
  • python -m compileall -q app/services app/launch.py scripts
  • cd ui && npm run build if the UI changed
  • No weights, CivitAI sidecars, or generated guides
  • Stays local-first (no required accounts / telemetry)

Posted by the repo PR review workflow. Re-runs on each push to the PR.

@github-actions

github-actions Bot commented Sep 14, 2026

Copy link
Copy Markdown

Code health

Quality score: 64.5/100

Higher is better. The score is a trend dashboard; the independent ratchet below remains the CI gate.

Component Weight Current Change
Cyclomatic health 45% 59.6 -0.1
File concentration 25% 74.4 +0.0
Oversized-file debt 20% 50.2 +0.0
Modularity 10% 90.5 +0.2

Change vs PR base: +0.0 points.

Metric Value
Production LOC 317,491
Production files 1,114
Test LOC 131,757
Functions measured 21,930
Functions complexity ≥ 15 916
Maximum complexity 674
Policy code-health-policy-v1
HEAD 4077fc60f5f62fedce39892aba48557176e767b6
Base 65dfca3fb5cf24dfc8ec2af059058e10c75c7a27
UI measurement complete

Markdown, JSON catalogs and tests are out of this table. Only app/ runtime + ui/src TS/JS count.

Most complex functions

Complexity Where
674 app/wgp.py:7230 generate_video
382 ui/src/stores/useStore.ts:4129 Async method 'startGeneration'
355 app/_launch_runtime.py:23675 _run_generation
308 app/wgp.py:12372 generate_video_tab
269 ui/src/components/Sidebar/SceneAnimatorPanel.tsx:510 Function 'SceneAnimatorPanel'
264 ui/src/stores/useStore.ts:8684 Async method 'loadSettingsFromOutput'
258 app/services/director/planners/short_film.py:3433 ShortFilmPlanner._plan_story_driven
248 app/services/director_pipeline.py:13616 _run_video_generation
245 app/services/director_pipeline.py:7716 _run_pipeline
233 ui/src/features/agent/agentActions.ts:1155 Function 'parseAction'
226 app/services/director_pipeline.py:6545 update_comic_preview
225 ui/src/features/agent/agentActions.ts:3004 Async function 'executeAgentActions'

Trend vs baseline

Metric Δ
Production LOC +354
Test LOC +533
Functions ≥ 15 +3
Maximum complexity +0

Warnings

  • production LOC increased by +354
  • functions at complexity >= 15 increased by +3
  • complexity hotspot app/services/media_paths.py rose 27 -> 28
  • complexity hotspot ui/src/features/characters/CharacterDefinitionEditor.tsx rose 23 -> 28
  • complexity hotspot ui/src/lib/sceneSpeech.ts rose 18 -> 19
  • hotspot app/_launch_runtime.py increased by +1 lines

Ratchet passed.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

Fix All in Cursor

Bugbot Autofix prepared a fix for the issue found in the latest run.

  • ✅ Fixed: Full recordings fail duration check
    • Se confirmó el desajuste entre el tope de 30s y duration > 30; ahora se acepta hasta 30.5s para el overshoot de timeslice/decode y se sigue rechazando 31s+.

Create PR

Or push these changes by commenting:

@cursor push 8b7b8875fa
Preview (8b7b8875fa)
diff --git a/ui/src/features/characters/characterVoiceReference.ts b/ui/src/features/characters/characterVoiceReference.ts
--- a/ui/src/features/characters/characterVoiceReference.ts
+++ b/ui/src/features/characters/characterVoiceReference.ts
@@ -9,7 +9,7 @@
   try { buffer = await new OfflineAudioContext(1, 1, 24000).decodeAudioData(await blob.arrayBuffer()) }
   catch { throw new Error('voiceFileDecode') }
   signal.throwIfAborted()
-  if (!Number.isFinite(buffer.duration) || buffer.duration < 3 || buffer.duration > 30) throw new Error('voiceFileDuration')
+  if (!Number.isFinite(buffer.duration) || buffer.duration < 3 || buffer.duration > 30.5) throw new Error('voiceFileDuration')
   const wav = await voiceWav(buffer)
   signal.throwIfAborted()
   const uploaded = await uploadLocalAsset(new File([wav], 'voice-reference.wav', { type: 'audio/wav' }), signal)

diff --git a/ui/tests/customCharacterVoiceFields.test.tsx b/ui/tests/customCharacterVoiceFields.test.tsx
--- a/ui/tests/customCharacterVoiceFields.test.tsx
+++ b/ui/tests/customCharacterVoiceFields.test.tsx
@@ -61,6 +61,24 @@
   assert.equal(uploads, 1, 'selecting a saved voice neither uploads nor generates')
 })
 
+test('a decoded duration just over the 30s stop timer still uploads the recording', async t => {
+  const { render, fireEvent, waitFor, cleanup } = await import('@testing-library/react')
+  const { CustomCharacterVoiceFields } = await import('../src/features/characters/CustomCharacterVoiceFields')
+  mockDecode(t, 30.25)
+  const original = globalThis.fetch; let uploads = 0
+  globalThis.fetch = async () => {
+    uploads++
+    return json({ filename: 'cap.wav', url: '/api/v1/uploads/cap.wav', path: '/server/private/cap.wav' })
+  }
+  t.after(() => { cleanup(); globalThis.fetch = original })
+  const changes: CustomCharacterVoice[] = []
+  const view = render(<CustomCharacterVoiceFields value={saved} onChange={value => changes.push(value)} />)
+  fireEvent.change(view.getByLabelText('Import audio sample'), { target: { files: [new File(['audio'], 'cap.wav', { type: 'audio/wav' })] } })
+  await waitFor(() => assert.equal(uploads, 1))
+  assert.equal(changes[0].referenceAudio, '/api/v1/uploads/cap.wav')
+  assert.equal(view.queryByText(/The sample must last 3–30 seconds/), null)
+})
+
 test('invalid duration leaves the previous voice intact and does not upload a silently cropped recording', async t => {
   const { render, fireEvent, cleanup } = await import('@testing-library/react')
   const { CustomCharacterVoiceFields } = await import('../src/features/characters/CustomCharacterVoiceFields')

You can send follow-ups to the cloud agent here.

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 3363f69. Configure here.

try { buffer = await new OfflineAudioContext(1, 1, 24000).decodeAudioData(await blob.arrayBuffer()) }
catch { throw new Error('voiceFileDecode') }
signal.throwIfAborted()
if (!Number.isFinite(buffer.duration) || buffer.duration < 3 || buffer.duration > 30) throw new Error('voiceFileDuration')

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Full recordings fail duration check

Medium Severity

A microphone take that runs until the advertised 30-second cap is rejected as too long. recordMicrophone stops on a 30s timer, then MediaRecorder still emits its current 250ms timeslice, so decoded buffer.duration often lands just above 30 and uploadVoiceReference throws voiceFileDuration. The user loses the recording after waiting the full allowed time.

Additional Locations (2)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 3363f69. Configure here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant