Skip to content

test(app): repeatable live acceptance, usage guide and fixes from real runs - #259

Merged
IAnMove merged 5 commits into
developmentfrom
test/live-app-acceptance
Sep 8, 2026
Merged

test(app): repeatable live acceptance, usage guide and fixes from real runs#259
IAnMove merged 5 commits into
developmentfrom
test/live-app-acceptance

Conversation

@IAnMove

@IAnMove IAnMove commented Sep 8, 2026

Copy link
Copy Markdown
Owner

The acceptance branch now certifies a current end-to-end Wizard pass against the isolated Pinokio QA backend. The visible Ask to the Wizard panel submitted two independent native generations, then the test observed their canonical tasks, downloaded their published outputs, read sidecar metadata and decoded each output in the browser. The app tour also closes responsive Director/Wizard overlays before continuing, so all main destinations can be audited in one session.

This PR also provides isolated browser workspace selection, explicit backend/profile preflight, preserved attempts and failed-test resume, plus an offline usage guide and capability matrix. Its earlier accepted slices fix the upscale route hiding its service import, a cancelled FIFO head blocking successors, and mobile utility controls consuming the category row. These remain part of the final branch diff.

The final live evidence is preserved under outputs/app-acceptance-20260908/:

  • wizard-media-live-v5/attempt-20260908T183522786577Z, workspace e2e_wizard_media_v5_20260908_7a01a2fc_mtt0f5be: Flux 2 Klein 9B image, task/job task-generation-209a8c26 / 209a8c26, command 0abab03d-21ea-41a6-98f8-283d451bf61e, 35 s inference, 137,291 bytes, browser decode 1280x720; ACE-Step v1.5 XL SFT LM_4B music, task/job task-generation-cb282a7f / cb282a7f, command 387e6932-72c1-4ff8-95f5-4041a374495d, 47 s inference, requested and decoded duration 20 s, 3,843,268 bytes.
  • app-tour-live-v5/attempt-20260908T185413143401Z: 52 feature and mobile navigation captures passed in 13 s. Workspace isolation reports the server folder as default before and after the browser-local test folder.

The implementation also fixes three live findings:

  • English Studio Audio generation requests are recognized by the local reconciliation guard, while educational questions such as “Explain how…”, “How can I…”, “How to…”, and model recommendations stay read-only. “Generate a voice…” prepares Speech instead of Music.
  • Selecting an audio model waits for its options before applying an explicit duration; positive values such as 20 seconds survive late model-default responses, while zero remains the auto sentinel.
  • Cold model loading gets a longer observation timeout on the existing task; the test never resubmits a live task after an observation timeout.

Validation on commit 556d77e4:

  • 81 directed UI tests covering agent reconciliation, Studio capabilities, tool guards and restore behavior passed.
  • npm run i18n:check, ESLint with --max-warnings=0, TypeScript/Vite build and the 28-test job-lifecycle unittest suite passed.
  • Direct intent probes cover all five educational English variants and the voice generation route.
  • The real Wizard media test and complete app tour passed on QA 42004 using isolated browser workspaces. Prompts, LLM replies, actions, command traces, task IDs, output metadata, decoded bytes and UI screenshots remain in the evidence artifacts; generated media and model weights are not committed.

Coverage remains representative rather than every model/parameter combination. The live media pass covers Flux 2 Klein 9B and ACE-Step only; broader model coverage is left to the existing simulated/targeted suites. The image evidence attachment is named .png by the generic test label while its canonical published bytes are JPEG; the decoded copy and sidecar metadata preserve the correct JPEG type. The full required CI now passes on 556d77e4, including Python, UI and Chromium E2E; both Cursor automations pass. Independent product and harness reviews approve the current delta within their documented scope. The formal Independent QA publisher remains neutral. The generic image test attachment is labelled PNG although its bytes are JPEG; canonical metadata and the preserved .jpg copy are correct. No merge or release was performed.


Note

Medium Risk
Changes affect the GPU FIFO queue, tools upscale wiring, and audio duration submission—areas that can block or mis-schedule generation jobs—alongside a large but mostly isolated E2E surface.

Overview
Adds a repeatable live acceptance harness with timestamped evidence roots, run.json manifests, safe --resume (failed-test IDs only), and acceptance_report.py for an offline HTML audit index. New Playwright scenarios app-tour, app-generate, app, and wizard-media exercise native UI navigation (including mobile), real Flux/ACE-Step/upscale flows, and Wizard-submitted media—with browser-only workspace isolation so shared server folders and global prefs are not overwritten.

Production fixes bundled with the audit work: rename tools_upscale import to tools_upscale_service so the HTTP route no longer shadows the Python module; drain cancelled FIFO queue heads in acquire_generation_slot when a worker never starts; mobile TabFilter stacks category tabs vs utility controls; Wizard reconciliation for English audio/voice vs educational “how to” prompts; await model options and treat only zero duration as “auto” so short ACE-Step requests (e.g. 20s) are not replaced by model defaults.

Docs add APP_USER_GUIDE.md, APP_ACCEPTANCE_COVERAGE.md, and expanded WIZARD_ACCEPTANCE_TESTING.md; comic/series Wizard specs gain stricter export and plan-then-apply steps.

Reviewed by Cursor Bugbot for commit 556d77e. Configure here.

@github-actions

github-actions Bot commented Sep 8, 2026

Copy link
Copy Markdown

PR Review — Loreframe Studio

Risk: medium
Scope: 28 file(s); +1550/-120; React UI, backend services, docs, repo scripts / CI

Automated review from scripts/analyze_pr.py. This is a heuristic pass (no LLM) so humans still own the merge decision.

Findings

  • medium — Dangerous dynamic execution (tests/test_tools_upscale_contract.py)
    tests/test_tools_upscale_contract.py adds exec(). Confirm the input is trusted and sandboxed.
  • medium — Dangerous dynamic execution (ui/e2e/helpers/liveWorkspacePolicy.ts)
    ui/e2e/helpers/liveWorkspacePolicy.ts adds exec(). Confirm the input is trusted and sandboxed.
  • medium — Large pull request
    1550 additions / 120 deletions. Reviewers will have an easier time with smaller, focused PRs.
  • low — UI changed — rebuild before merge
    Run cd ui && npm run build (CI already does this). Pinokio Update rebuilds for end users; keep ui/dist untracked.

Changed files

  • added: docs/APP_ACCEPTANCE_COVERAGE.md, docs/APP_USER_GUIDE.md, scripts/acceptance_report.py, tests/test_acceptance_runner.py, ui/e2e/helpers/appAudit.ts, ui/e2e/helpers/liveMedia.ts, ui/e2e/helpers/liveRead.ts, ui/e2e/helpers/liveWorkspace.ts, ui/e2e/helpers/liveWorkspacePolicy.ts, ui/e2e/live-specs/app-features.spec.ts, ui/e2e/live-specs/app-generation.spec.ts, ui/scripts/export-acceptance-capabilities.mts, ui/tests/liveRead.test.mjs, ui/tests/liveWorkspaceGuard.test.mjs
  • modified: app/_launch_runtime.py, app/services/job_lifecycle.py, docs/WIZARD_ACCEPTANCE_TESTING.md, scripts/run_wizard_acceptance.py, tests/fixtures/architecture_wire_inventory.json, tests/test_job_lifecycle.py, tests/test_tools_upscale_contract.py, ui/e2e/live-specs/wizard-generation.spec.ts, ui/e2e/playwright.live.config.ts, ui/src/components/MainContent/TabFilter.tsx, ui/src/features/agent/agentActions.ts, ui/src/features/studio/actions.ts, ui/src/stores/useStore.ts, ui/tests/agentActions.test.mjs

CONTRIBUTING checklist

  • python scripts/verify_clean_repo.py
  • python -m compileall -q app/services app/launch.py scripts
  • cd ui && npm run build if the UI changed
  • No weights, CivitAI sidecars, or generated guides
  • Stays local-first (no required accounts / telemetry)

Posted by the repo PR review workflow. Re-runs on each push to the PR.

@github-actions

github-actions Bot commented Sep 8, 2026

Copy link
Copy Markdown

Code health

Quality score: 55.4/100

Higher is better. The score is a trend dashboard; the independent ratchet below remains the CI gate.

Component Weight Current Change
Cyclomatic health 45% 54.6 +0.1
File concentration 25% 62.0 +0.4
Oversized-file debt 20% 37.9 +0.4
Modularity 10% 77.5 +1.3

Change vs PR base: +0.4 points.

Metric Value
Production LOC 266,491
Production files 734
Test LOC 96,237
Functions measured 17,337
Functions complexity ≥ 15 842
Maximum complexity 672
Policy code-health-policy-v1
HEAD c6d535479bd4d4fae12935209eeaa37bb6a3beff
Base 92c95500e1d63f242cc6a9fe1c107705de34a083
UI measurement complete

Markdown, JSON catalogs and tests are out of this table. Only app/ runtime + ui/src TS/JS count.

Most complex functions

Complexity Where
672 app/wgp.py:7219 generate_video
381 ui/src/stores/useStore.ts:4084 Async method 'startGeneration'
355 app/_launch_runtime.py:23613 _run_generation
308 app/wgp.py:12359 generate_video_tab
268 ui/src/stores/useStore.ts:8636 Async method 'loadSettingsFromOutput'
260 ui/src/components/Sidebar/SceneAnimatorPanel.tsx:496 Function 'SceneAnimatorPanel'
258 app/services/director/planners/short_film.py:3433 ShortFilmPlanner._plan_story_driven
248 app/services/director_pipeline.py:13607 _run_video_generation
245 app/services/director_pipeline.py:7712 _run_pipeline
244 ui/src/features/agent/agentActions.ts:1145 Function 'parseAction'
226 app/services/director_pipeline.py:6541 update_comic_preview
225 ui/src/features/agent/agentActions.ts:2898 Async function 'executeAgentActions'

Trend vs baseline

Metric Δ
Production LOC +1,097
Test LOC +1,295
Functions ≥ 15 +5
Maximum complexity +0

Warnings

  • production LOC increased by +1,097
  • functions at complexity >= 15 increased by +5
  • complexity hotspot ui/src/components/Sidebar/SceneAnimatorPanel.tsx rose 258 -> 260
  • complexity hotspot ui/src/features/scene3d/camera.ts rose 20 -> 22
  • complexity hotspot ui/src/features/scene3d/templates.ts rose 16 -> 19
  • hotspot ui/src/stores/useStore.ts increased by +5 lines
  • hotspot ui/src/features/agent/agentActions.ts increased by +6 lines
  • hotspot ui/src/components/Sidebar/SceneAnimatorPanel.tsx increased by +6 lines
  • hotspot ui/src/types/index.ts increased by +1 lines
  • hotspot ui/src/lib/sceneRecipe.ts increased by +6 lines

Ratchet passed.

@IAnMove
IAnMove marked this pull request as ready for review September 8, 2026 20:22
@cursor

cursor Bot commented Sep 8, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_4188a6f1-e732-46f8-af25-bfd316ad4497)

@IAnMove

IAnMove commented Sep 8, 2026

Copy link
Copy Markdown
Owner Author

Revisión independiente (worktree aislado; no se tocó el backend Pinokio ni el workspace por defecto).

Aporta: harness live repetible con aislamiento real (carpeta e2e_*, active del servidor no se cambia, DELETE/settings bloqueados), guía, y arreglos de corridas reales (alias upscale, FIFO cancelado, TabFilter móvil, intent EN, duración de audio, timeout de modelo frío). exec() del contrato upscale es AST de código del repo; el de liveWorkspacePolicy es RegExp.exec.

CI required ya verde en 556d77e4. Independent QA formal sigue NEUTRAL (el publisher no cuenta como revisión de producto). 13 commits por detrás de development actual (#257/#260) sin solape de archivos.

Marcado ready for review.

@IAnMove
IAnMove merged commit 0dbc7ea into development Sep 8, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant