Skip to content

[codex] Add Mini Studio web UI and fix LTX config loading#31

Draft
andyelka-creator wants to merge 5 commits into
Blaizzy:mainfrom
andyelka-creator:codex/mini-studio-ltx-config-fix
Draft

[codex] Add Mini Studio web UI and fix LTX config loading#31
andyelka-creator wants to merge 5 commits into
Blaizzy:mainfrom
andyelka-creator:codex/mini-studio-ltx-config-fix

Conversation

@andyelka-creator

@andyelka-creator andyelka-creator commented May 24, 2026

Copy link
Copy Markdown

Summary

  • add a lightweight local Gradio Mini Studio for LTX-2 and Wan job launching
  • persist queue state, per-job logs, and generated video gallery under outputs/
  • add a Web UI guide for local usage and troubleshooting
  • fix LTX-2 config loading when Hugging Face metadata keys such as _class_name are present
  • normalize unprefixed Hugging Face / Diffusers transformer weight keys before strict MLX loading
  • support unified Hugging Face vae/diffusion_pytorch_model.safetensors layout when generation asks for vae/encoder or vae/decoder
  • normalize unified Diffusers VAE encoder/decoder key names and map latent_channels into the encoder config
  • default missing unified VAE decoder per-channel statistics to identity values for strict loading
  • make top-level mlx_video imports lazy so the Web UI can import package metadata without eagerly loading model modules

Fixes #30
Fixes #32
Fixes #33
Fixes #34
Fixes #35
Fixes #36

Root Cause

LTX-2 loading had several compatibility failures against current Hugging Face files:

  1. config.json contains metadata keys that are not dataclass fields, so direct LTXModelConfig(**config_dict) fails.
  2. The transformer safetensors are unprefixed, but still use Diffusers module names such as linear_1, to_out.0, norm_q, time_embed, and proj_in.
  3. The video VAE is stored as unified vae/config.json and vae/diffusion_pytorch_model.safetensors, while generation requested split vae/encoder and vae/decoder directories.
  4. Unified VAE safetensors use unprefixed encoder.* and decoder.* Diffusers block names such as resnets/downsamplers/upsamplers, while the MLX modules expect res_blocks and interleaved sampling block indices.
  5. Diffusers AutoencoderKLLTX2Video uses out_channels=3 for RGB output and latent_channels=128 for latent width; the encoder must use latent_channels.
  6. The unified VAE checkpoint omits optional decoder per-channel statistics, but strict loading still expected those parameters even though the decoder initializes identity defaults.

Validation

  • .venv/bin/python -m pytest tests/test_ltx_model_loading.py tests/test_ltx_video_vae_loading.py tests/test_webui_job_store.py tests/test_webui_runner.py tests/test_rope.py::TestDoublePrecisionRopeConfig -q -> 24 passed, 2 warnings
  • real downloaded Lightricks/LTX-2 transformer key comparison after sanitizer -> 3510 sanitized keys, 3510 model keys, extra 0, missing 0
  • LTXModel.from_pretrained(.../Lightricks/LTX-2/.../transformer, strict=True) -> strict transformer load ok
  • VideoEncoder.from_pretrained(.../Lightricks/LTX-2/.../vae/encoder) -> image encode ok, latent shape (1, 128, 1, 16, 16)
  • VideoDecoder.from_pretrained(.../Lightricks/LTX-2/.../vae/decoder, strict=True) -> strict decoder load ok, identity statistics shape (128,)
  • unified VAE encoder sanitizer against real checkpoint -> extra 0, missing 2 default per-channel statistics
  • .venv/bin/mlx_video.webui --help
  • curl -sI http://127.0.0.1:7861/ -> HTTP/1.1 200 OK

Local deployment

  • The Gradio Web UI is running locally at http://127.0.0.1:7861/ in this workspace.
  • This repository does not include a cloud deployment target, so the deployment completed here is the local Web UI service.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment