Summary
Requesting an MLX port / pipeline for CogVideoX (e.g. THUDM/CogVideoX-5b, THUDM/CogVideoX-2b) so it can run natively on Apple Silicon alongside the existing ltx_2 and wan_2 pipelines.
Why
mlx-video currently ships only ltx_2 and wan_2 (confirmed under mlx_video/models/). CogVideoX is a popular open text-to-video / image-to-video diffusion model, but there is no native MLX path for it. Without one, downstream Mac inference servers (we run fusion-mlx on top of mlx-video) cannot offer CogVideoX and must either fall back to CPU/PyTorch or decline the request.
What's needed
An MLX implementation of the CogVideoX pipeline:
- 3D VAE (encoder/decoder) in MLX
- DiT transformer in MLX
- The CogVideoX noise scheduler (CogVideoXDDPM-style) in MLX
- A
generate_video(...) entrypoint mirroring the ltx_2/wan_2 signatures (prompt, num_frames, height, width, num_inference_steps, cfg_scale, seed, output_path, tiling)
Context
In our downstream server we currently expose CogVideoX as an explicitly unimplemented backend that auto-detects cogvideo/cog_video model paths and fails loudly pointing here. We would love to replace the NotImplementedError with a real delegator to mlx-video once a port lands.
Happy to help test a WIP port on M-series hardware if anyone picks this up.
Summary
Requesting an MLX port / pipeline for CogVideoX (e.g.
THUDM/CogVideoX-5b,THUDM/CogVideoX-2b) so it can run natively on Apple Silicon alongside the existingltx_2andwan_2pipelines.Why
mlx-video currently ships only
ltx_2andwan_2(confirmed undermlx_video/models/). CogVideoX is a popular open text-to-video / image-to-video diffusion model, but there is no native MLX path for it. Without one, downstream Mac inference servers (we run fusion-mlx on top of mlx-video) cannot offer CogVideoX and must either fall back to CPU/PyTorch or decline the request.What's needed
An MLX implementation of the CogVideoX pipeline:
generate_video(...)entrypoint mirroring theltx_2/wan_2signatures (prompt,num_frames,height,width,num_inference_steps,cfg_scale,seed,output_path,tiling)Context
In our downstream server we currently expose CogVideoX as an explicitly unimplemented backend that auto-detects
cogvideo/cog_videomodel paths and fails loudly pointing here. We would love to replace theNotImplementedErrorwith a real delegator to mlx-video once a port lands.Happy to help test a WIP port on M-series hardware if anyone picks this up.