Add arktts (Audio8-TTS-Preview-0.6b): DualAR TTS with zero-shot voice cloning

[xocialize]

Adds the arktts model family — Audio8/Audio8-TTS-Preview-0.6b (Apache-2.0), a 0.6B multilingual TTS model with zero-shot voice cloning and a bundled 44.1 kHz codec. Architecture is DualAR (Fish S2 Pro-style): a 24-layer slow AR transformer emits one semantic token per frame; a 4-layer fast AR emits the frame's 10 codec codebooks conditioned on the slow hidden state.

What's included

• mlx_audio/tts/models/arktts/ — model (arktts.py), codec (codec.py, DAC-style encoder/decoder + split semantic/residual RVQ + windowed transformer pre/post modules), mirroring the upstream remote-code structure file-for-file
• Reference-exact sampling: semantic-range logit filter, legacy top-k/top-p order (filter before temperature), exponential-race sampling, RAS repetition rescue
• sanitize() consumes either raw upstream checkpoints (folds both parametrized and legacy weight norm, transposes conv layouts, reshapes Snake alphas) or the pre-converted mlx repo (idempotent)
• mlx_audio/tts/tests/test_arktts.py — sanitize contract + tiny-model forward/generation tests (no weights needed)

Converted weights

mlx-community/Audio8-TTS-Preview-0.6b-bf16 (bf16 LM + fp32 codec), validated by fresh-download → mlx_audio.tts.utils.load → voice-clone generation.

Parity vs PyTorch reference (fp32, CPU stream)

• unit/block outputs: max-abs 1e-6…1e-4
• reference-audio codec encode: 100% code-exact
• greedy generation: 100% token-exact over a 102-frame validation utterance
• decoded waveform: max-abs 7.5e-6
• bf16 GPU smoke: healthy audio (−18 dBFS RMS), RTF ~0.8–0.9 on M-series

Usage

from mlx_audio.tts.utils import load
model = load("mlx-community/Audio8-TTS-Preview-0.6b-bf16")
for r in model.generate(text="Hello from MLX.", ref_audio="ref.wav", ref_text="Reference transcript."):
    ...

🤖 Generated with Claude Code

[lucasnewman]

These two cache arrays need to be materialized with e.g. mx.eval((self._freqs_cis, self._fast_freqs_cis)) at init time so the model doesn't crash moving across streams/threads.

[lucasnewman]

@xocialize Very nice, thank you! Can you fix the cache array issue I mentioned and add a README for the model, and then we can merge?

[xocialize]

Thanks for the review! Both addressed:

1. Cache arrays. Good catch — the two rope tables are plain attributes rather than module parameters, so nothing else was forcing them. They're now mx.eval'd at the end of __init__. Verified the failure mode you described is gone by constructing on the GPU stream and running a forward on the CPU stream, which is what would previously have evaluated the graph in the wrong context.

2. README added at mlx_audio/tts/models/arktts/README.md, following the Voxtral/confucius4 shape: supported model, Python + CLI usage, languages, options, architecture, conversion notes, parity numbers.

One small thing worth flagging: I'd initially written the CLI example with --ref-audio/--ref-text, then checked and found mlx_audio.tts.generate has no such flags. The CLI block now shows only the default-voice path and notes that cloning is Python-API only — same as confucius4 documents it. If exposing reference audio through the CLI would be useful I'm happy to add it in a follow-up, but it seemed out of scope here.

[lucasnewman]

Thanks!