Skip to content

Add Laguna S 2.1 model support - #1601

Draft
avlp12 wants to merge 1 commit into
ml-explore:mainfrom
avlp12:codex/laguna-s-2-1
Draft

Add Laguna S 2.1 model support#1601
avlp12 wants to merge 1 commit into
ml-explore:mainfrom
avlp12:codex/laguna-s-2-1

Conversation

@avlp12

@avlp12 avlp12 commented Jul 22, 2026

Copy link
Copy Markdown

Summary

  • add the 48-layer Laguna S 2.1 MLX runtime, including heterogeneous attention heads, YARN/default RoPE, per-head softplus gating, correction-bias routing, shared experts, and mixed KV caches
  • sanitize 36,096 individual expert tensors into ordered stacked SwitchGLU projections
  • exclude routers, norms, correction biases, and other control tensors from quantization

Validation

  • 10 Laguna unit tests pass
  • independent tiny PyTorch/MLX parity passes for routing, unbiased weights, attention gating, both RoPE variants, expert ordering, full logits, and cached logits
  • maximum observed full/cached logit absolute error: 2.98e-8
  • Ruff format/check and git diff check pass

Downstream release evidence

The PR contains code and weight-free tests only. Full-checkpoint artifacts and quality evidence are published separately in the downstream release repositories above.

@JuanColilla

JuanColilla commented Jul 22, 2026

Copy link
Copy Markdown

Same PR like mine: #1602

Although I see yours was first, which one shall we close?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants