fix(ling): apply Ling's trained per-layer SwiGLU clamp (+17pp HumanEval)

[True2456]

Problem

Ling-3.0-flash is trained with a clamped SwiGLU on its late layers, and ships the limits in config.json:

expert_swiglu_limit_list        layers 35-41 = 4
share_expert_swiglu_limit_list  layers 34-39 = 5, layers 40-41 = 7

Nothing in oMLX applies them. Neither the vendored bailing_hybrid in omlx/patches/bailing_hybrid/ nor the bailing_hybrid in the bundled mlx-lm build has any clamp handling, so Ling runs its last seven layers unclamped and activations there can run away.

inclusionAI's vLLM port does implement this — _get_layer_swiglu_limit + SwigluStepAndMul in bailing_moe_v3.py (commit e0040c3). The HF transformers reference does not, which is plausibly how it came to be omitted downstream. A third-party NVFP4/SGLang repack independently hit the same omission in SGLang's NVFP4 MoE path and reports it corrupting a large share of generated code.

Impact

HumanEval on Ling-3.0-flash-8fixed-5routed via oMLX's own accuracy bench — 164 problems, greedy, thinking off, MTP off:

	HumanEval	correct	eval time
clamped	88.41%	145/164	163.6s
unclamped	71.34%	117/164	167.0s

+17.07pp — 28 problems — at no measurable runtime cost. The clamp is two elementwise ops; the eval times are within noise. Reproduced independently on this branch: 88.41% (145/164).

Change

Semantics follow vLLM exactly:

silu(gate).clamp(max=L) * up.clamp(-L, L)

applied per layer, separately for routed experts (SwitchGLU.activation) and shared experts.

Argument order is checked against mlx-lm's switch_layers.SwiGLU.__call__(x, gate) -> swiglu(gate, x): SwitchGLU passes (x_up, x_gate), so silu lands on the gate. Reversing it would clamp the wrong tensor, and a test pins that.

Two code paths are covered, because the live mlx_lm.models.bailing_hybrid can come from either place:

• the vendored copy now implements the clamp in-source;
• swiglu_clamp.ensure_swiglu_clamp() installs it onto an mlx-lm build that already ships the module — which is what oMLX bundles today, so the vendored copy isn't otherwise reached. apply_bailing_hybrid_patch() calls it for whichever module resolves.

Models without the config keys are untouched (the helper returns None for absent/zero limits), so this is a no-op for every other bailing checkpoint.

OMLX_LING_NO_SWIGLU_CLAMP=1 restores the unclamped path — that's how the A/B above was produced, and it keeps the result reproducible by a reviewer.

Tests

8 new cases in tests/test_bailing_swiglu_clamp.py: limit resolution (absent/zero/short list), the clamp formula against the reference, that the clamp actually binds, the SwitchGLU argument order, per-layer wiring for routed and shared paths, that a model without limits stays unclamped, and a finite forward. They run against whichever build is live.

Verified on the real 127B checkpoint: Ling SwiGLU clamp applied to 15 expert paths, matching the config lists exactly (7 routed on layers 35-41, 8 shared on 34-41).

Upstream

The same fix is proposed against the mlx-lm branch this vendors from: scaryrawr/mlx-lm#1 (ling-3.0-flash). If that lands, the vendored copy here can simply be resynced and the runtime install path becomes a no-op (it already detects a build that implements the clamp natively and leaves it alone).

[jundot]

Thanks for tracking this down. I verified the clamp wiring against the checkpoint config and ran a real-server smoke with inclusionAI/Ling-3.0-flash-fp8: it loaded through the vendored bailing_hybrid path and returned clean deterministic text and Python code with no inference errors. The 7 routed and 8 shared expert limits match the trained configuration, while checkpoints without these lists stay unchanged. LGTM after the real-model smoke; merging.