[metal::malloc] Resource limit (499000) exceeded on qwen3_5_moe (non-MLA), single node — #839 does not cover this path

[veghadam]

Summary

[metal::malloc] Resource limit (499000) exceeded. still occurs on mlx_lm 0.31.3, on a single machine (no distributed / no mlx.launch), with a qwen3_5_moe model — i.e. an MoE model that does not use MLA.

This looks like the same symptom as #831, but reached through a different code path. #831 was reproduced and fixed for GLM 4.7 Flash via #839 (DSV3 MLA), and that fix is present in 0.31.3 (0.30.6, 2026-02-04 ≪ 0.31.3, 2026-04-22). Since qwen3_5_moe is not an MLA architecture, the MLA fix does not appear to cover it.

Filing separately rather than reopening #831, since the model family and the (presumed) allocation source differ.

Environment

mlx_lm	0.31.3
mlx	0.31.2
Python	3.11.15
macOS	26.4.1 (25E253)
Hardware	Apple M1 Max, 64 GB (32 GPU cores)
iogpu.wired_limit_mb	61440
Mode	single node, plain mlx_lm.server

Model architecture (config.json → text_config):

model_type      qwen3_5_moe
num_hidden_layers        40
num_experts             256
num_experts_per_tok       8

4-bit quant, ~22 GB on disk.

Server invocation

python -m mlx_lm server \
  --model <qwen3_5_moe 4-bit> \
  --host 0.0.0.0 --port 8081 \
  --max-tokens 8192 \
  --prompt-cache-bytes 4294967296 \
  --decode-concurrency 2 \
  --prompt-concurrency 1 \
  --prefill-step-size 2048

What happens

Under normal chat/agent traffic with long contexts (40k–75k prompt tokens), the server dies with:

RuntimeError: [metal::malloc] Resource limit (499000) exceeded.

It crashed 4 times over roughly a day of ordinary use (launchctl restart count), always during prompt processing on a large prefill.

Notably this is not an out-of-memory condition. Sampling vm_stat during a successful 58,340-token prefill on the same box:

• baseline (model loaded, idle): 32.6 GiB
• peak during prefill: 40.2 GiB
• ceiling (iogpu.wired_limit_mb): 60 GiB

So ~20 GiB spare at peak. The failure is the Metal resource/allocation count, not bytes — consistent with the 499000 figure being a count.

Workaround that works

Reducing --prefill-step-size from 2048 to 512 stops the crashes. With 512, prefills that previously failed now complete:

prompt tokens	result	prefill rate
84,612	completes	175 tok/s (32-core M1 Max)
84,612	completes	153 tok/s (24-core M1 Max)

No further metal::malloc occurrences and no restarts since the change, on two machines.

The cost is throughput: prefill drops from roughly 440 tok/s at --prefill-step-size 2048 to ~175 tok/s at 512 on the same hardware.

Guess at the cause

With 256 experts and 8 active per token, a large prefill chunk seems to reference a very large number of distinct expert weight buffers within a single command buffer. Halving/quartering the chunk proportionally reduces the resources referenced per command buffer, which matches the observed behaviour — but I have not profiled it, so this is inference from the workaround rather than a diagnosis.

If it would help, I'm happy to run instrumented builds or capture a Metal trace on this hardware.