Integrate DeepSeek V4 Flash 0731 into MTPLX

[davidtai]

Summary

Adds native DeepSeek-V4-Flash-0731 support to MTPLX: model loading, DSpark proposal execution, construction-bound K1-K3 routing, physical-M3 verification, benchmark coverage, and model-specific tests.

The optimized lane is installed only after its artifact and topology invariants pass. Once installed, it executes directly—there is no silent fallback to stock generation.

Model artifact

• Hugging Face artifact: mlx-community/DeepSeek-V4-Flash-0731-2.4bit-mixed
• Source revision: 10001e0065f8394e03e968e652cbbe7cd2ca122c
• Architecture: 284B target model, approximately 304B including MTP, approximately 13.8B active parameters per token
• Quantization: mixed affine MLX weights, 2.44 bits/weight effective; 92.8 GB decimal (86.5 GiB) on disk
• Weight policy: 2-bit routed experts, 6-bit attention, 8-bit shared/embedding/LM-head tensors, and 3-bit MTP experts
• Config SHA-256: 44735712733fcf8f299bdf1faa1d87fac88f1917efe1d3876d6d4c582f79a68f
• Weight-index SHA-256: f1332b2b209769c2db335954c2651652a8048e7d7dbf60296c2f2c0198715861

Every performance, memory, and correctness result in this PR is scoped to this exact artifact and revision.

What this adds

• DeepSeek-V4-Flash-0731 model and loader support.
• Native DSpark proposer with fixed K1-K3 construction routes.
• Physical-M3 verification for the qualified K2 path.
• Explicit greedy-sampling and nonexact-lane contracts.
• 262,144-token context support through the existing MTPLX context interface.
• Depth-sweep benchmarks reporting prefill, decode, end-to-end throughput, acceptance, and memory.

Performance

Apple M5 Max, 128 GB, MLX 0.32.0, greedy sampling (temperature=0, top_p=1, top_k=0), 14-token prompt, 128 generated tokens, one model load. These are the warmed rows only; the one-time K0 compile row is intentionally excluded from the comparison.

Depth	Target route	Evidence	Prefill tok/s	Decode tok/s	End-to-end tok/s	Exact vs K0
K0	Serial AR	matched MLX 0.32.0, 14 / 128	103.791	32.434	31.362	reference
K1	Primary + 1 draft, native M2 block	historical MLX 0.31.2, 9 / 64	not recorded	19.193	18.584	yes
K2	Primary + 2 drafts, compiled physical-M3 block	matched MLX 0.32.0, 14 / 128	103.509	35.700	34.393	no
K3	Primary + 3 drafts, native M4 block	historical MLX 0.31.2, 9 / 64	not recorded	21.413	20.609	no

K2 accepted 68 of 119 drafted tokens: 68.3% at draft position 1 and 45.8% at position 2.

K2 and K3 both use one block target verification call per full cycle and both currently require greedy sampling. They do not use the same target kernel route: K2 has the receipt-backed compiled physical-M3 target stack, while K3 uses the native M4 target shape. Only K0 and K2 are a matched performance comparison; K1 and K3 are historical filler included to show the complete depth picture until the MLX 0.32.0 rerun is available.

Memory

The table below comes from the short 14-prompt-token / 128-output-token bracket; it is not a 64K or 128K context measurement. It reports post-load active memory and growth from the same 86.4561 GiB baseline. The listed MLX peak counter includes model loading and is therefore not a valid request peak. A post-load-reset 128K context rerun is pending and will replace this table.

Lane	Post-load active GiB	Growth MiB	MLX peak counter GiB
K0 AR	86.4561	0.018	139.7061
K2 physical-M3	86.4561	0.019	139.7061

Historical K-depth sweep

The earlier MLX 0.31.2 sweep used a 9-token prompt and 64 generated tokens. It did not record prefill throughput or post-load growth, so neither is fabricated here. Peak delta is calculated against K0 within the same model load.

Depth	Decode tok/s	End-to-end tok/s	Peak GiB	Peak delta vs K0	Accepted / drafted	Exact vs K0
K0	24.565	23.312	86.5079	baseline	—	reference
K1	19.193	18.584	86.5175	+9.8 MiB	27 / 36	yes
K2	19.640	19.010	86.5240	+16.5 MiB	34 / 58	yes
K3	21.413	20.609	86.5392	+32.1 MiB	37 / 76	no

This historical sweep is diagnostic, not the promotion result. The current physical-M3 K2 lane is faster than warmed K0 but diverges from K0 at generated-token index 44, so it is exposed as an explicit throughput-over-exactness route.

Evaluation status

No HumanEval, HumanEval+, MBPP, or MBPP+ receipt exists for the 0731 artifact yet. The older DeepSeek-V4 EvalPlus run used DeepSeek-V4-Flash-2bit-DQ-mtp; those scores are deliberately excluded because they do not validate these weights or this K1-K3 implementation.

Artifact	HumanEval	HumanEval+	MBPP	MBPP+
DeepSeek-V4-Flash-0731 2.4-bit	pending	pending	pending	pending

Correctness contract

• The native block path currently requires greedy target and draft sampling.
• K1-K3 geometry is fixed at construction; unsupported states fail before generation.
• K2 physical-M3 is intentionally nonexact and never masquerades as exact speculative decoding.
• Profiler traces are diagnostic only; all throughput numbers above come from uninstrumented generation under the exclusive GPU lock.

Verification

python -m pytest -q \
  tests/test_public_cli.py \
  tests/test_deepseek_v4_dspark.py \
  tests/test_deepseek_v4_dspark_generation.py \
  tests/test_generation_deepseek_v4_dspark_integration.py \
  tests/test_runtime_deepseek_v4_dspark.py \
  tests/test_server_openai.py

Tracked performance details and raw-artifact hashes are in docs/perf/receipts/deepseek-v4-0731-dspark.md.