Qwen3.5-35B-A3B layer package ships MTP weights but omits generation.speculative_decoding, so native MTP can never be selected

[michaelneale]

Summary

The published layer package meshllm/Qwen3.5-35B-A3B-UD-Q4_K_XL-layers contains and successfully loads native MTP weights, but its model-package.json declares no generation.speculative_decoding block. As a result strategy = "auto" resolves native MTP off, and strategy = "mtp" would be rejected. The capability ships but is unreachable through the package.

This is a packaging/provenance gap, not a runtime MTP regression — the runtime behaved exactly as specified given the manifest it was handed.

Evidence the weights are there

Pinned revision 69cbd9fc1f928305e8e86e1148a1db6a6f1de8e9, manifest sha256 bac69a30665539535a4018006397ed257d383f96909d991d7a26c6eb3c54d54c.

• Source model is unsloth/Qwen3.5-35B-A3B-MTP-GGUF:UD-Q4_K_XL — MTP in the name.
• GGUF header of layers/layer-003.gguf: general.architecture=qwen35moe, block_count=41, nextn_predict_layers=1.
• Block 40 is the appended nextn/MTP block (20 tensors, versus 16 for full-attention blocks and 19 for SSM blocks).
• It loads. On a two-node split with the final stage owning 27..41, that stage reported skippy_model_open_from_parts ... layers=27..41 ... include_output=true ... status=Ok with 258 tensors — block 40 included.

Evidence nothing selects it

Both stages logged mtp_source=Disabled. Every speculative counter in the inference timings was zero:

"speculative_proposed_n": 0, "speculative_accepted_n": 0, "speculative_rejected_n": 0,
"speculative_windows": 0, "native_mtp_verify_window_verifications": 0, "verify_window_depth": 1

Mechanism

resolve_native_mtp_strategy under the default strategy = "auto" requires:

// crates/mesh-llm-host-runtime/src/inference/skippy/resolver/speculative.rs:727-741
package_generation_supports_default_native_mtp(generation) || direct_gguf_supports_native_mtp(model_path)

• First disjunct: generation.speculative_decoding — the manifest is schema_version: 1 with no generation key at all (top-level keys are schema_version, model_id, source_model, format, layer_count, activation_width, shared, layers, skippy_abi_version, created_at_unix_secs). So package_generation is None.
• Second disjunct: direct_gguf_supports_native_mtp(model_path) scans a local GGUF for nextn_predict_layers > 0. The stage reports source_model_path: "/source/Qwen3.5-35B-A3B-UD-Q4_K_XL.gguf" — a build-host path that does not exist on any serving machine. The scan cannot succeed for a package-backed load.

Forcing the issue does not help: strategy = "mtp" checks supports_native_mtp, built from the same two predicates plus an explicit draft-model path, and otherwise bail!("skippy speculative.strategy = \"mtp\" requires proven native MTP support").

So for any package-backed serve, the manifest is the only channel through which native MTP can be discovered, and this package does not use it.

Why this is worth fixing at the packaging layer

The information needed is already known at package-build time — the builder split a GGUF whose header declares nextn_predict_layers=1 and emitted the nextn block as layer-040.gguf. Emitting a generation.speculative_decoding block (the shape documented in docs/USAGE.md:881, e.g. as published for GLM-4.7-Flash-MTP) is a mechanical consequence of that.

Suggested:

• Package build: when the source GGUF has nextn_predict_layers > 0 or nextn tensors, emit a generation.speculative_decoding entry declaring the native-MTP strategy and its default.
• Backfill: republish or amend existing -layers packages built from MTP source models. At minimum meshllm/Qwen3.5-35B-A3B-UD-Q4_K_XL-layers; worth auditing the rest.
• Surface the gap: a package whose layers contain nextn tensors but whose manifest declares no MTP strategy is a detectable inconsistency. Warning at load ("package contains MTP weights but declares no speculative strategy; native MTP disabled") would have turned a silent capability loss into a one-line diagnosis.

Impact

Any operator serving this package gets ordinary autoregressive decode and no signal that the MTP weights they downloaded (the nextn block is part of the 21.72 GiB) are inert. Measured 3.17 tok/s decode on a two-Mac split — with no way to tell from the API whether MTP was tried and unhelpful, or never attempted.

Context

Found while spot-checking cross-host split serving of current main (fb58bf73ba375fba1f1921c432bcb0d4ac02ac2b) on two Macs. The split itself passed: locked topology 0..27 / 27..41 realized exactly, correct per-node partial fetch, both stages ready, correct inference output. MTP could not be exercised because of this.