Add native Qwen3.8 27B model lanes - #303
Merged
Merged
Conversation
Add pinned BF16 and 4-bit Qwen3.8 targets, the official MTP component, native runtime loading, managed-model provenance, benchmark coverage, and public documentation.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
vision-chat-q38-27b, pinned to Qwen's official BF16 Qwen3.8-27B snapshotvision-chat-q38-27b-4bit, combining LM Studio's pinned MLX 4-bit target with the official pinned BF16 MTP shard mounted undermtp/Validation
./scripts/check.shReal-checkpoint evidence
Measured on an M4 Max coding smoke:
A deterministic 24-task stride through the official HumanEval dataset scored 20/24 for both 4-bit target-only and 4-bit + MTP, with the same four failures. One failing case produced 177 target-only tokens versus 174 with MTP, so the fast path remains opt-in rather than claiming exact greedy-output parity.
Operational notes
MERERUN_Q35_MTP_SPECULATION=1