Skip to content

Add native Qwen3.8 27B model lanes - #303

Merged
a-github-name merged 1 commit into
mainfrom
codex/qwen38-27b
Aug 15, 2026
Merged

Add native Qwen3.8 27B model lanes#303
a-github-name merged 1 commit into
mainfrom
codex/qwen38-27b

Conversation

@a-github-name

Copy link
Copy Markdown
Member

Summary

  • add vision-chat-q38-27b, pinned to Qwen's official BF16 Qwen3.8-27B snapshot
  • add vision-chat-q38-27b-4bit, combining LM Studio's pinned MLX 4-bit target with the official pinned BF16 MTP shard mounted under mtp/
  • extend the native Qwen-family runtime for Qwen3.8 dense text, code, vision, tokenizer/template, published sampling, context, and image-sizing behavior
  • keep dense MTP greedy speculation explicitly opt-in and disabled for sampled or JSON-constrained generation
  • record composite source provenance in managed manifests and cover catalog validation, memory admission, CLI parsing, coding benchmarks, and docs

Validation

  • ./scripts/check.sh
    • strict SwiftLint: 1,237 files, 0 violations
    • XCTest: 2,859 executed, 207 checkpoint-gated skips, 0 failures
    • Swift Testing: 30 passed
  • rebased focused suites: 308 executed, 3 skips, 0 failures
  • release build passed
  • managed pull preflight recognized both pinned component sources and reported the 4-bit composite ready to pull
  • canonical managed-layout smoke reported a valid 19.47 GB composite manifest with primary and component provenance

Real-checkpoint evidence

Measured on an M4 Max coding smoke:

Lane Target-only warm decode MTP warm decode Speedup MTP acceptance
BF16 8.78 tok/s 18.81 tok/s 2.14x 75.6%
4-bit 26.5 tok/s 38.8 tok/s 1.46x 75.6%

A deterministic 24-task stride through the official HumanEval dataset scored 20/24 for both 4-bit target-only and 4-bit + MTP, with the same four failures. One failing case produced 177 target-only tokens versus 174 with MTP, so the fast path remains opt-in rather than claiming exact greedy-output parity.

Operational notes

  • BF16 pull size: 55.59 GB; 64 GB minimum unified memory, 96 GB recommended
  • 4-bit composite pull size: 19.47 GB; 32 GB minimum unified memory, 48 GB recommended
  • enable experimental dense MTP with MERERUN_Q35_MTP_SPECULATION=1
  • default, sampled, and constrained requests retain target-only decode

Add pinned BF16 and 4-bit Qwen3.8 targets, the official MTP component, native runtime loading, managed-model provenance, benchmark coverage, and public documentation.
@a-github-name
a-github-name marked this pull request as ready for review August 15, 2026 00:07
@a-github-name
a-github-name merged commit 49b624b into main Aug 15, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant