Add support for Kanana-2 Tiny (kanana2_tiny) - #1655
Open
choipilkyu wants to merge 1 commit into
Open
Conversation
Kakao's kanana-2-1.3b-{base,instruct} use Kanana2TinyForCausalLM. The
architecture is Qwen3 except that the rotary embedding differs by attention
type: full-attention layers use a yarn-scaled rope for long context, sliding
layers keep an unscaled rope and attend within a 1024-token window. The
checkpoint expresses this as nested rope_parameters keyed by attention type
plus an explicit layer_types array.
Without this, loading the checkpoint as qwen3 (the mapping Kakao ships in
sglang/config.json) fails at generation — mlx-lm reads rope_theta from the top
level of rope_parameters, which is absent — and would collapse both rope
variants into one even if it parsed.
The per-layer rope dispatch follows gemma3_text, which solves the same problem;
layer types are read from layer_types rather than computed from a modulus since
the checkpoint states them explicitly. Sliding layers get a RotatingKVCache.
Verified against the upstream fp32 implementation on kanana-2-1.3b-instruct:
top-50 logit cosine >= 0.99998 and matching argmax on Korean, English, mixed
and a 9,478-token prompt. The long prompt is the one that matters — it is past
the 4096-token pre-yarn context, so a single-rope implementation would diverge
there while passing the short probes.
choipilkyu
force-pushed
the
add-kanana2-tiny
branch
from
August 5, 2026 01:33
146e91d to
b95ccbf
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add support for the Kanana-2 Tiny architecture
This adds a
kanana2_tiny.pymodel implementation somlx-lmcan load and runKakao's Kanana-2 Tiny models, e.g.
kakaocorp/kanana-2-1.3b-instructand
kakaocorp/kanana-2-1.3b-base(
model_type: kanana2_tiny,Kanana2TinyForCausalLM).Architecture
Kanana-2 Tiny is Qwen3 with one distinctive feature, which is also what blocks
loading it as
qwen3today:driven by the config's
layer_typeslist (sliding_window= 1024), in arepeating
[sliding, sliding, sliding, full]pattern (24 sliding, 8 full).original_max_position_embeddings4096), sliding-window layers use plainRoPE. Both rope configs come from the config's
rope_parametersmappingkeyed by attention type. The design intent is legible: local context is
handled at native RoPE by the sliding layers, long context is extended to 32k
by the 8 full layers.
is Qwen3, and Kakao's own modeling code says so explicitly.
The implementation composes the
qwen3attention/MLP block with thegemma3_texthybrid-mask scheme (KVCachefor full layers,RotatingKVCachefor sliding ones) and per-layer rope selection viainitialize_rope, following the same shape as the recently mergedmellum.Layer types are read from
layer_typesrather than computed from a modulus,since the checkpoint states them explicitly.
Why the
qwen3mapping is not enough. Kakao ships asglang/config.jsonthat declares
Qwen3ForCausalLM, but loading the checkpoint that way inmlx-lmfails at generation withValueError: [rope] Neither base nor freqs has a value—rope_thetais nested under the attention-type keys, not at thetop level of
rope_parameters. Even if it parsed, both attention types wouldcollapse onto a single rope.
Testing
tests/test_models.py::test_kanana2_tinycovers the standard model runner plusan assertion that the two attention types do not share a rope instance.
Numerical parity was checked against the upstream
transformersimplementation (fp32,
trust_remote_code=True) onkakaocorp/kanana-2-1.3b-instruct, comparing last-position logits foridentical token ids:
The long probe is the one that matters: it is well past the 4096-token
pre-yarn context, so an implementation that collapsed the two ropes into one
would diverge there while still passing the short probes.
Generation is coherent in both languages, e.g. asked to describe Kakao in one
sentence it answers "카카오(Kakao)는 모바일 메신저, 포털, 핀테크, 콘텐츠 등
다양한 서비스를 제공하는 대한민국의 대표적인 IT 기업이다."
pre-commit run --files(black, isort) passes, and the fulltests/test_models.pysuite is green (77 passed, 1 skipped).