deepseek-v2-lite: registry + adapter scaffolding (argmax-match deferred)

[Lazarus-931]

Summary

• Wires sk.load(\"deepseek-v2-lite\") through the central registry to a DeepSeek adapter from_spec that parses HF config.json into Config (27 layers, 2048 d_model, 16 heads, 64 routed + 2 shared experts, top-6, kv_lora_rank=512, partial-RoPE qk_head_dim=192).
• Adds the deepseek tokenizer-family entry (<｜begin▁of▁sentence｜> / <｜end▁of▁sentence｜>) and the models/deepseek/__init__.py package export.
• Honest scope: this PR is registry + Python plumbing only. The from_spec factory builds the Config then raises NotImplementedError with a pointer to the SPEC blockers; argmax-match against HF is not yet achieved.

Honest status vs the original brief

Landed

• Registry row + __init__.py + tokenizer family + Config parsing from HF config.json (including YaRN rope_scaling.{factor, mscale_all_dim, original_max_position_embeddings}).
• sk.load(\"deepseek-v2-lite\") reaches Config construction cleanly and exits with a documented blocker list.

Deferred (in priority order, all tracked in SuperKittens/models/deepseek/SPEC.md)

The SPEC port-status table lists 14 ✅ / 4 ⚠ needs-fix / 11 ❌ missing. For V2-Lite argmax-match the remaining items are:

1. Safetensors → packed-numpy loader. The existing C ABI sk_deepseek_load_weights takes flat host pointers for every weight in _WEIGHT_FIELDS; the in-tree C++ loader (weights.c++) uses GGUF tensor names (blk.L.attn_q_a.weight) that V2-Lite's safetensors do not ship with. The minimal path is a Python streamer that materialises packed bf16→fp16 arrays from model.safetensors.index.json.
2. first_k_dense_replace=1 dense-MLP layer-0 packing. Layer 0 uses a plain MLP with intermediate_size=10944, not shared_n_int=2816. The shared-expert slot needs a per-layer width or a separate L=0 buffer.
3. RoPE interleave validation. dispatch_attn already has the rope_interleave branch and a rope_interleave_f32 PSO is bound; needs numerical validation vs HF's apply_rotary_pos_emb_interleave.
4. MoE router sigmoid-then-top-k (moe_router_v3 PSO): needs validation that V2-Lite's topk_method=\"greedy\" path (no bias, no group restriction, no norm_topk_prob) is correctly handled at runtime.
5. Untied LM head: V2-Lite has tie_word_embeddings=False. The dispatch already falls back to w_embed if w_lm_head is null — a dedicated w_lm_head load path is required.

Download

DeepSeek-V2-Lite weights were started downloading on derek (~/SuperKittens/SuperKittens/model_weights/DeepSeek-V2-Lite) under tmux session dsdl. Progress at PR-open time: ~8.5 GB / ~31 GB.

Test plan

• python3 -c \"from SuperKittens.inference.registry import get_spec; print(get_spec('deepseek-v2-lite'))\" succeeds
• python3 -c \"import SuperKittens as sk; sk.load('deepseek-v2-lite')\" raises NotImplementedError with the documented blocker list (not an import or attribute error)
• [follow-up PR] argmax for [5, 100, 200, 300, 400] matches HF transformers' top-1
• [follow-up PR] m.chat(\"Hi\") produces a coherent first token

Why ship this as-is

The original 6-hour wall-clock plan called for landing 5 commits including the weight loader, kernel fixes (RoPE interleave wiring, dense-MLP layer-0 branch, MoE router sigmoid path, untied LM head), and end-to-end validation on derek. The weight download alone is ~31 GB at a ~50 MB/s ceiling and the kernel/loader bisection is multi-day work. Per the brief's hard-stop guidance, this PR ships the parts that are clean and mergeable today — the registry surface, the Config parsing, the tokenizer family entry — so the next PR can focus entirely on the loader + kernel bring-up without re-litigating the Python plumbing.

🤖 Generated with Claude Code