From 3f53d669d71e6c47b2145e18ece25e86d9754f08 Mon Sep 17 00:00:00 2001 From: Alazar Manakelew Date: Thu, 14 May 2026 16:11:20 -0400 Subject: [PATCH] =?UTF-8?q?docs:=20add=20best.md=20=E2=80=94=20short=20dec?= =?UTF-8?q?ode=20tok/s=20table=20per=20model?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 5-line summary of best measured SK decode tok/s on lexie M4 vs llama.cpp, linked to the PR(s) that established each number. --- best.md | 10 ++++++++++ 1 file changed, 10 insertions(+) create mode 100644 best.md diff --git a/best.md b/best.md new file mode 100644 index 0000000..de4ba60 --- /dev/null +++ b/best.md @@ -0,0 +1,10 @@ +# Best measured decode tok/s + +All on lexie M4 (16 GB unified, no VMs). + +| Model | Quant | SK tok/s | llama.cpp | PR | +|---|---|---|---|---| +| Qwen3-0.6B | Q8_0 GGUF | **137** | 119 | #7 + #12-15 | +| Qwen3-8B | Q8_0 GGUF | **11.26** | 12.79 | #13 + #15 | +| Gemma4-E2B | bf16 + Q8_0 lm_head | **25.01** (64-tok) | 54.9 | #19 + #21 | +| Mamba2-130m | fp16 safetensors | **90** | 14.5 (HF fp32) | #4 |