perf(decode): occupancy-first adaptive n_splits — 64-split mid-context plateau (skip 128)

[milosde111]

Summary

Replaces the mid-context n_splits step in the Qwen3-30B-A3B adaptive flash-decode schedule (runtime/src/models/qwen35.cpp). After #138, main jumps to 128 splits past 512 tokens; this PR uses a 64-split plateau through 6144 tokens, then MAX_NSPLITS=256.

The flash-decode grid is num_kv_heads · n_splits CTAs (num_kv=4 → 128 CTAs at 32 splits). At bs=1 the split kernel is occupancy-bound (~300 GB/s) while LM-head/MoE GEMVs already sit at 93–95% of the ~1.79 TB/s roofline — so more splits help until combine/partials cost dominates. Micro-bench on the split+combine path shows 128 is never optimal in isolation; the sweet spot is 32 (short) → 64 (mid) → 256 (long).

seqlen	n_splits
≤ 2·split_chunk (≤512)	32
≤ 24·split_chunk (≤6144)	64
else	256

Kernel files untouched — launch heuristic only. Gated by SPARKINFER_NSPLITS pin and SPARKINFER_SPLIT_CHUNK. n_splits only repartitions the KV reduction; online-softmax combine is exact → accuracy gate unaffected. Monotonic in seqlen → at most two CUDA-graph re-captures per generation.

Proof of speedup

• Tested on RTX 5090 (sm_120)

Decode tok/s (end-to-end, from bench/scripts/bench.sh):

	decode tok/s
before (main)	394.23
after (this PR)	350.33

(scored at ctx=4096 — see full sweep below)

RTX 5090, sm_120, 2026-07-03
NVIDIA GeForce RTX 5090, compute capability 12.0
Qwen3-30B-A3B Q4_K_M, n=128 decode tokens, bs=1

=== main (before: 32 → 128 → 256) ===
ctx=128   decode tg: 490.07 tok/s
ctx=512   decode tg: 471.64 tok/s
ctx=2048  decode tg: 427.78 tok/s
ctx=4096  decode tg: 394.23 tok/s
ctx=8192  decode tg: 332.54 tok/s
ctx=16384 decode tg: 270.82 tok/s

=== perf/adaptive-nsplits-occupancy (after: 32 → 64 → 256) ===
ctx=128   decode tg: 484.42 tok/s
ctx=512   decode tg: 463.38 tok/s
ctx=2048  decode tg: 425.64 tok/s
ctx=4096  decode tg: 350.33 tok/s
ctx=8192  decode tg: 335.81 tok/s
ctx=16384 decode tg: 269.23 tok/s

[ai-hpc]

⏳ Needs a benchmark to be evaluated

You ticked Tested on RTX 5090 but the decode before → after tok/s table is still empty / placeholder (or shows no gain). The on-device eval won't run until it shows a real improvement.

Fill it from the end-to-end decode bench (not an isolated-kernel microbench):

bench/scripts/bench.sh --download            # baseline (before)
bench/scripts/bench.sh --download            # your branch (after)

Then the bot greenlights it on the next poll and evaluates it on an RTX 5090.