Skip to content

Add measured LLM decode kernel suite#3

Merged
WaffleBits merged 1 commit into
mainfrom
codex/decode-kernels-nsight-20260614
Jun 14, 2026
Merged

Add measured LLM decode kernel suite#3
WaffleBits merged 1 commit into
mainfrom
codex/decode-kernels-nsight-20260614

Conversation

@WaffleBits

Copy link
Copy Markdown
Owner

Summary

  • add QK dot, paged KV gather, fused selected attention, residual RMSNorm, and signed INT4 GEMV Triton kernels
  • check RTX 5070 Ti timing artifacts and a specification-based roofline report
  • install a filtered Nsight Compute capture path with an explicit host counter-permission error

Verification

  • ruff check .
  • pytest -q (13 passed)
  • measured GPU runs with correctness checks before timing

Claim boundary

Bandwidth and roofline traffic remain logical models. Nsight hardware counters are not claimed because Windows host counter access is disabled.

@WaffleBits
WaffleBits merged commit 47badda into main Jun 14, 2026
1 check passed
@WaffleBits
WaffleBits deleted the codex/decode-kernels-nsight-20260614 branch June 14, 2026 10:57
WaffleBits added a commit that referenced this pull request Jul 16, 2026
Co-authored-by: WaffleBits <19478926+WaffleBits@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant