Skip to content

[None][perf] Optimize MiniMax-M3 NVFP4 context (prefill) path on Rubin (SM107) #20629

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
peihu-nv wants to merge 31 commits into
base: main
Choose a base branch
from
+282,567 −4,934
Open

[None][perf] Optimize MiniMax-M3 NVFP4 context (prefill) path on Rubin (SM107) #20629

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
658817a
[None][feat] MiniMax-M3 NVFP4 TP4 decode target for modeling_v2, with…
peihu-nv Oct 14, 2026
c76faa4
[None][fix] Satisfy mypy on the MiniMax-M3 sampler fast-greedy path
peihu-nv Oct 14, 2026
24cccaf
[None][feat] MiniMax-M3 NVFP4 TP8 decode target for modeling_v2
peihu-nv Oct 14, 2026
bb8674c
[None][fix] Drop alignas(64) on MiniMax-M3 CUtensorMap members
peihu-nv Oct 15, 2026
eeb1ffd
[None][chore] Merge main into MiniMax-M3 megakernel PR
peihu-nv Oct 15, 2026
a427f56
[None][fix] Fix MiniMax-M3 megakernel PR unit tests on current main
peihu-nv Oct 15, 2026
8cb295c
[None][refactor] MiniMax-M3 modeling_v2: flat TP4/TP8 target layout
peihu-nv Oct 19, 2026
6d9af78
[None][chore] Merge main into MiniMax-M3 megakernel PR
peihu-nv Oct 19, 2026
dcdbd8c
[None][chore] MiniMax-M3: keep the M3 torch ops, kernels and fake imp…
peihu-nv Oct 19, 2026
dd6a5b7
[None][fix] Decoder runner warmup: read the model's runtime hooks thr…
peihu-nv Oct 19, 2026
f1d15d6
[None][fix] MiniMax-M3 KV cache manager: reused pool views follow the…
peihu-nv Oct 19, 2026
87b9d8d
[None][chore] Merge main into MiniMax-M3 megakernel PR
peihu-nv Oct 19, 2026
8ccd79f
[None][test] Decoder runner init test: the runner config fixture decl…
peihu-nv Oct 20, 2026
5a9c484
[None][chore] Merge main into MiniMax-M3 megakernel PR
peihu-nv Oct 20, 2026
2343e84
[None][chore] Merge main into MiniMax-M3 megakernel PR
peihu-nv Oct 20, 2026
7333ac8
[None][feat] MiniMax-M3 NVFP4 low latency decode megakernel for Rubin…
peihu-nv Oct 20, 2026
8b49063
[None][test] Port the kv_lens_update_once test to DecoderInputPreparer
peihu-nv Oct 20, 2026
cf8a77e
[None][perf] MiniMax-M3 NVFP4 context (prefill) path on Rubin (SM107)
peihu-nv Oct 20, 2026
300c1c1
[None][chore] MiniMax-M3 modeling_v2: standard route interface, layou…
peihu-nv Oct 21, 2026
284e5af
[None][chore] Merge the MiniMax-M3 megakernel PR head 300c1c1e64
peihu-nv Oct 21, 2026
ffa76ab
[None][chore] MiniMax-M3 sm107 targets: kernels under ops, routing de…
peihu-nv Oct 21, 2026
a8f0b89
[None][chore] Merge the MiniMax-M3 Rubin decode PR head ffa76abd2c
peihu-nv Oct 21, 2026
076590d
[None][fix] Run the MiniMax-M3 E4M3 cast probe on the CPU under any d…
peihu-nv Oct 22, 2026
246d06e
[None][chore] Merge main d1c54cb654 into the MiniMax-M3 megakernel PR
peihu-nv Oct 22, 2026
f71400b
[None][feat] modeling_v2 'default' mode: MiniMax-M3 GB300 targets on …
peihu-nv Oct 22, 2026
4f2d4ff
[None][chore] Merge main d4cfc12f19 into the MiniMax-M3 megakernel PR
peihu-nv Oct 22, 2026
eea8a02
[None][chore] Merge main 3205aeaedb into the MiniMax-M3 megakernel PR
peihu-nv Oct 26, 2026
15714e9
[None][chore] Merge the MiniMax-M3 megakernel PR head eea8a0213e (mai…
peihu-nv Oct 26, 2026
3dac8ae
[None][test] Move the MXFP8 CuTe pinned-tactics test next to the dens…
peihu-nv Oct 26, 2026
a521432
[None][chore] Merge the MiniMax-M3 Rubin decode PR head 3dac8aedcd (m…
peihu-nv Oct 26, 2026
413eabf
[None][fix] MiniMax-M3 prefill: zero the V tail past kv_len before fm…
peihu-nv Oct 26, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
11 changes: 11 additions & 0 deletions 3rdparty/patches/fa4_fp8_rescale0.patch
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
--- a/flash_attn/cute/flash_fwd_sm100.py
+++ b/flash_attn/cute/flash_fwd_sm100.py
@@ -2021,7 +2021,7 @@

rescale_threshold = (
8.0 if const_expr(self.q_dtype.width == 16) else
- 4.0 if const_expr(self.q_dtype.width == 8) else
+ 0.0 if const_expr(self.q_dtype.width == 8) else
0.0
)
softmax = SoftmaxSm100.create(
11,145 changes: 6,603 additions & 4,542 deletions 3rdparty/patches/msa_strided_paged_kv.patch
View file
Open in desktop

Large diffs are not rendered by default.

Loading
Loading