[None][perf] Optimize MiniMax-M3 NVFP4 context (prefill) path on Rubin (SM107) by peihu-nv · Pull Request #20629 · NVIDIA/TensorRT-LLM

peihu-nv · 2026-10-19T21:16:47Z

Description

Depends on #20623 (Rubin decode targets, which depends on #20306). This PR adds only the last commit; please review that commit.

This PR speeds up the MiniMax-M3 NVFP4 context (prefill) worker on Rubin (sm_107). Changes are on by default and engage on sm_107 only; each change that alters numerics or adds a kernel has an environment kill switch, and other GPUs keep the stock path.

Performance

Cold prefill (no cached prefix) on the TP2 context worker (Rubin, MiniMax-M3 NVFP4, Eagle3): 1,807–1,902 ms at 165K tokens (43–46K tok/s/GPU), 3.9–4.1 s at 323K and 9.7–10.0 s at 650K, about 1.3× faster at 165K and 1.5× at 650K than an earlier stock-path image (measured on different nodes; indicative).

Projections and boundaries

Fused QKV-index GEMM + cache producer (kernels/rubin/m3_qkv_fusion, TRTLLM_CTX_QKV_FUSED): above 1024 rows one MXFP8 GEMM (2-CTA, CLC tile scheduler) runs the QKV/index projection and writes the NVFP4 K/V and FP8 index-K caches from its epilogue; breakable-graph buckets take it through an eager break.
CLC MXFP8 GEMM (modules/m3_ctx_clc_gemm.py, TRTLLM_CTX_CLC_GEMM) for o_proj, the shared-expert gate_up and the dense MLP at ≥ 1024 rows.
Fused boundary norm + quantize (TRTLLM_CTX_NORM_MXFP8, TRTLLM_CTX_NORM_FP4): at ≥ 8192 rows the residual RMSNorm after the all-reduce also emits the next linear's MXFP8 input and the routed experts' NVFP4 input.
SwiGLU-OAI + MXFP8 quantize for the dense MLP (TRTLLM_CTX_SWIGLU_MXFP8), and one-entry MXFP8 hand-off slots between producers and the consuming linear, kept out of compiled graphs.

Attention

FP8 dense prefill on FA4 over the paged cache (fmha/msa_prefill.py, kill TRTLLM_MSA_DISABLE_DENSE_FA4); it needs flash-attn-4 4.0.0b19 with the FP8 rescale_threshold change (3rdparty/patches/fa4_fp8_rescale0.patch, applied to a copy of the installed package by examples/minimax_m3/rubin/setup_fa4.sh), and without it the dense layers keep fmha_sm100.
MSA combine writes the o_proj input as MXFP8 (TRTLLM_CTX_COMBINE_MXFP8), plus prefill tile selection, a segmented/split indexer proxy with SM-wide KV splits, and an in-place NVFP4 prefill output.
Eagle3 drafter: context attention on each request's last row only (TRTLLM_CTX_DRAFTER_LAST_ROW), o_proj/MLP only on the rows the draft loop reads (TRTLLM_CTX_DRAFTER_DEAD_ROWS), and the partial last page's K/V tails zeroed on private pages (TRTLLM_CTX_DRAFTER_KV_TAIL_ZERO).
MSA prefill host path: the adaptive plan's split decision is deferred, the k2q CSR build uses a scratch arena and per-shape buffers with a direct extension call, and per_token_valid_blocks is one NumPy pass.
MSA dependency (3rdparty/patches/msa_strided_paged_kv.patch): two correctness fixes for every GPU (V tail zeroed past kv_valid_cols, griddepcontrol.wait restored); its performance changes are gated to sm_107 at runtime.

MoE, router and scheduling

Deterministic CuteDSL NVFP4 MoE (TLLM_CUTEDSL_MOE_DETERMINISTIC): FC2 per permuted row with a fixed-order unpermute, optionally folding the shared-expert output (TLLM_M3_MOE_FOLD_SHARED).
Batch-invariant fixed-tile TF32 router for context steps (TRTLLM_CTX_ROUTER_TF32_FIXED), and the TF32 tensor-core router for 9–16-row generation steps on the built-in model (TRTLLM_M3_ROUTER_TC_GEN). The context router GEMM splits K into 8 fixed slices up to 8192 rows.
Context append reserve (opt-in, TLLM_CTX_APPEND_RESERVE=<tokens>): behind a long prefill a reserved token budget lets short appends join the same step, optionally shortest-first (TLLM_CTX_APPEND_RESERVE_SJF); unset, every model keeps the stock chunking.
Breakable prefill CUDA graphs capped at 2048 rows, and an example context-worker recipe with an opt-in FP4 MoE tactic table (TLLM_FP4_MOE_TACTIC_TABLE).

The sm_107 TP2 target (modeling_v2)

Context steps past 48 rows route on the fixed-tile TF32 router.
Prefill steps with at least 4096 context rows take the built-in CTX stack above (fused QKV GEMM + producer, the fused boundary norm with its MXFP8 / NVFP4 outputs, the built-in MoE forward). Eager steps only; Eagle3 capture layers keep the target's own boundary.
An example aggregated TP2 serving recipe in examples/minimax_m3/rubin/agg/.

Numerics. Most items are bitwise equal to their off switch. The fused QKV GEMM, the CLC GEMM, the FA4 dense prefill, the deterministic MoE FC2, the TF32 routers and the router's split-K use a different reduction order: deterministic, but not bitwise to the stock kernels. All of them except the router split-K are chunk-size invariant; the TF32 router splits K only up to 8192 rows, so its reduction order changes at 8192 rows and router logits (and, at near-ties, the selected experts) can differ between chunks of at most 8192 rows and larger ones.

External dependencies. flash-attn-4 is pinned to 4.0.0b19: setup_fa4.sh checks the FA4 file and the patch by SHA256. The MiniMax-M3 Rubin maintainers own it; an FA4 update re-derives the patch, re-runs the FP8 dense-prefill check and updates both pins, and the patch is dropped once FA4 ships the change.

Test Coverage

Bitwise vs the reference build on the TP2 context worker (cold 163,840-token prompt plus appends of 6 to 2,000 tokens): identical first-token logits on every turn, and every item's engagement line on both ranks.
Accuracy vs the reference build: GSM8K (1319) greedy and sampled with no significant McNemar difference, and identical per-request output and Eagle3 acceptance over 264 natural long-context requests.
Prefill timing parity with the reference build, same node, both run orders.
Unit tests on Rubin, one session per directory: attention/ (fmha, sparse/msa incl. the sm_107 gate cells), models/test_minimax_m3.py, moe/, thop/serial/, modeling/, the sm_107 modeling_v2 TP2 / TP4 / TP8 target directories, and the scheduler's append-reserve tests, all passing.
CI scope: the device-free tests (arch guards, resolution, tactic table) are cpu_only and listed in l0_cpu.yml; the sm_107 GPU tests have no CI stage yet (no Rubin test list), and the TF32 router / unpermute / fold tests exercise paths used only on the sm_107 context worker, so they are not registered on other GPUs in this PR.

PR Checklist

Please review the following before submitting your PR:

PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.
PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.
Test cases are provided for new code paths (see test instructions)
If PR introduces API changes, an appropriate PR label is added - either api-compatible or api-breaking. For api-breaking, include BREAKING in the PR title.
Any new dependencies have been scanned for license and vulnerabilities
CODEOWNERS updated if ownership changes
Documentation updated as needed
Update tava architecture diagram if there is a significant design change in PR.
The reviewers assigned automatically/manually are appropriate for the PR.
Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

… step-loop runtime hooks Adds the modeling_v2 target minimax_m3_nvfp4__sm_103__tp4 (GB300, TP4, Eagle3 one-model speculation) and the executor seams it opts into. - Runtime hooks (off unless a model declares them): packed step inputs, skipped unchanged H2D uploads, a private graph metadata pool, sampler greedy-token and step-fold paths, MSA runtime policy, and rank-merged autotuned tactics. Greedy-only outputs are kept only when nothing reads the logits (no guided decoder, non-empty batch). - TP4 decode target: fused attention with long-context selection, a pre-wait-staged NVFP4 QKV/indexer producer and index scorer, multicast all-reduce + RMSNorm boundaries with the MoE finalize folded in, MXFP8 GEMVs, CuTe DSL MoE engines, an exact greedy and sampled verify, a fused draft loop, and lossless 12-bit coded BF16 target and draft heads. - Explicit dispatch: every covered shape is a certified catalog cell; under modeling_v2: require a covered shape that cannot run raises. Every item is on by default with one constant that turns it off; no environment reads. - Numerics match stock up to summation order; the long-context selector breaks >1024-way score ties by its contract (smaller block). Signed-off-by: peihengh <259410613+peihu-nv@users.noreply.github.com>

Pick the row source explicitly, raise when neither logits nor greedy tokens are present, and assert logits before the fast greedy kernel. Signed-off-by: peihengh <259410613+peihu-nv@users.noreply.github.com>

Adds the modeling_v2 target minimax_m3_nvfp4__sm_103__tp8 (GB300, 8 ranks over two trays, tensor-parallel attention and experts, one-model Eagle3 or no speculation) and routes world = tp = moe_tp = 8 to it; the tp4 branch of the router is unchanged. - Decode path: the tp4 target's exact decode path at the TP8 geometry, plus the TP8 items: the 1..8-token MoE engine at expert width 384 (logits replicated 32x at <= 4 tokens), an 8-head one-launch long-context sparse decode staged before its grid wait at <= 8 rows, a fused index score + top-16 select for spans of <= 2 requests, the producer releasing its dependents on entry, the long-row select's early release at <= 8 rows, the TP8 MXFP8 GEMV table, fixed-tactic low-M BF16 GEMMs for the drafter linears, the drafter decode's KV split capped at 16K for one request, BF16 target and draft heads, and the drafter fc forked beside the verify head. - Isolation: the target imports nothing from the tp4 target; the MoE engine and the MX GEMV op it changes are its own copies inside the target. The two new kernels are compiled ops (cpp/tensorrt_llm/kernels/minimaxM3Tp8, trtllm::minimax_m3_tp8_*). A drift test pins the files copied unchanged from the tp4 target. - fmha/flashinfer_trtllm_gen.py applies a layer's KV-split cap (msa_decode.dense_decode_max_seq_len); a no-op unless a layer sets one. - Same contract as the tp4 target: every kernel a catalog entry with CELLS and a torch reference, dispatch.py listing each fast path's rows, a covered shape that cannot run raising under modeling_v2: require, kernels built before warmup, every item on by default with one constant to turn it off. - Numerics match stock up to summation order (BF16 heads and drafter, NVFP4 experts, MXFP8 projections). Signed-off-by: peihengh <259410613+peihu-nv@users.noreply.github.com>

CUtensorMap already carries its own alignment, which is stricter than 64 bytes in newer CUDA toolkits; clang-hosted nvcc rejects the weaker alignas with "alignment cannot be set to less than the default alignment" (SBSA build). Signed-off-by: peihengh <259410613+peihu-nv@users.noreply.github.com>

Resolve two additive conflicts with the Qwen4-exp modeling_v2 target by keeping both router entries and both routing tests. Port the sampler's greedy_tokens_usable to main's request_row_params after the TorchSampler strategy refactor (NVIDIA#20188) removed _request_strategy. Signed-off-by: peihengh <259410613+peihu-nv@users.noreply.github.com>

- Register fake implementations for the new step_inputs_scatter and MiniMax-M3 router, low-M GEMM, producer and allreduce-norm ops. - Give the warmup tests' stand-in engines the default runtime hooks the decoder runner now reads. - Leave the identity table unset in the piecewise-attention metadata mock. - Stub the eager K/V view resolution only while the runtime-policy fixture builds its manager, so the pool-view tests still exercise get_buffers. Signed-off-by: peihengh <259410613+peihu-nv@users.noreply.github.com>

Restructure the minimax_m3_nvfp4__sm_103__tp4 and __tp8 targets into the flat layout the modeling_v2 exemplars use, with no change to what they compute. Each target owns its own copy of every file (the targets share no code): * modeling.py: the dispatch tables (FAST_PATHS, row ranges, miss/uncovered policy), the flat core, PrefillTarget / DecodeTarget, the one-model Eagle3 drafter and worker, the step fold, runtime hooks and the shell. The core selects the target on num_contexts alone: every pure generation step is DecodeTarget's (its decode kernels up to 48 tokens, its own generic layers for wider steps or embeddings), every step with context rows PrefillTarget's. The two targets share no body: each owns its layer, attention and MoE compositions, and its step_args is the one projection of the MSA metadata its forward reads (DecodeTarget states num_contexts=0 / num_ctx_tokens=0). Shared module-level code is limited to single-call op adapters that read no step metadata. * weights.py: the language-model and drafter weight tables (host-side TP slicing, consumed checkpoint keys released during the load; at TP8 the 4 KV / index heads are duplicated on rank pairs as the built-in loader does). * ops/: catalog entries with certified cells for every call the model makes; kernel wrappers route through their entries. The MSA and drafter attention entries carry a numeric reference and GPU tests that drive every cell against real caches and prepared metadata. * kernels/: the CuTe DSL GEMV, MoE engine and embedding+first-norm kernels move into each target (the old shared cute_dsl_kernels/minimax_m3 copy is removed); capacity annotations on every constant. * The legacy dispatch/layers/linears/runtime/spec_fold/spec_worker modules are removed; the unit tests are ported to the flat targets. * The unfused QKV / index projection runtime (fuse_qkv_index_projection=False, FP8 indexer, FP8 or NVFP4 main KV) runs as the built-in model does. Routing (models/minimax_m3/routing.py) now declines what the targets do not build or run, so modeling_v2="auto" falls back to the built-in model and "require" names the missed criterion: * Eagle3 drafters other than the targets' one-layer drafter (read from the drafter checkpoint's config.json). * Runtimes other than MSA with an FP8 indexer and an FP8 or NVFP4 main KV cache. The resolver hands routing modules that set ROUTES_ON_RUNTIME a RuntimeFacts (sparse attention config and the KV cache quantization the engine will run with); only MiniMax-M3 sets it. _validate_runtime stays as the backstop. Adds a whole-model accuracy test under modeling_v2="require" (GSM8K without speculation, and with one-model Eagle3 plus a chat-GSM8K acceptance probe), on the 4-GPU GB300 list (TP4) and the 2-node GB300 list (TP8). Bitwise identical to the previous targets on GB300 (tokens and per-step outputs on every rank, no speculation and Eagle3, fused and unfused projections, TP4 and TP8, including pure generation steps wider than the decode kernels' 48 tokens). Signed-off-by: peihengh <259410613+peihu-nv@users.noreply.github.com>

Resolve cache_manager.py: keep the decode shortcuts and pooled views, and the HND main KV storage with the strided NHD view from NVIDIA#20159. Signed-off-by: peihengh <259410613+peihu-nv@users.noreply.github.com>

…ls in M3 folders Review follow-up, no logic change: - cpp/tensorrt_llm/kernels/stepInputsScatter.{cu,h} move to kernels/minimaxM3/. - The MiniMax-M3 torch ops (thop/minimaxM3*Op.cpp and stepInputsScatterOp.cpp) move to thop/minimaxM3/, which adds them to th_common_attention with target_sources() (as thop/moe/ does for th_common_moe). - The fake impls of the ops the decode targets add (step_inputs_scatter, minimax_m3_mxfp8_low_m_gemm, minimax_m3_router_gemv / _gemv_fp4 / _tc, minimax_m3_producer_pw, minimax_m3_mc_allreduce_norm) move from custom_ops/cpp_custom_ops.py to modeling_v2/models/minimax_m3/fake_ops.py, imported by the MiniMax-M3 modeling_v2 package, by pyexecutor/step_inputs.py and by the fake-registration test. Signed-off-by: peihengh <259410613+peihu-nv@users.noreply.github.com>

…ough runtime_hooks_of The autotuner and CUDA-graph warmup passes read merge_tuned_tactics_across_ranks from the runner's cached hooks, which only the constructor sets; engines built another way (main's warmup tests drive DecoderRunner._run_autotuner_warmup on a stub) failed with an AttributeError. They now read runtime_hooks_of(self.model), the all-off default for a model that declares no hooks, the same value the constructor caches. Signed-off-by: peihengh <259410613+peihu-nv@users.noreply.github.com>

… HND storage layout Since NVIDIA#20159 the dense KV storage is head-major and get_buffers hands a token-major request a transposed view. The reused pool views (reuse_pool_views) cached the untransposed view and transposed it on every call, so a reused view was a new tensor each time. The transpose is now part of the view's cache key and applied before caching. The runtime-policy tests expect the view in the manager's own layout over its physical one. Signed-off-by: peihengh <259410613+peihu-nv@users.noreply.github.com>

Resolve runner.py after NVIDIA#20134 split DecoderRunner by phase: the model's runtime hooks now reach the phase that uses them. The runner keeps the step scopes, the graph-runner config and the greedy-token logits; the input preparer the H2D skip copies, the step-input pack and the speculative step fold; the metadata the MSA policy and the greedy RNG upload skip; the graphs the short/long split and the post-capture tactic merge; the warmup the post-autotune tactic merge (through runtime_hooks_of). Signed-off-by: peihengh <259410613+peihu-nv@users.noreply.github.com>

peihu-nv · 2026-10-20T02:29:05Z

/bot run

tensorrt-cicd · 2026-10-20T02:35:17Z

PR_Github #79102 [ run ] triggered by Bot. Commit: c1400ba Link to invocation

tensorrt-cicd · 2026-10-20T03:14:42Z

PR_Github #79102 [ run ] completed with state FAILURE. Commit: c1400ba
/LLM/main/L0_MergeRequest_PR pipeline #65161 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

…ares is_spec_decode DecoderRunner reads DecoderRunnerConfig.is_spec_decode at construction (the spec step fold binds the engine buffers). The compiled-body validation test builds the runner from a SimpleNamespace config; give it the field the real config always has. Signed-off-by: peihengh <259410613+peihu-nv@users.noreply.github.com>

Resolve the modeling_v2 router against NVIDIA#20267: route() now takes the resolver's KV-cache algorithm as its fifth argument for every routing module, and modules that set ROUTES_ON_RUNTIME (MiniMax-M3) still receive runtime=RuntimeFacts; the MiniMax-M3 route() accepts the new argument. Keep both sides' new helpers (RuntimeFacts / resolved_mode and tp_reaches_across_nodes) and both test-list additions. Signed-off-by: peihengh <259410613+peihu-nv@users.noreply.github.com>

peihu-nv · 2026-10-20T12:18:37Z

/bot run --disable-fail-fast

Resolve the MiniMax-M3 index-score kernels against NVIDIA#19275 (c146094), which split cute_dsl_kernels/ into the domain kernel trees: the TP4 and TP8 m3_index_score imports now take cute_ptx_utils and utils from tensorrt_llm._torch.kernels.blackwell, and the docstring points at attention/kernels/blackwell/minimax_m3_index_decode_score.py. Both moved modules are unchanged (pure renames). Signed-off-by: peihengh <259410613+peihu-nv@users.noreply.github.com>

… (sm_107) TP2 / TP4 / TP8 in modeling_v2 Three modeling_v2 targets for the MiniMax-M3 NVFP4 checkpoint on Rubin (sm_107), minimax_m3_nvfp4__sm_107__tp2, __tp4 and __tp8, in the flat layout of the sm_103 targets: one modeling.py per target, a weights.py weight table and an ops/ catalog entry with certified cells and a test for every kernel the model calls. Routing picks a target only for the matching checkpoint, SM and width. sm_107 enablement: Arch.SM_107 in the catalog and the (10, 7) routing entries; the compiled trtllm::minimax_m3_sm107* kernels (built for sm_107 only; torch ops in thop/minimaxM3/, fake impls in the M3 fake_ops module) and the shared TF32 router GEMV and spec-tail draft-input kernels; the CuTe DSL decode MoE engines, MX GEMV and fused embedding + first RMSNorm; the decode runtime pieces (step-input pack, graph runner hooks, step fold, MSA decode metadata reuse, the SM107 NVFP4 -> FP8 KV fallback skipping MiniMax-M3 MSA, late-compile detection), each inert until a target's runtime hooks turn it on; the FMHA fallback serving SM107 paged prefill at the TRT-LLM Gen extra page sizes (the 128-token KV pages its MiniMax-M3 caches opt into); and the scoped PrimTS work-tile rebind in the vendored FlashInfer patch. The TP2 target serves decode steps of 1..192 rows: context steps of 9..48 rows route on the tensor-core router, steps with context rows run under breakable prefill CUDA graphs (the drafter's attention in an eager break) and take the MC finalize boundary at 9..48 rows, and decode steps of >= 33 requests take the stock MSA merge path (MergeMxfp8.MAX_ROWS 128; no canon profile exceeds 96 rows). Runtime items: pinned FlashInfer CuTe DSL MXFP8 tactics above 32 rows on SM107, for models whose runtime hooks opt in (the MiniMax-M3 sm_107 targets do); the AllReduce tuner leaves NCCL_SYMMETRIC out of its candidates only when TRTLLM_AR_TUNE_NCCL_SYMMETRIC=0 (opt-in; the default 1 keeps the old candidates); PyExecutor emits a request's first content after two iterations when the model's runtime hooks ask for it (the MiniMax-M3 sm_107 targets do; other models keep the stock schedule). No lossy steps: every kernel keeps the stock op's precision. The TP4 and TP8 targets produce bitwise the same tokens and per-step outputs as the pre-restructure sm_107 targets, with and without Eagle3. Two known deltas against the previous wide TP2 target (the 9..128-row cells this commit folds in), both measured with greedy sampling: under Eagle3, at 12 or more concurrent requests the drafter's BF16 linears past its low-M tactic table can resolve a near-tie differently from the in-tree drafter, so some draft tokens differ, while the target's tokens are identical and its logits bitwise; without speculation (not a production configuration), decode steps of 9..48 rows are bitwise on the first such step and differ after it (cause not isolated), with identical output tokens in every comparison. Steps of 1..8 rows are bitwise to the previous TP2 target. Signed-off-by: peihengh <259410613+peihu-nv@users.noreply.github.com>

peihu-nv · 2026-10-20T15:22:37Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-10-20T15:30:32Z

PR_Github #79290 [ run ] triggered by Bot. Commit: 9fe0dfb Link to invocation

The main merge moved DecoderRunner._preprocess_inputs to DecoderInputPreparer.preprocess_inputs (NVIDIA#20134); the test still built a DecoderRunner, so every case failed with an AttributeError. Build the input preparer instead, with the overlap offsets under its buffers. Signed-off-by: peihengh <259410613+peihu-nv@users.noreply.github.com>

yuxianq · 2026-10-27T22:54:09Z

@PerkzZheng, could you help to review this PR as trt-llm-torch-attention-devs? Thanks~

Assignment explanation

Components	kernels/sparse (30%); kernels/auxiliary (25%); fmha/msa-sparse-gqa (15%); runtime/sparse-cache (10%); fmha/fallback (5%); kernels/trtllm-gen (4%); kernels/prims-ts (4%); fmha/orchestration (3%); integration/dependencies (2%); integration/build-config (2%)
Tags	algorithm:sparse (70%); algorithm:dense (30%); specific-algorithm:minimax-m3 (75%); specific-algorithm:msa (25%); phase:prefill (55%); phase:decode (45%); features:quantization (40%); features:paged-kv (25%); features:rope (15%); features:qk-norm (10%); features:speculative-decoding (10%); context:llm (100%)
PR effort	13 points — New optimized execution paths for existing MiniMax-M3 attention support feature classification. Cluster-level sparse attention, fused quantized cache production, producer-consumer handoff and dispatch/cache contracts justify 13 attention-review points independently of file count. The supplied scope includes both prefill and decode.
Importance	Important — Sensitive attention execution changes span the cluster/DSMEM sparse-decode contract in sparseDecodeLc.cu, multi-token fused QKV/cache production in fusedQKNormRopeKernel.cu, MXFP8 combine handoff and optional cache-mapping tensors. These require numerical, synchronization and cache-layout reasoning.
Retained GitHub review requests	litaotju: preserve direct code-owner review request
Affinity (0–1)	0.448 component/tag match; 0.336 after PIC preference
Outstanding points at selection	42 → 55 (+13)
Points already reserved on this PR	0
Effective weight	1
Completed review points (preceding 168 hours)	27
Combined points after assignment	82
Demand-derived target	71.00 points (soft share)
Additional fair share before assignment	2.00 points
Share explanation	Whole PR / qualification or coverage constraint exceeds a soft share; assignment is allowed.
Selection rule	affinity with 0.3-share guard
Assignment scope (PerkzZheng)	all

Affinity and workload are snapshots at selection, not live counters. Existing reservations on this PR are counted once.

peihu-nv added 11 commits October 14, 2026 20:10

[None][fix] Satisfy mypy on the MiniMax-M3 sampler fast-greedy path

c76faa4

Pick the row source explicitly, raise when neither logits nor greedy tokens are present, and assert logits before the fast greedy kernel. Signed-off-by: peihengh <259410613+peihu-nv@users.noreply.github.com>

[None][chore] Merge main into MiniMax-M3 megakernel PR

6d9af78

Resolve cache_manager.py: keep the decode shortcuts and pooled views, and the HND main KV storage with the strided NHD view from NVIDIA#20159. Signed-off-by: peihengh <259410613+peihu-nv@users.noreply.github.com>

github-actions Bot assigned peihu-nv Oct 19, 2026

peihu-nv force-pushed the peihengh/m3-vr-sm107-ctx branch from 43a4124 to c1400ba Compare October 20, 2026 02:28

peihu-nv mentioned this pull request Oct 20, 2026

[None][feat] MiniMax-M3 NVFP4 low latency decode megakernel for Rubin (sm_107) TP2 / TP4 / TP8 in modeling_v2 #20623

Open

1 task done

peihu-nv added 2 commits October 19, 2026 20:18

peihu-nv mentioned this pull request Oct 20, 2026

[None][feat] Enable MiniMax-M3 on Rubin (SM107): NVFP4 KV exemption and 128-token paged FMHA kernels #20167

Merged

1 task done

peihu-nv force-pushed the peihengh/m3-vr-sm107-ctx branch from c1400ba to 5a3dd8b Compare October 20, 2026 12:18

peihu-nv added 2 commits October 20, 2026 05:40

peihu-nv changed the title ~~[None][perf] MiniMax-M3 NVFP4 context (prefill) path on Rubin (SM107)~~ [None][perf] Optimize MiniMax-M3 NVFP4 context (prefill) path on Rubin (SM107) Oct 20, 2026

peihu-nv force-pushed the peihengh/m3-vr-sm107-ctx branch from 5a3dd8b to 9fe0dfb Compare October 20, 2026 15:22

peihu-nv requested review from a team, Wanli-Jiang, WeiHaocheng and litaotju as code owners October 27, 2026 22:39

peihu-nv requested review from Mgluhovskoi, QiJune, Shixiaowei02, heyuhhh, kaiyux, leslie-fang25, liji-nv, lowsfer, mzweilz, niukuo, pcastonguay and zhaoyuanh-nvidia October 27, 2026 22:39

yuxianq requested review from PerkzZheng and removed request for QiJune and heyuhhh October 27, 2026 22:54

Navigation Menu

Repository navigation

[None][perf] Optimize MiniMax-M3 NVFP4 context (prefill) path on Rubin (SM107) - #20629

[None][perf] Optimize MiniMax-M3 NVFP4 context (prefill) path on Rubin (SM107)#20629
peihu-nv wants to merge 31 commits into
NVIDIA:mainfrom
peihu-nv:peihengh/m3-vr-sm107-ctx

peihu-nv commented Oct 19, 2026 •

edited

Loading

Uh oh!

peihu-nv commented Oct 20, 2026

Uh oh!

tensorrt-cicd commented Oct 20, 2026

Uh oh!

tensorrt-cicd commented Oct 20, 2026

Uh oh!

peihu-nv commented Oct 20, 2026

Uh oh!

peihu-nv commented Oct 20, 2026

Uh oh!

tensorrt-cicd commented Oct 20, 2026

Uh oh!

yuxianq commented Oct 27, 2026 •

edited

Loading

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

peihu-nv commented Oct 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Performance

Projections and boundaries

Attention

MoE, router and scheduling

The sm_107 TP2 target (modeling_v2)

Test Coverage

PR Checklist

GitHub Bot Help

Uh oh!

peihu-nv commented Oct 20, 2026

Uh oh!

tensorrt-cicd commented Oct 20, 2026

Uh oh!

tensorrt-cicd commented Oct 20, 2026

Uh oh!

peihu-nv commented Oct 20, 2026

Uh oh!

peihu-nv commented Oct 20, 2026

Uh oh!

tensorrt-cicd commented Oct 20, 2026

Uh oh!

yuxianq commented Oct 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

peihu-nv commented Oct 19, 2026 •

edited

Loading

yuxianq commented Oct 27, 2026 •

edited

Loading