Repository navigation
feat: update grouped GEMM, fused MoE, and convolution support - #6
Merged
Merged
Conversation
… CK instances Port the standalone uniform Direct-to-LDS NN (dgrad) / TN (wgrad) kernels into the ck_tile framework as device-args instances: - gemm_pipeline_v7_uniform.hpp: uniform (non warp-specialized) pipeline, 256x256x16, 512 threads, 4-stage 16KB Direct-to-LDS ring, two-group wave phase interleave; V7UniformKind::NN walks a K-contiguous A, ::TN walks the [K, M] panel (both operands m32x16). - grouped_gemm_v7_uniform_epilogue.hpp: per-wave LDS slab staging, 16B vector stores, no cross-warp barrier. - instances 5000 (v7_bf16_nn) / 5001 (v8_bf16_tn), family v7, opt-in via CK_TILE_GROUPED_GEMM_ENABLE_BF16_V7_NN / _V8_TN (disable wins); selector gates: gfx936, bf16, K>=64 && K%16==0, N%256==0, G<=16, fixed K, no empty groups. - entries are NOT registry candidates: a dedicated archive carries the tuned codegen contract (-mllvm -support-768-vgprs et al.); the flat over-covered grid early-exits blocks past the last group's block_end (FindGroupId only converges in-range), and the LDS ring is DYNAMIC shared memory addressed from offset 0 per the reference kernels. - shared-header decoupling: kernel.hpp only includes the 10-line forward header (complete tile-parameter POD + enum); the pipeline body and the GemmConfigV7Uniform configs compile only inside the v7/v8 TUs, so tuning recompiles two translation units, not the archive. - grouped_gemm_v7_instance_test: correctness (CPU reference + default instance triangle) and perf through the versioned device-args ABI; selector/query tests extended to the 20-candidate registry. gfx936, groups=4 M=[8100,8100,8091,8091] N=4096 K=4096: NN 2.741 ms / 396.7 TF (default instance 6.508 ms / 167.1 TF) TN 2.700 ms / 402.7 TF
Add the CK Wasp producer-consumer pipeline, grouped single-group launch path, direct-store epilogue, and focused correctness/performance driver. Preserve the C/Cp stage schedule while hoisting producer wave-ID reads out of the K loop. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
AI Review发现 4 个问题。 [P2] conv 示例 pad 参数向 ConvParam 传入错位
[P3] 标注待删除的 TEMP-DEBUG 探针代码遗留在示例中
[P3] 计时区间内包含 hipMemset,测得的性能数据包括清零开销
[P3] 退化输出早退分支跳过所有 hipFree,泄漏设备显存
|
WG670
approved these changes
Oct 19, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds grouped GEMM tile-loop, AB-scale and WASP pipelines, integrates BF16 NN/TN kernels as CK instances, and adds fused MoE and FP32 convolution support. Includes GEMM argument checks and compiler compatibility fixes. Preserves existing public CI files and all 11 individual commits.
Validation: per-commit metadata and tree verification, complete Linux checkout, and strict Git object checks passed. GPU compilation, correctness, and performance tests were not rerun for this synchronization.
Please merge while preserving the branch commit IDs. Dependent AITER snapshots use exact public commit mappings; squash or rebase would require reconciling those mappings.