Basic docs for universal gemm & ck-tile gemm. #2014

aosewski · 2025-03-25T11:36:46Z

Proposed changes

Please describe the motivation behind the pull request, whether it enables a new feature or fixes a bug. If there are associated pull requests or issues, please link them to the pull request.

Checklist

Please put an x into the boxes that apply. You can also fill these out after creating the PR. If you're not sure, please don't hesitate to ask.

I have added tests relevant to the introduced functionality, and the unit tests are passing locally
I have added the test to REGRESSION_TESTS list defined at the top of CMakeLists.txt in tests/CMakeLists.txt, IF the test takes more than 30 seconds to run.
I have added inline documentation which enables the maintainers with understanding the motivation
I have removed the stale documentation which is no longer relevant after this pull request
(If this change is user-facing) I have added release notes which provide the end users with a brief summary of the improvement from this pull request
I have run clang-format on all changed files
Any dependent changes have been merged

Discussion

If this is a relatively large or complex change, feel free to start a discussion by explaining why you chose the solution you did and what alternatives you considered

aosewski · 2025-03-25T11:38:22Z

@spolifroni-amd Please feel free to polish those docs and make it render better if necessary :)

aosewski · 2025-03-25T11:39:35Z

@bartekxk @aska-0096 could you please review?

include/ck/tensor_operation/gpu/device/impl/device_gemm_xdl_cshuffle_v3.hpp

include/ck/tensor_operation/gpu/grid/gridwise_gemm_xdl_cshuffle_v3.hpp

include/ck/tensor_operation/gpu/device/impl/device_gemm_xdl_cshuffle_v3.hpp

include/ck/tensor_operation/gpu/grid/gridwise_gemm_xdl_cshuffle_v3.hpp

include/ck/tensor_operation/gpu/device/impl/device_gemm_xdl_cshuffle_v3.hpp

include/ck/tensor_operation/gpu/grid/gridwise_gemm_xdl_cshuffle_v3.hpp

include/ck/tensor_operation/gpu/device/impl/device_gemm_xdl_cshuffle_v3.hpp

spolifroni-amd

There's a lot of issues that need to be addressed so it can be contextualized.

If you want a rewrite, I can open a PR for it.

include/ck/tensor_operation/gpu/device/impl/device_gemm_xdl_cshuffle_v3.hpp

…huffle_v3.hpp Co-authored-by: Bartłomiej Kocot <[email protected]>

…e_v3.hpp Co-authored-by: Bartłomiej Kocot <[email protected]>

…huffle_v3.hpp Co-authored-by: Bartłomiej Kocot <[email protected]>

…e_v3.hpp Co-authored-by: Bartłomiej Kocot <[email protected]>

…huffle_v3.hpp Co-authored-by: Bartłomiej Kocot <[email protected]>

…e_v3.hpp Co-authored-by: Bartłomiej Kocot <[email protected]>

…huffle_v3.hpp Co-authored-by: Bartłomiej Kocot <[email protected]>

…huffle_v3.hpp Co-authored-by: spolifroni-amd <[email protected]>

…e_v3.hpp Co-authored-by: Bartłomiej Kocot <[email protected]>

…huffle_v3.hpp Co-authored-by: Bartłomiej Kocot <[email protected]>

…huffle_v3.hpp Co-authored-by: spolifroni-amd <[email protected]>

…huffle_v3.hpp Co-authored-by: Bartłomiej Kocot <[email protected]>

…huffle_v3.hpp Co-authored-by: spolifroni-amd <[email protected]>

aosewski · 2025-03-27T09:28:54Z

@spolifroni-amd Could you please re-review?

spolifroni-amd

Suggested a couple of rewrites.

include/ck/tensor_operation/gpu/device/impl/device_gemm_xdl_cshuffle_v3.hpp

spolifroni-amd · 2025-03-27T14:07:49Z

include/ck/tensor_operation/gpu/device/impl/device_gemm_xdl_cshuffle_v3.hpp

+///         pipelines optimized for memory-bound, latency and compute-bound cases.
+///
+/// @note   This Kernel implementation supports SplitK algorithm - that is it may be configured
+///         to split the work of K dimension dot product accumulation into multiple work groups.


I didn't correct "work group/workgroup". I assumed since it was corrected by another reviewer on the CK team that the term you used is "workgroup". It can be spelled either way as long as its consistent throughout. So you either use "work group" everywhere or "workgroup" everywhere.

In terms of the meaning, so this can be rewritten then as

"Split the dot product accumulated over the k dimension into multiple working groups"

(If "work group/workgroup" isn't a specialized term, using working groups can eliminate having to make a decision about whether work groups is one or two words)

AviralGoelAMD · 2025-03-27T22:24:35Z

Looks good!

…_doc

AviralGoelAMD · 2025-03-31T01:47:51Z

(If "work group/workgroup" isn't a specialized term, using working groups can eliminate having to make a decision about whether work groups is one or two words) @spolifroni-amd

Just to chime in, the formal term is "workgroup" across academia and industry.
https://rocm.docs.amd.com/projects/rocprofiler-compute/en/latest/conceptual/definitions.html#desc-workgroup

spolifroni-amd · 2025-03-31T14:00:47Z

(If "work group/workgroup" isn't a specialized term, using working groups can eliminate having to make a decision about whether work groups is one or two words) @spolifroni-amd

Just to chime in, the formal term is "workgroup" across academia and industry. https://rocm.docs.amd.com/projects/rocprofiler-compute/en/latest/conceptual/definitions.html#desc-workgroup

I've seen both but I'll add this to the ROCm style guide. Thanks!

* Basic docs for universal gemm & ck-tile gemm. * Update include/ck/tensor_operation/gpu/device/impl/device_gemm_xdl_cshuffle_v3.hpp Co-authored-by: Bartłomiej Kocot <[email protected]> * Update include/ck/tensor_operation/gpu/grid/gridwise_gemm_xdl_cshuffle_v3.hpp Co-authored-by: Bartłomiej Kocot <[email protected]> * Update include/ck/tensor_operation/gpu/device/impl/device_gemm_xdl_cshuffle_v3.hpp Co-authored-by: Bartłomiej Kocot <[email protected]> * Update include/ck/tensor_operation/gpu/grid/gridwise_gemm_xdl_cshuffle_v3.hpp Co-authored-by: Bartłomiej Kocot <[email protected]> * Update include/ck/tensor_operation/gpu/device/impl/device_gemm_xdl_cshuffle_v3.hpp Co-authored-by: Bartłomiej Kocot <[email protected]> * Update include/ck/tensor_operation/gpu/grid/gridwise_gemm_xdl_cshuffle_v3.hpp Co-authored-by: Bartłomiej Kocot <[email protected]> * Update include/ck/tensor_operation/gpu/device/impl/device_gemm_xdl_cshuffle_v3.hpp Co-authored-by: Bartłomiej Kocot <[email protected]> * Update include/ck/tensor_operation/gpu/device/impl/device_gemm_xdl_cshuffle_v3.hpp Co-authored-by: spolifroni-amd <[email protected]> * Update include/ck/tensor_operation/gpu/grid/gridwise_gemm_xdl_cshuffle_v3.hpp Co-authored-by: Bartłomiej Kocot <[email protected]> * Update include/ck/tensor_operation/gpu/grid/gridwise_gemm_xdl_cshuffle_v3.hpp Co-authored-by: Bartłomiej Kocot <[email protected]> * Update include/ck/tensor_operation/gpu/grid/gridwise_gemm_xdl_cshuffle_v3.hpp Co-authored-by: Bartłomiej Kocot <[email protected]> * Update include/ck/tensor_operation/gpu/device/impl/device_gemm_xdl_cshuffle_v3.hpp Co-authored-by: Bartłomiej Kocot <[email protected]> * Update include/ck/tensor_operation/gpu/device/impl/device_gemm_xdl_cshuffle_v3.hpp Co-authored-by: spolifroni-amd <[email protected]> * Update include/ck/tensor_operation/gpu/device/impl/device_gemm_xdl_cshuffle_v3.hpp Co-authored-by: Bartłomiej Kocot <[email protected]> * Update include/ck/tensor_operation/gpu/device/impl/device_gemm_xdl_cshuffle_v3.hpp Co-authored-by: spolifroni-amd <[email protected]> * Update include/ck/tensor_operation/gpu/device/impl/device_gemm_xdl_cshuffle_v3.hpp Co-authored-by: spolifroni-amd <[email protected]> * Update include/ck/tensor_operation/gpu/device/impl/device_gemm_xdl_cshuffle_v3.hpp Co-authored-by: spolifroni-amd <[email protected]> * Update include/ck/tensor_operation/gpu/device/impl/device_gemm_xdl_cshuffle_v3.hpp Co-authored-by: spolifroni-amd <[email protected]> * Update include/ck/tensor_operation/gpu/device/impl/device_gemm_xdl_cshuffle_v3.hpp Co-authored-by: spolifroni-amd <[email protected]> * Update include/ck/tensor_operation/gpu/device/impl/device_gemm_xdl_cshuffle_v3.hpp Co-authored-by: spolifroni-amd <[email protected]> * Update include/ck/tensor_operation/gpu/device/impl/device_gemm_xdl_cshuffle_v3.hpp Co-authored-by: spolifroni-amd <[email protected]> * Reviewers suggestions. * Align tparam names in doc with class tparams. * More reviewers fine tuning ;) --------- Co-authored-by: Bartłomiej Kocot <[email protected]> Co-authored-by: spolifroni-amd <[email protected]>

Basic docs for universal gemm & ck-tile gemm.

8b37f94

aosewski requested review from illsilin, carlushuang, qianfengz, poyenc, geyyer, bartekxk, andriy-ca, afagaj, asleepzzz and tenpercent as code owners March 25, 2025 11:36

aosewski self-assigned this Mar 25, 2025

aosewski requested review from spolifroni-amd and aska-0096 March 25, 2025 11:38

bartekxk reviewed Mar 25, 2025

View reviewed changes

spolifroni-amd requested changes Mar 25, 2025

View reviewed changes

aosewski and others added 12 commits March 26, 2025 09:41

Update include/ck/tensor_operation/gpu/device/impl/device_gemm_xdl_cs…

a12fa13

…huffle_v3.hpp Co-authored-by: Bartłomiej Kocot <[email protected]>

Update include/ck/tensor_operation/gpu/grid/gridwise_gemm_xdl_cshuffl…

b985b09

…e_v3.hpp Co-authored-by: Bartłomiej Kocot <[email protected]>

Update include/ck/tensor_operation/gpu/device/impl/device_gemm_xdl_cs…

9cc3431

…huffle_v3.hpp Co-authored-by: Bartłomiej Kocot <[email protected]>

Update include/ck/tensor_operation/gpu/grid/gridwise_gemm_xdl_cshuffl…

b945cd9

…e_v3.hpp Co-authored-by: Bartłomiej Kocot <[email protected]>

Update include/ck/tensor_operation/gpu/device/impl/device_gemm_xdl_cs…

010aad1

…huffle_v3.hpp Co-authored-by: Bartłomiej Kocot <[email protected]>

Update include/ck/tensor_operation/gpu/grid/gridwise_gemm_xdl_cshuffl…

3d6aea2

…e_v3.hpp Co-authored-by: Bartłomiej Kocot <[email protected]>

Update include/ck/tensor_operation/gpu/device/impl/device_gemm_xdl_cs…

5efbe86

…huffle_v3.hpp Co-authored-by: Bartłomiej Kocot <[email protected]>

Update include/ck/tensor_operation/gpu/device/impl/device_gemm_xdl_cs…

ad1b09f

…huffle_v3.hpp Co-authored-by: spolifroni-amd <[email protected]>

Update include/ck/tensor_operation/gpu/grid/gridwise_gemm_xdl_cshuffl…

c671eae

…e_v3.hpp Co-authored-by: Bartłomiej Kocot <[email protected]>

Update include/ck/tensor_operation/gpu/grid/gridwise_gemm_xdl_cshuffl…

2534f77

…e_v3.hpp Co-authored-by: Bartłomiej Kocot <[email protected]>

Update include/ck/tensor_operation/gpu/grid/gridwise_gemm_xdl_cshuffl…

e8a237c

…e_v3.hpp Co-authored-by: Bartłomiej Kocot <[email protected]>

Update include/ck/tensor_operation/gpu/device/impl/device_gemm_xdl_cs…

5b4d443

…huffle_v3.hpp Co-authored-by: Bartłomiej Kocot <[email protected]>

aosewski and others added 10 commits March 26, 2025 13:06

Update include/ck/tensor_operation/gpu/device/impl/device_gemm_xdl_cs…

5832414

…huffle_v3.hpp Co-authored-by: spolifroni-amd <[email protected]>

Update include/ck/tensor_operation/gpu/device/impl/device_gemm_xdl_cs…

da89b40

…huffle_v3.hpp Co-authored-by: Bartłomiej Kocot <[email protected]>

Update include/ck/tensor_operation/gpu/device/impl/device_gemm_xdl_cs…

bb95119

…huffle_v3.hpp Co-authored-by: spolifroni-amd <[email protected]>

Update include/ck/tensor_operation/gpu/device/impl/device_gemm_xdl_cs…

782cba4

…huffle_v3.hpp Co-authored-by: spolifroni-amd <[email protected]>

Update include/ck/tensor_operation/gpu/device/impl/device_gemm_xdl_cs…

b57ebf6

…huffle_v3.hpp Co-authored-by: spolifroni-amd <[email protected]>

Update include/ck/tensor_operation/gpu/device/impl/device_gemm_xdl_cs…

5eef1e9

…huffle_v3.hpp Co-authored-by: spolifroni-amd <[email protected]>

Update include/ck/tensor_operation/gpu/device/impl/device_gemm_xdl_cs…

1b66dc3

…huffle_v3.hpp Co-authored-by: spolifroni-amd <[email protected]>

Update include/ck/tensor_operation/gpu/device/impl/device_gemm_xdl_cs…

eca5168

…huffle_v3.hpp Co-authored-by: spolifroni-amd <[email protected]>

Update include/ck/tensor_operation/gpu/device/impl/device_gemm_xdl_cs…

faff800

…huffle_v3.hpp Co-authored-by: spolifroni-amd <[email protected]>

Reviewers suggestions.

66821a2

aosewski requested review from spolifroni-amd and bartekxk March 26, 2025 13:25

Align tparam names in doc with class tparams.

cbff8c2

spolifroni-amd requested changes Mar 27, 2025

View reviewed changes

AviralGoelAMD added the documentation Improvements or additions to documentation label Mar 27, 2025

aosewski added 2 commits March 28, 2025 09:53

More reviewers fine tuning ;)

dc01b65

Merge remote-tracking branch 'origin/develop' into aosewski/ck_tparam…

50828b1

…_doc

aosewski requested a review from spolifroni-amd March 28, 2025 09:55

spolifroni-amd approved these changes Mar 28, 2025

View reviewed changes

Merge branch 'develop' into aosewski/ck_tparam_doc

510870e

bartekxk approved these changes Apr 1, 2025

View reviewed changes

aosewski merged commit e5ad48a into develop Apr 2, 2025
34 of 39 checks passed

aosewski deleted the aosewski/ck_tparam_doc branch April 2, 2025 09:03

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Basic docs for universal gemm & ck-tile gemm. #2014

Basic docs for universal gemm & ck-tile gemm. #2014

aosewski commented Mar 25, 2025 •

edited

Loading

aosewski commented Mar 25, 2025

aosewski commented Mar 25, 2025

spolifroni-amd left a comment

aosewski commented Mar 27, 2025

spolifroni-amd left a comment

spolifroni-amd Mar 27, 2025

AviralGoelAMD commented Mar 27, 2025

AviralGoelAMD commented Mar 31, 2025

spolifroni-amd commented Mar 31, 2025

Basic docs for universal gemm & ck-tile gemm. #2014

Basic docs for universal gemm & ck-tile gemm. #2014

Conversation

aosewski commented Mar 25, 2025 • edited Loading

Proposed changes

Checklist

Discussion

aosewski commented Mar 25, 2025

aosewski commented Mar 25, 2025

spolifroni-amd left a comment

Choose a reason for hiding this comment

aosewski commented Mar 27, 2025

spolifroni-amd left a comment

Choose a reason for hiding this comment

spolifroni-amd Mar 27, 2025

Choose a reason for hiding this comment

AviralGoelAMD commented Mar 27, 2025

AviralGoelAMD commented Mar 31, 2025

spolifroni-amd commented Mar 31, 2025

aosewski commented Mar 25, 2025 •

edited

Loading