Skip to content

Add custom GPU kernels playbook: Vector Addition and Matrix Multiplication kernels - #100

Merged
sdevinenamd merged 63 commits into
mainfrom
custom-gpu-kernels
Apr 7, 2026
Merged

Add custom GPU kernels playbook: Vector Addition and Matrix Multiplication kernels#100
sdevinenamd merged 63 commits into
mainfrom
custom-gpu-kernels

Conversation

@sdevinenamd

Copy link
Copy Markdown
Collaborator

Summary

Adds a new supplemental playbook that teaches users how to write, compile, and launch custom GPU kernels on AMD hardware using PyTorch's ROCm build. The playbook covers two end-to-end flows, a high-level JIT path and a low-level native C++ extension path, applied across two examples: vector addition and matrix multiplication.

File Structure

  • playbook.json — Playbook metadata
  • README.md — Full playbook content covering GPU kernel fundamentals, the HIP/ROCm stack, PyTorch's CUDA→HIP remapping, and two worked examples with both compilation flows
  • assets/Vector Addition/add_one_kernel.py — Flow 1: JIT-compiled kernel via torch.cuda._compile_kernel, with background GPU utilization monitoring via rocm-smi
  • assets/Vector Addition/add_one_kernel.cu — Flow 2: Native HIP C++ extension (kernel + pybind11 binding)
  • assets/Vector Addition/setup.py — Build script using CUDAExtension (auto-remapped to hipcc by PyTorch ROCm)
  • assets/Matrix Multiplication/matmul_kernel.py — Flow 1: 2D-grid JIT matmul kernel with timing and torch.mm verification
  • assets/Matrix Multiplication/matmul_kernel.cu — Flow 2: Native HIP C++ extension
  • assets/Matrix Multiplication/setup.py — Build script

Playbook Structure

  • Overview — Two compilation flows side by side (JIT vs. native extension).
  • What is a GPU Kernel?
  • PyTorch + AMD/HIP
  • Installing Dependencies — Full ROCm 7.1.1 install sequence (uninstall old stack, amdgpu-install, PyTorch, and ROCm wheel)
  • Example 1: Vector Addition — Flow 1 (JIT, GPU utilization monitoring) and Flow 2 (C++ extension, pybind11 binding, .so output)
  • Example 2: Matrix Multiplication — Row-major memory layout math, 2D thread indexing model, Flow 1 (JIT + torch.mm verification) and Flow 2 (native extension returning output tensor)
  • Next Steps

@adamlam2-amd adamlam2-amd left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pretty good! just a couple of minor things.

Address these, and then should be okay to merge for QA to check.

Comment thread playbooks/supplemental/pytorch-kernels/README.md
Comment thread playbooks/supplemental/pytorch-kernels/README.md Outdated
Comment thread playbooks/supplemental/pytorch-kernels/README.md Outdated
Comment thread playbooks/supplemental/pytorch-kernels/README.md Outdated
Comment thread playbooks/supplemental/pytorch-kernels/README.md
Comment thread playbooks/supplemental/pytorch-kernels/playbook.json Outdated
Comment thread playbooks/supplemental/pytorch-kernels/README.md

@danielholanda danielholanda left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Please take a look at the recommended suggestions before merging

@jammm jammm left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good overall with some minor nits here and there. This should be running on Windows too. If you face any issues, report them in this PR so we can help you fix it.

Also, I suggest renaming the folders to have underscore instead of space, e.g., `Matrix_Multiplication" instead of "Matrix Multiplication".

Comment thread playbooks/supplemental/pytorch-kernels/platform.md
Comment thread playbooks/supplemental/pytorch-kernels/platform.md Outdated
@sdevinenamd

Copy link
Copy Markdown
Collaborator Author

Also, I suggest renaming the folders to have underscore instead of space, e.g., `Matrix_Multiplication" instead of "Matrix Multiplication".

Addressed

Comment thread website/package.json
Comment thread website/package-lock.json

@jammm jammm left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM with a couple minor nitpicks. Good to merge overall!

Also take a look at my comment in https://github.com/amd/playbooks/pull/100/changes#r3043808230

Comment thread playbooks/supplemental/pytorch-kernels/README.md
Comment thread playbooks/supplemental/pytorch-kernels/README.md
@sdevinenamd
sdevinenamd merged commit dfc8a7d into main Apr 7, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants