[SS-JIA]

Stack from ghstack (oldest at bottom):

• -> [ET-VK][conv2d] Use row-aligned work groups for non-reducing pointwise convs on Mali #24859
• [ET-VK][conv2d] Fold the accumulation of Half pointwise convs on Adreno #24858
• [ET-VK][conv2d] Skip the store rounding of texels computed in f16 #24857
• [ET-VK][conv2d] Use * and + instead of fma() on f16 vectors #24856
• [ET-VK][tools] Report RMS error and dispatches in etvk_ops_bench validation #24855

Ulterior Motive

Optimizing the FP16 1x1 convolutions of the DSR model on Mali (Pixel 9 Pro XL).

Rationale

What: On Mali, Half wide tile conv2d_pw_tiled convolutions with at least as many 4-channel output slices as input slices, 8, 16, 32 or 64 channel tiles and at least 8192 pixel tiles use work groups of up to 896 threads: y spans the channel tiles and x divides the row of pixel tiles. Other convolutions keep the 512 thread picks. On every GPU, wide tile work groups also keep x large enough for the work group count on x to stay within the device limit.

Why: The six DSR 1x1 layers that qualify take 0.771 to 1.005 of the time with these work groups on a Pixel 9 Pro XL (Mali-G715), and the nine 1x1 layers of the model take 0.928. Applied to every convolution with at least as many output as input slices, the same work groups took up to 1.11 of the time on other channel tile counts and 0.92 to 1.06 on outputs below 8192 pixel tiles, so those shapes keep their current work groups. The rule stays on Mali: on an earlier version of this stack, Adreno 740, 750 and 840 took 1.030, 1.038 and 1.045 of the time over the nine layers with it (G7 up to 1.33), and Xclipse 940 up to 1.99 per layer.

Details

• Before: Mali Half wide tile dispatches use 512 thread work groups: y is the channel tile count rounded up to a power of two (at most 32), and x takes the rest.
• After: with N4 = ceil(C_out / 4), K4 = ceil(C_in / 4) and P = ceil(N4 / 2) (the channel tiles), a convolution with N4 >= K4, P equal to 8, 16, 32 or 64 and at least 8192 pixel tiles (ceil(W / 2) * ceil(H / 2)) uses the local size {x, P, 1}, where x is the largest divisor of ceil(W / 2) with x * P <= 896, or floor(896 / P) if that divisor is below half of it. The 896 threads are halved while they exceed the device's work group invocation or x size limit, as the 512 thread pick already is. No shader changes; Adreno, Xclipse, Float and the narrow tile are unchanged.
• Work group count on x
  • Before: Wide tile dispatches put every pixel tile on x, and nothing bounds the resulting work group count. With 64 thread work groups (GPUs other than Mali and Adreno, x = 4 above 8 channel tiles), a 1024x1024 output already needs 65536 work groups on x, one more than the usual maxComputeWorkGroupCount[0], and the dispatch throws.
  • After: x is at least ceil(pixel tiles / maxComputeWorkGroupCount[0]). The row-aligned pick falls back to floor(896 / P) when its divisor is smaller, and the other picks raise min_x to the smallest power of two at or above it, up to the thread count. Below the limit, every pick is unchanged.

Differential Revision: D130986963

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/24859

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

⏳ 1 Pending, 4 Unrelated Failures

As of commit 25901e1 with merge base 115655c ():

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• pull / unittest / linux / linux-job (gh) (trunk failure)
  [ FAILED ] OpLeTensorOutTest.BroadcastBoolInputs
• pull / unittest / macos / macos-job (gh) (trunk failure)
  [ FAILED ] OpLeTensorOutTest.BroadcastBoolInputs
• pull / unittest / windows / windows-job (gh) (trunk failure)
  [ FAILED ] OpLeTensorOutTest.BroadcastBoolInputs
• Test Native Engines / native-ggml (gh) (trunk failure)
  backends/native/test/test_hf_whisper.py::WhisperNativeExportTest::test_ggml_runner_matches_eager

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.