[SS-JIA]

Stack from ghstack (oldest at bottom):

• [ET-VK][conv2d] Use row-aligned work groups for non-reducing pointwise convs on Mali #24859
• [ET-VK][conv2d] Fold the accumulation of Half pointwise convs on Adreno #24858
• [ET-VK][conv2d] Skip the store rounding of texels computed in f16 #24857
• -> [ET-VK][conv2d] Use * and + instead of fma() on f16 vectors #24856
• [ET-VK][tools] Report RMS error and dispatches in etvk_ops_bench validation #24855

Ulterior Motive

Optimizing the FP16 1x1 convolutions of the DSR model on mobile GPUs.

Rationale

What: A mul_add(a, b, c) helper in common.glslh computes a * b + c. In shaders that define HAS_FLOAT16_ACCUMULATION, it uses * and + for f16vec4 and fma() for vec4; in other shaders it is fma(). The GEMM tile helper and conv2d_tiled call it instead of fma(), and the conv2d shaders that accumulate in f16 vectors define the marker: conv2d_pw_tiled, conv2d_tiled and the buffer input conv2d_gemm.

Why: Adreno 740 and 750 run fma() on f16 vectors at about half the speed of a multiply and an add. With this commit, Half pointwise convolutions take 0.37 to 0.49 of the time on a Galaxy S23 (Adreno 740) and a Galaxy S24 (Adreno 750). Replacing fma() in every shader regardless of type made Float shaders up to 1.35 times as slow on the same phones, so the form depends on the type.

Details

• Before: Multiply-accumulates call fma() for both dtypes.
• After: In the Half variants of conv2d_pw_tiled, conv2d_tiled and the buffer input conv2d_gemm, multiply-accumulates on f16 vectors compute c + a * b. Every other shader still calls fma(), including the other users of the GEMM tile helper (linear, matmul, Q4 and Q8 linear, fused SDPA).

f16vec4 is a type only when GL_EXT_shader_explicit_arithmetic_types_float16 is enabled, so a shader opts in by defining HAS_FLOAT16_ACCUMULATION before it includes common.glslh; without it, mul_add is a macro for fma(). Two alternatives do not work:

• Enabling the extension in common.glslh for every shader changes implicit conversions in shaders that do not use f16 arithmetic (max_pool2d's mix() call becomes ambiguous).
• The extension's own macro cannot be the guard: glslang defines GL_EXT_shader_explicit_arithmetic_types_float16 whether or not the extension is enabled.

Other shaders that accumulate in f16 vectors can opt in once their accuracy and performance are measured.

c + a * b rounds the product before the add, which fma() does not. On Adreno 740 and 750, the f16 error statistics of some conv2d correctness configurations change; all still pass. Outputs on Adreno 840, Mali-G715 and Xclipse 940 are bit-identical to the parent's.

Only 16 SPIR-V modules change: the Half conv2d_pw_tiled (4), conv2d_tiled (11) and conv2d_gemm_buffer_half modules.

Differential Revision: D130986954

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/24856

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure, 4 Unrelated Failures

As of commit 6ed8fce with merge base 115655c ():

NEW FAILURE - The following job has failed:

• Test Vulkan Backend (specialized runners) / test-model-vulkan-e2e / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• pull / unittest / linux / linux-job (gh) (trunk failure)
  [ FAILED ] OpLeTensorOutTest.BroadcastBoolInputs
• pull / unittest / macos / macos-job (gh) (trunk failure)
  [ FAILED ] OpLeTensorOutTest.BroadcastBoolInputs
• pull / unittest / windows / windows-job (gh) (trunk failure)
  [ FAILED ] OpLeTensorOutTest.BroadcastBoolInputs
• Test Native Engines / native-ggml (gh) (trunk failure)
  backends/native/test/test_hf_whisper.py::WhisperNativeExportTest::test_ggml_runner_matches_eager

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.