[SS-JIA]

Stack from ghstack (oldest at bottom):

• -> [executorch][native-ggml] Enable CUDA graphs in CMake builds #24849
• [executorch][native] Load Hugging Face tokenizer.json in the LLM runner #24848
• [executorch][native-ggml] Size the scheduler hash for graph leafs too #24847
• [executorch][native-ggml] Leave zero-frequency RoPE dims unrotated #24846
• [executorch][native-ggml] Accept GGUF q5_k weights #24845

Ulterior Motive

Closing the decode gap between ExecuTorch Native's GGML engine and llama.cpp on CUDA, measured with Gemma 4 31B Q4_K_M on an A100.

Rationale

What: The GGML engine's CMake turns ggml's GGML_CUDA_GRAPHS option on unless the user sets it.

Why: ggml defaults the option to off and llama.cpp's own CMake turns it on. The engine adds ggml as a subdirectory, so its CMake CUDA builds launched each kernel of a decode step separately: about 1,900 launches per Gemma 4 31B step, which cost 2.2–2.5 ms of a ~26 ms step.

Details

• Before: GGML_CUDA_GRAPHS off unless set; ggml-cuda never captured or replayed CUDA graphs.
• After: on unless set, as in llama.cpp's CMake; -DGGML_CUDA_GRAPHS=OFF still turns it off.

Buck builds already compile ggml-cuda with CUDA graphs.

Authored with Claude Code.

Differential Revision: D130721993

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/24849

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ You can merge normally! (4 Unrelated Failures), 4 Unclassified Failures

As of commit 8ecfa50 with merge base 4529a94 ():

UNCLASSIFIED FAILURES - DrCI could not classify the following jobs because the workflow did not run on the merge base. The failures may be pre-existing on trunk or introduced by this PR:

• Build Linux CUDA Wheels / pytorch/executorch / build-manywheel-py3_11-cuda13_2 (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
• Build Linux CUDA Wheels / pytorch/executorch / upload / upload-manywheel-py3_11-cuda13_2 (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
  Unable to download artifact(s): Artifact not found
• Build Linux Wheels / pytorch/executorch / build-manywheel-py3_11-cpu (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
• Build Linux Wheels / pytorch/executorch / upload / upload-manywheel-py3_11-cpu (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
  Unable to download artifact(s): Artifact not found

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• Test QNN Backend / test-qnn / test-backend-linux (qnn, models) / linux-job (gh) (trunk failure)
  curl: (22) The requested URL returned error:
• Test QNN Backend / test-qnn / test-backend-linux (qnn, operators) / linux-job (gh) (trunk failure)
  curl: (22) The requested URL returned error:
• Test QNN Windows MSVC build / build-qnn-windows-arm64 (gh) (trunk failure)
• Test QNN Windows MSVC build / build-qnn-windows-x64 / windows-job (gh) (trunk failure)

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.