ci(release): full self-contained Windows Vulkan/HIP bundles + README fork note#78
Merged
Conversation
readme: add Prism fork note (start with Bonsai-demo, main caveats)
github-actions
Bot
added
documentation
There was a problem hiding this comment.
Pull request overview
Makes Windows Vulkan/HIP releases self-contained and adds fork compatibility guidance.
Changes:
- Builds and packages complete Windows Vulkan/HIP binaries.
- Disables OpenMP to avoid runtime DLL dependencies.
- Adds PrismML model and release guidance.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.
| File | Description |
|---|---|
README.md |
Adds fork and model compatibility guidance. |
.github/workflows/release-prism.yml |
Produces complete Windows GPU bundles. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| > | ||
| > If you use this fork directly, the main caveats: | ||
| > - **Use a complete matching build** — download the full bundle for your OS+backend from our [Releases](https://github.com/PrismML-Eng/llama.cpp/releases), or build from source. Do **not** drop this fork's `ggml-*` libraries into a stock llama.cpp build (ABI mismatch — this will fail to load models). | ||
| > - **Models:** use the `*-Q2_0_g64.gguf` files (official **group-64** `Q2_0`). The legacy `*-Q2_0.gguf` (group 128) won't load on group-64 builds; `*-PQ2_0.gguf` is this fork's group-128 variant. |
| > [!IMPORTANT] | ||
| > **This is the PrismML fork of llama.cpp.** It adds the `Q2_0` / `PQ2_0` 2-bit quantization used by the [Bonsai](https://huggingface.co/collections/prism-ml/bonsai) models. | ||
| > | ||
| > **New here? Start with the [Bonsai-demo](https://github.com/PrismML-Eng/Bonsai-demo) repo** — it downloads the right model files and the correct prebuilt binaries for your hardware/backend automatically. |
| > **New here? Start with the [Bonsai-demo](https://github.com/PrismML-Eng/Bonsai-demo) repo** — it downloads the right model files and the correct prebuilt binaries for your hardware/backend automatically. | ||
| > | ||
| > If you use this fork directly, the main caveats: | ||
| > - **Use a complete matching build** — download the full bundle for your OS+backend from our [Releases](https://github.com/PrismML-Eng/llama.cpp/releases), or build from source. Do **not** drop this fork's `ggml-*` libraries into a stock llama.cpp build (ABI mismatch — this will fail to load models). |
| > If you use this fork directly, the main caveats: | ||
| > - **Use a complete matching build** — download the full bundle for your OS+backend from our [Releases](https://github.com/PrismML-Eng/llama.cpp/releases), or build from source. Do **not** drop this fork's `ggml-*` libraries into a stock llama.cpp build (ABI mismatch — this will fail to load models). | ||
| > - **Models:** use the `*-Q2_0_g64.gguf` files (official **group-64** `Q2_0`). The legacy `*-Q2_0.gguf` (group 128) won't load on group-64 builds; `*-PQ2_0.gguf` is this fork's group-128 variant. | ||
| > - `Q2_0` is already upstream in mainline llama.cpp for **CPU and Metal** — those work without this fork. Vulkan/CUDA come from this fork for now. |
…Q2_0 future) + link demo status; ASCII punctuation
| cmake --build build --config Release --target ggml-vulkan | ||
| # Full self-contained bundle (CPU backends + Vulkan), not just the ggml-vulkan.dll | ||
| # add-on, so users can download this one zip and run without merging with the CPU zip. | ||
| cmake -S . -B build -DGGML_VULKAN=ON -DGGML_NATIVE=OFF -DGGML_BACKEND_DL=ON -DGGML_CPU_ALL_VARIANTS=ON -DGGML_OPENMP=OFF -DLLAMA_BUILD_BORINGSSL=ON |
| -DLLAMA_BUILD_BORINGSSL=ON | ||
| cmake --build build --target ggml-hip -j ${env:NUMBER_OF_PROCESSORS} | ||
| # Full self-contained bundle (CPU backend + HIP), not just the ggml-hip.dll add-on. | ||
| cmake --build build -j ${env:NUMBER_OF_PROCESSORS} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Two fixes prompted by #71 (a user couldn't run the Windows Vulkan release — it shipped only
ggml-vulkan.dll, and grafting it onto stock llama.cpp failed with an ABI/offset mismatch):Windows Vulkan & HIP releases now ship complete, self-contained bundles.
Previously these jobs built with
-DGGML_CPU=OFF --target ggml-<backend>and packed only thesingle backend DLL — an add-on that had to be merged into the CPU zip. Now they build the full
set (CPU backends + the GPU backend) and pack the whole
build/bin, so one download runs out ofthe box. This matches what the Linux
ubuntu-vulkanjob already does.windows-vulkan: build withGGML_CPU_ALL_VARIANTS=ON -DGGML_VULKAN=ON(droppedGGML_CPU=OFFand
--target ggml-vulkan), packbuild/bin/Release/*.windows-hip: droppedGGML_CPU=OFF, full build (dropped--target ggml-hip); pack alreadyincluded the whole
build/bin.GGML_OPENMP=OFFon both to avoid an extra OpenMP runtime DLL dependency in the bundle.README fork note. Many users land on this fork directly (not the Bonsai-demo). Added a note
at the top: start with Bonsai-demo; use a complete matching build (don't mix our libs into stock
llama.cpp); use
*-Q2_0_g64.ggufmodels (group-64 official Q2_0) vs legacy*-Q2_0.gguf(g128)/
*-PQ2_0.gguf.Note
Windows CI can't be built locally (no Windows here) — relying on this PR's CI to validate the
Vulkan/HIP full builds. Targeting
prism(production) as requested; migration to newer prism branchlater.
Closes #71 (once released).