Skip to content

Commit a2dd6ee

Browse files
authored
[Docs] Add neural speed backend (#226)
* docs: add neural speed backend Signed-off-by: grorge <[email protected]> * fix: backend name Signed-off-by: grorge <[email protected]> --------- Signed-off-by: grorge <[email protected]>
1 parent 2f6bf43 commit a2dd6ee

File tree

1 file changed

+44
-0
lines changed

1 file changed

+44
-0
lines changed

docs/contribute/source/plugin/wasi_nn.md

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -277,6 +277,50 @@ cmake --build build
277277
cmake --install build
278278
```
279279

280+
#### Apple Silicon Model
281+
282+
You can build and install WasmEdge from source directly on the macOS arm64 platform. It will use the built-in GPU by default.
283+
284+
```bash
285+
cd <path/to/your/wasmedge/source/folder>
286+
# Enable METAL on arm64 macOS.
287+
cmake -GNinja -Bbuild -DCMAKE_BUILD_TYPE=Release \
288+
-DWASMEDGE_PLUGIN_WASI_NN_BACKEND="GGML" \
289+
-DWASMEDGE_PLUGIN_WASI_NN_GGML_LLAMA_METAL=ON \
290+
-DWASMEDGE_PLUGIN_WASI_NN_GGML_LLAMA_BLAS=OFF \
291+
.
292+
cmake --build build
293+
# For the WASI-NN plugin, you should install this project.
294+
cmake --install build
295+
```
296+
297+
## Build WasmEdge with WASI-NN Neural Speed Backend
298+
299+
The Neural Speed backend relies on Neural Speed, we recommend the following commands to install Neural Speed.
300+
301+
```bash
302+
sudo apt update
303+
sudo apt upgrade
304+
sudo apt install python3-dev
305+
wget https://raw.githubusercontent.com/intel/neural-speed/main/requirements.txt
306+
pip install -r requirements.txt
307+
pip install neural-speed
308+
```
309+
310+
Then build and install WasmEdge from source:
311+
312+
```bash
313+
cd <path/to/your/wasmedge/source/folder>
314+
315+
cmake -GNinja -Bbuild -DCMAKE_BUILD_TYPE=Release -DWASMEDGE_PLUGIN_WASI_NN_BACKEND="neuralspeed"
316+
cmake --build build
317+
318+
# For the WASI-NN plugin, you should install this project.
319+
cmake --install build
320+
```
321+
322+
Then you will have an executable `wasmedge` runtime under `/usr/local/bin` and the WASI-NN with Neural Speed backend plug-in under `/usr/local/lib/wasmedge/libwasmedgePluginWasiNN.so` after installation.
323+
280324
### Appendix
281325

282326
<!-- prettier-ignore -->

0 commit comments

Comments
 (0)