From 1e4a8382db5710d9c528575d6918eeaeedd1fb19 Mon Sep 17 00:00:00 2001 From: Alvaro Bartolome <36760800+alvarobartt@users.noreply.github.com> Date: Fri, 12 Sep 2025 11:26:57 +0200 Subject: [PATCH] Fix `cargo install` commands for `candle` with CUDA The `cargo install --path router ...` commands for the features `candle-cuda` and `candle-cuda-turing` were using the flag `--no-default-features` that was ignoring the default features in the `text-embeddings-router` being `candle,http,dynamic-linking`, hence the command was failing as `cudarc` requires to have any linking out of `static-linking`, `dynamic-linking` or `dynamic-loading` for the CUDA Libraries, hence when ignoring the `default` features, and just adding `-F http` either the e.g. `-F dynamic-loading` feature was missing or just to only keep the e.g. `-F candle-cuda`, as `dynamic-linking` and `http` are default features already. --- README.md | 6 +++--- docs/source/en/local_gpu.md | 6 +++--- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index f2508852..9231a32e 100644 --- a/README.md +++ b/README.md @@ -530,13 +530,13 @@ export PATH=$PATH:/usr/local/cuda/bin Then run: ```shell -# This can take a while as we need to compile a lot of cuda kernels +# This can take a while as we need to compile a lot of CUDA kernels # On Turing GPUs (T4, RTX 2000 series ... ) -cargo install --path router -F candle-cuda-turing -F http --no-default-features +cargo install --path router -F candle-cuda-turing # On Ampere and Hopper -cargo install --path router -F candle-cuda -F http --no-default-features +cargo install --path router -F candle-cuda ``` You can now launch Text Embeddings Inference on GPU with: diff --git a/docs/source/en/local_gpu.md b/docs/source/en/local_gpu.md index 7af94df8..2dbe86f4 100644 --- a/docs/source/en/local_gpu.md +++ b/docs/source/en/local_gpu.md @@ -39,18 +39,18 @@ curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh ## Step 3: Install necessary packages -This step can take a while as we need to compile a lot of cuda kernels. +This step can take a while as we need to compile a lot of CUDA kernels. ### For Turing GPUs (T4, RTX 2000 series ... ) ```shell -cargo install --path router -F candle-cuda-turing -F http --no-default-features +cargo install --path router -F candle-cuda-turing ``` ### For Ampere and Hopper ```shell -cargo install --path router -F candle-cuda -F http --no-default-features +cargo install --path router -F candle-cuda ``` ## Step 4: Launch Text Embeddings Inference