Add README.md

Signed-off-by: Adrien Gallouët <angt@huggingface.co>
This commit is contained in:
Adrien Gallouët 2025-02-07 12:12:13 +00:00
parent 0702e0bfda
commit 508d47f80f
No known key found for this signature in database

View File

@ -0,0 +1,24 @@
# Llamacpp backend
If all your dependencies are installed at the system level, running
cargo build should be sufficient. However, if you want to experiment
with different versions of llama.cpp, some additional setup is required.
## Install llama.cpp
LLAMACPP_PREFIX=$(pwd)/llama.cpp.out
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp
cmake -B build \
-DCMAKE_INSTALL_PREFIX="$LLAMACPP_PREFIX" \
-DLLAMA_BUILD_COMMON=OFF \
-DLLAMA_BUILD_TESTS=OFF \
-DLLAMA_BUILD_EXAMPLES=OFF \
-DLLAMA_BUILD_SERVER=OFF
cmake --build build --config Release -j
cmake --install build
## Build TGI
PKG_CONFIG_PATH="$LLAMACPP_PREFIX/lib/pkgconfig" cargo build