mirror of
https://github.com/huggingface/text-generation-inference.git
synced 2025-09-11 04:14:52 +00:00
Add README.md
Signed-off-by: Adrien Gallouët <angt@huggingface.co>
This commit is contained in:
parent
0702e0bfda
commit
508d47f80f
24
backends/llamacpp/README.md
Normal file
24
backends/llamacpp/README.md
Normal file
@ -0,0 +1,24 @@
|
||||
# Llamacpp backend
|
||||
|
||||
If all your dependencies are installed at the system level, running
|
||||
cargo build should be sufficient. However, if you want to experiment
|
||||
with different versions of llama.cpp, some additional setup is required.
|
||||
|
||||
## Install llama.cpp
|
||||
|
||||
LLAMACPP_PREFIX=$(pwd)/llama.cpp.out
|
||||
|
||||
git clone https://github.com/ggerganov/llama.cpp
|
||||
cd llama.cpp
|
||||
cmake -B build \
|
||||
-DCMAKE_INSTALL_PREFIX="$LLAMACPP_PREFIX" \
|
||||
-DLLAMA_BUILD_COMMON=OFF \
|
||||
-DLLAMA_BUILD_TESTS=OFF \
|
||||
-DLLAMA_BUILD_EXAMPLES=OFF \
|
||||
-DLLAMA_BUILD_SERVER=OFF
|
||||
cmake --build build --config Release -j
|
||||
cmake --install build
|
||||
|
||||
## Build TGI
|
||||
|
||||
PKG_CONFIG_PATH="$LLAMACPP_PREFIX/lib/pkgconfig" cargo build
|
Loading…
Reference in New Issue
Block a user