huggingface/text-generation-inference
5
0
Fork 0
mirror of https://github.com/huggingface/text-generation-inference.git synced 2025-09-11 04:14:52 +00:00
Code Issues Packages Projects Releases Wiki Activity

Update docs

Browse Source 
Signed-off-by: Adrien Gallouët <angt@huggingface.co>
This commit is contained in:
Adrien Gallouët 2025-02-06 10:31:05 +00:00
parent 2b0d99c1cf
commit 8bc10d37ee
No known key found for this signature in database
2 changed files with 4 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
docs/source/_toctree.yml
View File

@ -52,6 +52,8 @@
- sections:
- local: backends/trtllm
title: TensorRT-LLM
- local: backends/llamacpp
title: Llamacpp
title: Backends
- sections:
- local: reference/launcher

2
docs/source/multi_backend_support.md
View File

@ -11,3 +11,5 @@ TGI remains consistent across backends, allowing you to switch between them seam
* **[TGI TRTLLM backend](./backends/trtllm)**: This backend leverages NVIDIA's TensorRT library to accelerate LLM inference.
It utilizes specialized optimizations and custom kernels for enhanced performance.
However, it requires a model-specific compilation step for each GPU architecture.
* **[TGI Llamacpp backend](./backends/llamacpp)**: This backend facilitates the deployment of large language models
(LLMs) by integrating [llama.cpp][llama.cpp], an advanced inference engine optimized for both CPU and GPU computation.