huggingface/text-generation-inference
5
0
Fork 0
mirror of https://github.com/huggingface/text-generation-inference.git synced 2025-09-11 12:24:53 +00:00
Code Issues Packages Projects Releases Wiki Activity

doc: Formatting

Browse Source
This commit is contained in:
Hugo Larcher 2024-12-04 10:49:41 +01:00
parent 491b5726a6
commit 7788a6b849
No known key found for this signature in database
GPG Key ID: 3DAF63124699CA2B
1 changed files with 2 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
docs/source/multi_backend_support.md
View File

@ -6,8 +6,8 @@ whether you prioritize performance, ease of use, or compatibility with specific
TGI remains consistent across backends, allowing you to switch between them seamlessly. TGI remains consistent across backends, allowing you to switch between them seamlessly.
**Supported backends:** **Supported backends:**
* TGI CUDA backend: This high-performance backend is optimized for NVIDIA GPUs and serves as the default option * **TGI CUDA backend**: This high-performance backend is optimized for NVIDIA GPUs and serves as the default option
within TGI. Developed in-house, it boasts numerous optimizations and is used in production by various projects, including those by Hugging Face. within TGI. Developed in-house, it boasts numerous optimizations and is used in production by various projects, including those by Hugging Face.
* [TGI TRTLLM backend](./backends/trtllm): This backend leverages NVIDIA's TensorRT library to accelerate LLM inference. * **[TGI TRTLLM backend](./backends/trtllm)**: This backend leverages NVIDIA's TensorRT library to accelerate LLM inference.
It utilizes specialized optimizations and custom kernels for enhanced performance. It utilizes specialized optimizations and custom kernels for enhanced performance.
However, it requires a model-specific compilation step for each GPU architecture. However, it requires a model-specific compilation step for each GPU architecture.