huggingface/text-generation-inference
5
0
Fork 0
mirror of https://github.com/huggingface/text-generation-inference.git synced 2025-09-11 12:24:53 +00:00
Code Issues Packages Projects Releases Wiki Activity

doc: Formatting

Browse Source
This commit is contained in:
Hugo Larcher 2024-12-04 10:49:41 +01:00
parent 491b5726a6
commit 7788a6b849
No known key found for this signature in database
GPG Key ID: 3DAF63124699CA2B
1 changed files with 2 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
docs/source/multi_backend_support.md
View File

@ -6,8 +6,8 @@ whether you prioritize performance, ease of use, or compatibility with specific
TGI remains consistent across backends, allowing you to switch between them seamlessly.
**Supported backends:**
* TGI CUDA backend: This high-performance backend is optimized for NVIDIA GPUs and serves as the default option
* **TGI CUDA backend**: This high-performance backend is optimized for NVIDIA GPUs and serves as the default option
within TGI. Developed in-house, it boasts numerous optimizations and is used in production by various projects, including those by Hugging Face.
* [TGI TRTLLM backend](./backends/trtllm): This backend leverages NVIDIA's TensorRT library to accelerate LLM inference.
* **[TGI TRTLLM backend](./backends/trtllm)**: This backend leverages NVIDIA's TensorRT library to accelerate LLM inference.
It utilizes specialized optimizations and custom kernels for enhanced performance.
However, it requires a model-specific compilation step for each GPU architecture.