huggingface/text-generation-inference
5
0
Fork 0
mirror of https://github.com/huggingface/text-generation-inference.git synced 2025-07-28 02:40:16 +00:00
Code Issues Packages Projects Releases Wiki Activity
no_root_user
text-generation-inference/backends/trtllm
History
Hugo Larcher d8ff7f2623
feat: add support for HF_HUB_USER_AGENT_ORIGIN to add user-agent Origin field in Hub requests. () 
* feat: add support for HF_HUB_USER_AGENT_ORIGIN to add user-agent Origin field in Hub requests.

* fix: Rust version for Neuron

* fix: PR comments, use rust-toolchain.toml
2025-03-04 16:43:50 +01:00
..
cmake [Backend] Bump TRTLLM to v.0.17.0 () 2025-02-06 16:45:03 +01:00
csrc Give TensorRT-LLMa proper CI/CD 😍 () 2025-01-21 10:19:16 +01:00
scripts [Backend] Bump TRTLLM to v.0.17.0 () 2025-02-06 16:45:03 +01:00
src feat: add support for HF_HUB_USER_AGENT_ORIGIN to add user-agent Origin field in Hub requests. () 2025-03-04 16:43:50 +01:00
tests Give TensorRT-LLMa proper CI/CD 😍 () 2025-01-21 10:19:16 +01:00
build.rs [Backend] Bump TRTLLM to v.0.17.0 () 2025-02-06 16:45:03 +01:00
Cargo.toml [TRTLLM] Expose finish reason () 2025-01-23 16:48:26 +01:00
CMakeLists.txt [Backend] Bump TRTLLM to v.0.17.0 () 2025-02-06 16:45:03 +01:00
README.md Rebase TRT-llm () 2024-07-31 10:33:10 +02:00

README.md

Text Generation Inference - TensorRT-LLM Backend Implementation

Description

This folder provides the sources of the TensorRT-LLM backend implementation powered by TensorRT-LLM Executor new API

Simplified Request Sequence