huggingface/text-generation-inference
5
0
Fork 0
mirror of https://github.com/huggingface/text-generation-inference.git synced 2025-10-11 16:05:24 +00:00
Code Issues Packages Projects Releases Wiki Activity
dda015f2aa
text-generation-inference/backends/trtllm/README.md

250 B
Raw Blame History 

sequenceDiagram
    TensorRtLlmBackend -->> TensorRtLlmBackendImpl: New thread which instantiates actual backend impl
    TensorRtLlmBackendImpl -->> TensorRtLlmBackendImpl.Receiver: Awaits incoming request sent throught the queue