huggingface/text-generation-inference
5
0
Fork 0
mirror of https://github.com/huggingface/text-generation-inference.git synced 2025-10-11 07:55:24 +00:00
Code Issues Packages Projects Releases Wiki Activity
9c60c9ca43
text-generation-inference/backends/trtllm/README.md

250 B
Raw Blame History 

sequenceDiagram
    TensorRtLlmBackend -->> TensorRtLlmBackendImpl: New thread which instantiates actual backend impl
    TensorRtLlmBackendImpl -->> TensorRtLlmBackendImpl.Receiver: Awaits incoming request sent throught the queue