mirror of
https://github.com/huggingface/text-generation-inference.git
synced 2025-10-12 08:25:22 +00:00
6 lines
250 B
Markdown
6 lines
250 B
Markdown
|
```mermaid
|
||
|
|
sequenceDiagram
|
||
|
|
TensorRtLlmBackend -->> TensorRtLlmBackendImpl: New thread which instantiates actual backend impl
|
||
|
|
TensorRtLlmBackendImpl -->> TensorRtLlmBackendImpl.Receiver: Awaits incoming request sent throught the queue
|
||
|
|
|
||
|
|
```
|