huggingface/text-generation-inference
5
0
Fork 0
mirror of https://github.com/huggingface/text-generation-inference.git synced 2025-09-11 20:34:54 +00:00
Code Issues Packages Projects Releases Wiki Activity

chore(looper): cleanup a bit more

Browse Source
This commit is contained in:
Morgan Funtowicz 2024-10-21 16:57:26 +02:00 committed by Nicolas Patry
parent 7f383bf4dc
commit 3af45189b3
No known key found for this signature in database
GPG Key ID: D2920555C90F704C
1 changed files with 0 additions and 5 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
backends/trtllm/src/looper.rs
View File

@ -27,11 +27,6 @@ use crate::utils::first_line;
type InferResult<T> = Result<T, InferError>;
struct IdentifiableRequest<T> {
request_id: u64,
inner: T,
}
/// Wrap the requests along with the channel used to stream back to the client the decoded tokens
struct GenerationContext {
request: ValidGenerateRequest,