huggingface/text-generation-inference
5
0
Fork 0
mirror of https://github.com/huggingface/text-generation-inference.git synced 2025-10-11 07:55:24 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,258 Commits 240 Branches 66 Tags 36 MiB
e7facf692f
Commit Graph

4 Commits

Author SHA1 Message Date
Adrien Gallouët
e7facf692f
Handle max_batch_size 
Signed-off-by: Adrien Gallouët <angt@huggingface.co>
 2025-02-04 13:32:58 +00:00
Adrien Gallouët
3eb4823f3e
Use max_batch_total_tokens 
Signed-off-by: Adrien Gallouët <angt@huggingface.co>
 2025-02-04 13:32:58 +00:00
Adrien Gallouët
bd0cc9905c
Get rid of llama_batch_get_one() 
Signed-off-by: Adrien Gallouët <angt@huggingface.co>
 2025-02-04 13:32:58 +00:00
Adrien Gallouët
95e221eece
Add llamacpp backend 
Signed-off-by: Adrien Gallouët <angt@huggingface.co>
 2025-02-04 13:32:56 +00:00