huggingface/text-generation-inference
5
0
Fork 0
mirror of https://github.com/huggingface/text-generation-inference.git synced 2025-04-24 08:22:07 +00:00
Code Issues Packages Projects Releases Wiki Activity
0d794af6a5
text-generation-inference/router/client/src
History
OlivierDehaene 532146338b
feat(router): add max_batch_size (#1542) 
Some hardware require a maximum batch size.
2024-02-09 12:38:41 +01:00
..
pb Init 2022-10-08 12:30:12 +02:00
client.rs feat(router): add max_batch_size (#1542) 2024-02-09 12:38:41 +01:00
lib.rs Speculative (#1308) 2023-12-11 12:46:30 +01:00
sharded_client.rs feat(router): add max_batch_size (#1542) 2024-02-09 12:38:41 +01:00