huggingface/text-generation-inference
5
0
Fork 0
mirror of https://github.com/huggingface/text-generation-inference.git synced 2025-11-18 23:15:59 +00:00
Code Issues Packages Projects Releases Wiki Activity
782 Commits 241 Branches 66 Tags 36 MiB
feat/backend_feature
Commit Graph

2 Commits

Author SHA1 Message Date
OlivierDehaene
8aece3bd68
feat: move allocation logic to rust (#1835) 
Close #2007
 2024-06-05 12:18:38 +02:00
OlivierDehaene
757223b352
feat: add SchedulerV3 (#1996) 
- Refactor code to allow supporting multiple versions of the
generate.proto at the same time
- Add v3/generate.proto (ISO to generate.proto for now but allow for
future changes without impacting v2 backends)
- Add Schedule trait to abstract queuing and batching mechanisms that
will be different in the future
- Add SchedulerV2/V3 impl
 2024-06-04 15:56:56 +02:00