huggingface/text-generation-inference
5
0
Fork 0
mirror of https://github.com/huggingface/text-generation-inference.git synced 2025-04-22 15:32:08 +00:00
Code Issues Packages Projects Releases Wiki Activity
f3d6e8476e
text-generation-inference/launcher/src
History
Nicolas Patry a04356fb8c
Attempt for cleverer auto batch_prefill values (some simplifications). (#2808) 
* Attempt for cleverer auto batch_prefill values (some simplifications).

* Less flaky tests.

* Fixing typo insertion.

* Update launcher/src/main.rs

Co-authored-by: Daniël de Kok <me@danieldk.eu>

* Adding small comment for source of calculation.

* Adding L40.

* Adding L40s.

---------

Co-authored-by: Daniël de Kok <me@danieldk.eu>
2024-12-09 19:44:32 +01:00
..
env_runtime.rs add intel xpu support for TGI (#1475) 2024-04-26 15:48:58 +02:00
gpu.rs Remove compute capability lazy cell (#2580) 2024-09-30 08:48:47 +02:00
main.rs Attempt for cleverer auto batch_prefill values (some simplifications). (#2808) 2024-12-09 19:44:32 +01:00