huggingface/text-generation-inference
5
0
Fork 0
mirror of https://github.com/huggingface/text-generation-inference.git synced 2025-10-16 18:35:23 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,442 Commits 240 Branches 66 Tags 36 MiB
efb94e0d3d
Commit Graph

2 Commits

Author SHA1 Message Date
Nicolas Patry
a04356fb8c
Attempt for cleverer auto batch_prefill values (some simplifications). (#2808) 
* Attempt for cleverer auto batch_prefill values (some simplifications).

* Less flaky tests.

* Fixing typo insertion.

* Update launcher/src/main.rs

Co-authored-by: Daniël de Kok <me@danieldk.eu>

* Adding small comment for source of calculation.

* Adding L40.

* Adding L40s.

---------

Co-authored-by: Daniël de Kok <me@danieldk.eu>
 2024-12-09 19:44:32 +01:00
Nicolas Patry
38fcafcf96
Adding a test for FD. (#2516) 
* Adding a test for FD.

* Fixing flashdecoding (empty batch doesn't work).

* Fixing the invalid popping.

* Fixing radix with block_size > 1

* Last reference.

* Use an actual hash.

* Update hash for slice.len() == 1

* Update the locks.

* Increasing docker timeout.
 2024-09-16 17:00:54 +02:00