huggingface/text-generation-inference
5
0
Fork 0
mirror of https://github.com/huggingface/text-generation-inference.git synced 2025-04-22 15:32:08 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,184 Commits 228 Branches 59 Tags 47 MiB
ff905aeff3
Commit Graph

3 Commits

Author SHA1 Message Date
OlivierDehaene
85f10ec5c9 feat(fp8): use fbgemm kernels and load fp8 weights directly (#2248) 
* feat(fp8): add support for fbgemm

* allow loading fp8 weights directly

* update outlines

* fix makefile

* build fbgemm

* avoid circular import and fix dockerfile

* add default dtype

* refactored weights loader

* fix auto conversion

* fix quantization config parsing

* force new nccl on install

* missing get_weights implementation

* increase timeout
 2024-09-25 05:30:41 +00:00
OlivierDehaene
62646c2a54 v1.3.4 2024-04-22 09:08:34 +03:00
OlivierDehaene
7eeabb9cda feat: update exllamav2 kernels (#1370) 
Co-authored-by: Nicolas Patry <patry.nicolas@protonmail.com>
 2024-04-22 09:02:53 +03:00