huggingface/text-generation-inference
5
0
Fork 0
mirror of https://github.com/huggingface/text-generation-inference.git synced 2025-04-20 14:22:08 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,260 Commits 227 Branches 59 Tags 47 MiB
4b8cda684b
Commit Graph

3 Commits

Author SHA1 Message Date
OlivierDehaene
53ec0b790b
feat(fp8): use fbgemm kernels and load fp8 weights directly (#2248) 
* feat(fp8): add support for fbgemm

* allow loading fp8 weights directly

* update outlines

* fix makefile

* build fbgemm

* avoid circular import and fix dockerfile

* add default dtype

* refactored weights loader

* fix auto conversion

* fix quantization config parsing

* force new nccl on install

* missing get_weights implementation

* increase timeout
 2024-07-20 19:02:04 +02:00
OlivierDehaene
630800eed3 v1.3.4 2023-12-22 15:46:04 +01:00
OlivierDehaene
564199bab3
feat: update exllamav2 kernels (#1370) 
Co-authored-by: Nicolas Patry <patry.nicolas@protonmail.com>
 2023-12-21 17:25:22 +01:00