huggingface/text-generation-inference
5
0
Fork 0
mirror of https://github.com/huggingface/text-generation-inference.git synced 2025-05-28 01:32:07 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,062 Commits 234 Branches 61 Tags 29 MiB
e9ba044250
Commit Graph

2 Commits

Author SHA1 Message Date
OlivierDehaene
34c472bd64 chore: update to torch 2.4 (#2259) 
* chore: update to torch 2.4

* remove un-necessary patch

* fix
 2024-09-25 05:39:14 +00:00
OlivierDehaene
85f10ec5c9 feat(fp8): use fbgemm kernels and load fp8 weights directly (#2248) 
* feat(fp8): add support for fbgemm

* allow loading fp8 weights directly

* update outlines

* fix makefile

* build fbgemm

* avoid circular import and fix dockerfile

* add default dtype

* refactored weights loader

* fix auto conversion

* fix quantization config parsing

* force new nccl on install

* missing get_weights implementation

* increase timeout
 2024-09-25 05:30:41 +00:00