huggingface/text-generation-inference
5
0
Fork 0
mirror of https://github.com/huggingface/text-generation-inference.git synced 2025-10-08 22:45:23 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,041 Commits 240 Branches 66 Tags 36 MiB
48fec7b198
Commit Graph

2 Commits

Author SHA1 Message Date
OlivierDehaene
34c472bd64 chore: update to torch 2.4 (#2259) 
* chore: update to torch 2.4

* remove un-necessary patch

* fix
 2024-09-25 05:39:14 +00:00
OlivierDehaene
85f10ec5c9 feat(fp8): use fbgemm kernels and load fp8 weights directly (#2248) 
* feat(fp8): add support for fbgemm

* allow loading fp8 weights directly

* update outlines

* fix makefile

* build fbgemm

* avoid circular import and fix dockerfile

* add default dtype

* refactored weights loader

* fix auto conversion

* fix quantization config parsing

* force new nccl on install

* missing get_weights implementation

* increase timeout
 2024-09-25 05:30:41 +00:00