text-generation-inference/server/text_generation_server/layers/gptq
Daniël de Kok 9256d7c38c
Some small fixes for the Torch 2.4.0 update (#2304)
* Fix GPTQ autotune data type to be compatible with Torch 2.4.0

* Update poetry lock file

* Fix small PaliGemma logprob differences after the torch update
2024-07-25 13:34:44 +02:00
..
__init__.py Add support for repacking AWQ weights for GPTQ-Marlin (#2278) 2024-07-23 13:08:20 +02:00
custom_autotune.py Some small fixes for the Torch 2.4.0 update (#2304) 2024-07-25 13:34:44 +02:00
exllama.py Fix GPTQWeight import (#2020) 2024-06-05 14:49:15 +02:00
exllamav2.py feat(fp8): use fbgemm kernels and load fp8 weights directly (#2248) 2024-07-20 19:02:04 +02:00
quant_linear.py Refactor layers. (#1866) 2024-05-13 12:44:30 +02:00
quantize.py Use symmetric quantization in the quantize subcommand (#2120) 2024-07-12 12:20:12 +02:00