Daniël de Kok 9935720c87 Add support for repacking AWQ weights for GPTQ-Marlin (#2278) ... * Add support for repacking AWQ weights for GPTQ-Marlin So far we couldn't support AWQ because virtually all AWQ models use symmetric quantization, which GPTQ-Marlin did not suppors. GPTQ-Marlin has recently added support AWQ repacking and AWQ asymmetric quantization (zero_point=True). This change updates all GPTQ-Marlin kernels from upstream and wires up AWQ support. For now enabling AWQ using Marlin requires running TGI with `--quantize gptq`. * Enable Marlin for supported AWQ configurations by default This makes the AWQ -> GPTQ repack test redundant, since we are now testing this with the regular AWQ test.		2024-07-23 13:08:20 +02:00
..
__init__.py	Add support for repacking AWQ weights for GPTQ-Marlin (#2278)	2024-07-23 13:08:20 +02:00
custom_autotune.py	Refactor layers. (#1866)	2024-05-13 12:44:30 +02:00
exllama.py	Fix GPTQWeight import (#2020)	2024-06-05 14:49:15 +02:00
exllamav2.py	feat(fp8): use fbgemm kernels and load fp8 weights directly (#2248)	2024-07-20 19:02:04 +02:00
quant_linear.py	Refactor layers. (#1866)	2024-05-13 12:44:30 +02:00
quantize.py	Use symmetric quantization in the quantize subcommand (#2120)	2024-07-12 12:20:12 +02:00