Daniël de Kok 2ce8019480 Use GPTQ-Marlin for supported GPTQ configurations (#2111) ... GPTQ-Marlin is currently the best-performing kernel for GPTQ models. So let's use it by default if the kernels are installed, the GPU supports it, and the kernels support the configuration. For models generated by `text-generation-server quantize`, use `sym=False`. This subcommand symmetric quantization since the beginning and incorrectly reporting the model to be symmetric will use GPTQ-Marlin (which does not support asymmetric quantization).		2024-07-01 12:59:12 +02:00
..
adapters	Enable multiple LoRa adapters (#2010)	2024-06-25 14:46:27 -04:00
layers	Use GPTQ-Marlin for supported GPTQ configurations (#2111)	2024-07-01 12:59:12 +02:00
models	fix: use weights from base_layer (#2141)	2024-07-01 12:58:40 +02:00
pb	chore: add pre-commit (#1569)	2024-02-16 11:58:58 +01:00
utils	Use GPTQ-Marlin for supported GPTQ configurations (#2111)	2024-07-01 12:59:12 +02:00
__init__.py	feat(clients): Python client (#103)	2023-03-07 18:52:22 +01:00
cache.py	fix(server): decrease memory fragmentation (#557)	2023-07-06 14:28:33 +02:00
cli.py	Enable multiple LoRa adapters (#2010)	2024-06-25 14:46:27 -04:00
interceptor.py	v2.0.0 (#1736)	2024-04-12 18:38:34 +02:00
server.py	Enable multiple LoRa adapters (#2010)	2024-06-25 14:46:27 -04:00
tracing.py	Add OTLP Service Name Environment Variable (#2076)	2024-06-25 09:33:01 +02:00