Daniël de Kok 093a27c528 Add support for GPTQ Marlin (#2052) ... Add support for GPTQ Marlin kernels GPTQ Marlin extends the Marlin kernels to support common GPTQ configurations: - bits: 4 or 8 - groupsize: -1, 32, 64, or 128 - desc_act: true/false Using the GPTQ Marlin kernels requires repacking the parameters in the Marlin quantizer format. The kernels were contributed by Neural Magic to VLLM. We vendor them here for convenience.		2024-06-14 09:45:42 +02:00
..
layers	Add support for GPTQ Marlin (#2052)	2024-06-14 09:45:42 +02:00
models	Add support for GPTQ Marlin (#2052)	2024-06-14 09:45:42 +02:00
pb	chore: add pre-commit (#1569)	2024-02-16 11:58:58 +01:00
utils	Add support for GPTQ Marlin (#2052)	2024-06-14 09:45:42 +02:00
__init__.py	feat(clients): Python client (#103)	2023-03-07 18:52:22 +01:00
cache.py	fix(server): decrease memory fragmentation (#557)	2023-07-06 14:28:33 +02:00
cli.py	ROCm and sliding windows fixes (#2033)	2024-06-10 15:09:50 +08:00
interceptor.py	v2.0.0 (#1736)	2024-04-12 18:38:34 +02:00
server.py	ROCm and sliding windows fixes (#2033)	2024-06-10 15:09:50 +08:00
tracing.py	feat(clients): Python client (#103)	2023-03-07 18:52:22 +01:00