Daniël de Kok 288bcb0027 Add support for GPTQ-quantized MoE models using MoE Marlin (#2557) ... This change add support for MoE models that use GPTQ quantization. Currently only models with the following properties are supported: - No `desc_act` with tensor parallelism, unless `group_size=-1`. - No asymmetric quantization. - No AWQ.		2024-10-25 09:07:52 +00:00
..
images	Pali gemma modeling (#1895)	2024-07-17 05:36:58 +00:00
models	Add support for GPTQ-quantized MoE models using MoE Marlin (#2557)	2024-10-25 09:07:52 +00:00
conftest.py	Adding a test for FD. (#2516)	2024-09-25 06:17:09 +00:00
poetry.lock	Prefix test - Different kind of load test to trigger prefix test bugs. (#2490)	2024-09-25 06:14:07 +00:00
pyproject.toml	Prefix test - Different kind of load test to trigger prefix test bugs. (#2490)	2024-09-25 06:14:07 +00:00
pytest.ini	chore: add pre-commit (#1569)	2024-04-24 15:32:02 +03:00
requirements.txt	Prefix test - Different kind of load test to trigger prefix test bugs. (#2490)	2024-09-25 06:14:07 +00:00