Daniël de Kok 628d6a13da Add support for exl2 quantization ... Mostly straightforward, changes to existing code: * Wrap quantizer parameters in a small wrapper to avoid passing around untyped tuples and needing to repack them as a dict. * Move scratch space computation to warmup, because we need the maximum input sequence length to avoid allocating huge scratch buffers that OOM.		2024-09-24 03:19:39 +00:00
..
awq	Refactor layers. (#1866)	2024-07-17 05:36:58 +00:00
gptq	Add support for exl2 quantization	2024-09-24 03:19:39 +00:00
__init__.py	MLPSpeculator. (#1865)	2024-07-17 05:36:58 +00:00
bnb.py	Update torch import reference in bnb quantization (#1902)	2024-07-17 05:36:58 +00:00
conv.py	Refactor layers. (#1866)	2024-07-17 05:36:58 +00:00
eetq.py	Refactor layers. (#1866)	2024-07-17 05:36:58 +00:00
exl2.py	Add support for exl2 quantization	2024-09-24 03:19:39 +00:00
fp8.py	Refactor layers. (#1866)	2024-07-17 05:36:58 +00:00
layernorm.py	MI300 compatibility (#1764)	2024-07-17 05:36:58 +00:00
linear.py	Add support for exl2 quantization	2024-09-24 03:19:39 +00:00
medusa.py	fix: use path inside of speculator config (#1935)	2024-07-17 05:36:58 +00:00
mlp.py	MLPSpeculator. (#1865)	2024-07-17 05:36:58 +00:00
rotary.py	reenable xpu for tgi (#1939)	2024-07-17 05:36:58 +00:00
speculative.py	MLPSpeculator. (#1865)	2024-07-17 05:36:58 +00:00
tensor_parallel.py	Add support for exl2 quantization	2024-09-24 03:19:39 +00:00