3 Commits

Author	SHA1	Message	Date
Nicolas Patry	57b3495823	Fixing exl2 and other quanize tests again. (#2419) ... * Fixing exl2 and other quanize tests again. * Mark exl2 as non release (so CI tests them, needs to be removed latet). * Fixing exl2 (by disabling cuda graphs) * Fix quantization defaults without cuda graphs on exl2 (linked to new issues with it). * Removing serde override. * Go back to released exl2 and remove log. * Adding warnings for deprecated bitsandbytes + upgrade info to warn.	2024-08-15 11:12:51 +02:00
Daniël de Kok	fc9c3153e5	Add pytest release marker (#2114) ... * Add pytest release marker Annotate a test with `@pytest.mark.release` and it only gets run with `pytest integration-tests --release`. * Mark many models as `release` to speed up CI	2024-06-25 16:53:20 +02:00
Daniël de Kok	36dd16017c	Add support for exl2 quantization ... Mostly straightforward, changes to existing code: * Wrap quantizer parameters in a small wrapper to avoid passing around untyped tuples and needing to repack them as a dict. * Move scratch space computation to warmup, because we need the maximum input sequence length to avoid allocating huge scratch buffers that OOM.	2024-05-30 11:28:05 +02:00