OlivierDehaene bf94df3c71 fix(server): use mem_get_info to get kv cache size (#664) ... Close https://github.com/huggingface/text-generation-inference/issues/649 Close https://github.com/huggingface/text-generation-inference/issues/651 Close https://github.com/huggingface/text-generation-inference/issues/653 Close #636		2023-07-20 17:23:49 +02:00
..
custom_autotune.py	feat(server): Add inference support for GPTQ (llama + falcon tested) + Quantization script (#438)	2023-06-26 12:27:01 +02:00
quant_linear.py	feat(server): Add inference support for GPTQ (llama + falcon tested) + Quantization script (#438)	2023-06-26 12:27:01 +02:00
quantize.py	fix(server): use mem_get_info to get kv cache size (#664)	2023-07-20 17:23:49 +02:00