huggingface/text-generation-inference
5
0
Fork 0
mirror of https://github.com/huggingface/text-generation-inference.git synced 2025-04-23 16:02:10 +00:00
Code Issues Packages Projects Releases Wiki Activity
98d0093660
text-generation-inference/server/text_generation_server/models
History
Daniël de Kok 72ab60fdd5
Use FP8 KV cache when specified by compressed-tensors (#2761) 
The compressed-tensors configuration can specify the configuration of
the KV cache as well. Use an FP8 KV cache when the configuration tells
us to do so (all other options and types are ignored for now).
2024-11-26 08:27:41 +01:00
..
custom_modeling fix: adjust llama MLP name from dense to mlp to correctly apply lora (#2760) 2024-11-19 15:10:22 -05:00
__init__.py Use FP8 KV cache when specified by compressed-tensors (#2761) 2024-11-26 08:27:41 +01:00
bloom.py Refactor dead code - Removing all flash_xxx.py files. (#2166) 2024-07-05 10:29:56 +02:00
causal_lm.py feat: prefill chunking (#2600) 2024-10-16 12:49:33 +02:00
flash_causal_lm.py feat: add payload limit (#2726) 2024-11-21 18:20:15 +00:00
galactica.py feat: add ruff and resolve issue (#2262) 2024-07-26 10:29:09 -04:00
globals.py feat: prefill chunking (#2600) 2024-10-16 12:49:33 +02:00
idefics_causal_lm.py feat: prefill chunking (#2600) 2024-10-16 12:49:33 +02:00
mamba.py Choosing input/total tokens automatically based on available VRAM? (#2673) 2024-10-28 04:59:49 +01:00
metadata_kernels.py feat: add payload limit (#2726) 2024-11-21 18:20:15 +00:00
mllama_causal_lm.py feat: add triton kernels to decrease latency of large batches (#2687) 2024-10-25 21:10:00 +00:00
model.py Choosing input/total tokens automatically based on available VRAM? (#2673) 2024-10-28 04:59:49 +01:00
pali_gemma.py feat: add ruff and resolve issue (#2262) 2024-07-26 10:29:09 -04:00
seq2seq_lm.py feat: prefill chunking (#2600) 2024-10-16 12:49:33 +02:00
types.py feat: prefill chunking (#2600) 2024-10-16 12:49:33 +02:00
vlm_causal_lm.py fix cuda graphs for qwen2-vl (#2708) 2024-11-01 03:05:34 +01:00