Daniël de Kok 653193a942 Improve support for GPUs with capability < 8 (#2575) ... * Improve support for GPUs with capability < 8 - For models that cannot use flashinfer, use flash-attn v1 + paged attention for models with a compute capability older than 8. - Disable prefix caching when using paged attention. - When using flash-attn v1, pass the key/value, rather than the cache, since v1 cannot use block tables. * nix: add flash-attn-v1 to the server environment * Move disabling prefix caching into the block of exceptions * Capability as `usize`s		2024-10-25 09:01:04 +00:00
..
custom_modeling	Improve support for GPUs with capability < 8 (#2575)	2024-10-25 09:01:04 +00:00
__init__.py	Add missing import package	2024-10-25 08:52:24 +00:00
bloom.py	Make Gaudi adapt to the tgi 2.3.0	2024-09-26 06:04:55 +00:00
causal_lm.py	Simplify the warmup	2024-10-25 08:38:59 +00:00
flash_causal_lm.py	hotfix : enable intel ipex cpu and xpu in python3.11 (#2517)	2024-09-25 06:15:35 +00:00
galactica.py	feat: add ruff and resolve issue (#2262)	2024-09-25 05:46:24 +00:00
globals.py	Make Gaudi adapt to the tgi 2.3.0	2024-09-26 06:04:55 +00:00
idefics_causal_lm.py	Upgrading exl2. (#2415)	2024-09-25 06:07:40 +00:00
idefics.py	Upgrading exl2. (#2415)	2024-09-25 06:07:40 +00:00
mamba.py	Fixing exl2 and other quanize tests again. (#2419)	2024-09-25 06:08:38 +00:00
model.py	Pass the max_batch_total_tokens to causal_lm	2024-10-23 08:28:26 +00:00
pali_gemma.py	feat: add ruff and resolve issue (#2262)	2024-09-25 05:46:24 +00:00
seq2seq_lm.py	Fixing exl2 and other quanize tests again. (#2419)	2024-09-25 06:08:38 +00:00
starcoder.py	Make Gaudi adapt to the tgi 2.3.0	2024-09-26 06:04:55 +00:00
types.py	feat: add ruff and resolve issue (#2262)	2024-09-25 05:46:24 +00:00
vlm_causal_lm.py	Merge branch 'habana-main' into 2.3.0	2024-10-23 16:32:12 +08:00