Nicolas Patry b70ae0969f Prefix caching (#2402) ... * Prefix caching WIP * Fixing prefix attention. * Fixing flashinfer import. * Fixing black. * Fixing medusa (still wrong outputs, but functional). * Just medusa values now. * Fixing medusa without prefix caching. * Fixing prefix caching. * Medusa requires reshaping. * Removing the logs. * Remove router.nix * Fixup: - Remove logs - Disable VLMs (they do not work) - Disable prefix caching when user wants prefill logprobs. * Update flake.lock --------- Co-authored-by: Daniël de Kok <me@danieldk.eu>		2024-08-20 11:15:30 +02:00
..
custom_modeling	Prefix caching (#2402)	2024-08-20 11:15:30 +02:00
__init__.py	feat: validate template variables before apply and improve sliding wi… (#2403)	2024-08-12 10:58:40 -04:00
bloom.py	Refactor dead code - Removing all flash_xxx.py files. (#2166)	2024-07-05 10:29:56 +02:00
causal_lm.py	Fixing exl2 and other quanize tests again. (#2419)	2024-08-15 11:12:51 +02:00
flash_causal_lm.py	Prefix caching (#2402)	2024-08-20 11:15:30 +02:00
galactica.py	feat: add ruff and resolve issue (#2262)	2024-07-26 10:29:09 -04:00
globals.py	Prefix caching (#2402)	2024-08-20 11:15:30 +02:00
idefics_causal_lm.py	Upgrading exl2. (#2415)	2024-08-14 11:58:08 +02:00
idefics.py	Upgrading exl2. (#2415)	2024-08-14 11:58:08 +02:00
mamba.py	Fixing exl2 and other quanize tests again. (#2419)	2024-08-15 11:12:51 +02:00
model.py	feat: add ruff and resolve issue (#2262)	2024-07-26 10:29:09 -04:00
pali_gemma.py	feat: add ruff and resolve issue (#2262)	2024-07-26 10:29:09 -04:00
seq2seq_lm.py	Fixing exl2 and other quanize tests again. (#2419)	2024-08-15 11:12:51 +02:00
types.py	feat: add ruff and resolve issue (#2262)	2024-07-26 10:29:09 -04:00
vlm_causal_lm.py	Prefix caching (#2402)	2024-08-20 11:15:30 +02:00