mirror of
https://github.com/huggingface/text-generation-inference.git
synced 2025-04-20 14:22:08 +00:00
c1cf36c0dc
* feat: refactor model, improve startup and re enable tests * fix: improve multimodal rotary embed caching * fix: limit vision flop calc to qwen2 vl models and update config typing * fix: include clippy lint * feat: refactor position ids in warmup and bump tests * fix: prefer default dtype * fix: enable all cuda graphs and bump snapshots * fix: adjust rotaty init path * fix: simplify get position ids and remove usused vision config * fix: update position ids so first dim is batch, simplify rotary and bump vlm default token limit * fix: improve position id init during cuda warmup for mrope and simplfy rotary forward * fix: check existance before accessing rope type in cuda warmup * fix: check key before access * fix: improve mrope check in cuda graph warmup * fix: remove check for default rope type * fix: add more test and improve model generation * fix: improve and simplify get_cos_sin, refactors and cleanup get_position_ids * fix: adjust signatures with types |
||
|---|---|---|
| .. | ||
| __init__.py | ||
| bloom_modeling.py | ||
| clip.py | ||
| flash_cohere_modeling.py | ||
| flash_dbrx_modeling.py | ||
| flash_deepseek_v2_modeling.py | ||
| flash_deepseek_v3_modeling.py | ||
| flash_gemma2_modeling.py | ||
| flash_gemma_modeling.py | ||
| flash_gpt2_modeling.py | ||
| flash_gptj_modeling.py | ||
| flash_llama_modeling.py | ||
| flash_mistral_modeling.py | ||
| flash_mixtral_modeling.py | ||
| flash_neox_modeling.py | ||
| flash_pali_gemma_modeling.py | ||
| flash_phi_modeling.py | ||
| flash_phi_moe_modeling.py | ||
| flash_qwen2_modeling.py | ||
| flash_rw_modeling.py | ||
| flash_santacoder_modeling.py | ||
| flash_starcoder2_modeling.py | ||
| idefics2.py | ||
| idefics3.py | ||
| idefics_config.py | ||
| idefics_image_processing.py | ||
| idefics_modeling.py | ||
| idefics_perceiver.py | ||
| idefics_processing.py | ||
| idefics_vision.py | ||
| llava_next.py | ||
| mamba_modeling.py | ||
| mllama.py | ||
| mpt_modeling.py | ||
| neox_modeling.py | ||
| opt_modeling.py | ||
| phi_modeling.py | ||
| qwen2_vl.py | ||
| siglip.py | ||
| t5_modeling.py | ||
| vlm.py | ||