huggingface/text-generation-inference
5
0
Fork 0
mirror of https://github.com/huggingface/text-generation-inference.git synced 2025-04-24 00:12:08 +00:00
Code Issues Packages Projects Releases Wiki Activity
b36c0f8436
text-generation-inference/server/text_generation_server/models/custom_modeling
History
drbh d888bc2828 feat: support force downcast after FastRMSNorm multiply for Gemma (#1658) 
This PR adds `force_downcast_after` to `FastRMSNorm.forward` which is
used in the Gemma model. References
https://github.com/huggingface/transformers/pull/29402 and
https://github.com/huggingface/transformers/pull/29729

Setting `force_downcast_after=True` will perform the `hidden_states *
weight` multiplication in f32 and then downcast to half. This differs
slightly from the current implementation which first casts the
`hidden_states` to a half and then multiples.
2024-04-25 12:32:42 +03:00
..
__init__.py feat(server): flash santacoder (#153) 2023-04-03 19:06:42 +02:00
bloom_modeling.py feat: Qwen2 (#1608) 2024-04-25 09:21:22 +03:00
flash_gemma_modeling.py feat: support force downcast after FastRMSNorm multiply for Gemma (#1658) 2024-04-25 12:32:42 +03:00
flash_llama_modeling.py Revamp medusa implementation so that every model can benefit. (#1588) 2024-04-25 09:13:03 +03:00
flash_mistral_modeling.py Revamp medusa implementation so that every model can benefit. (#1588) 2024-04-25 09:13:03 +03:00
flash_mixtral_modeling.py Revamp medusa implementation so that every model can benefit. (#1588) 2024-04-25 09:13:03 +03:00
flash_neox_modeling.py Revamp medusa implementation so that every model can benefit. (#1588) 2024-04-25 09:13:03 +03:00
flash_phi_modeling.py Revamp medusa implementation so that every model can benefit. (#1588) 2024-04-25 09:13:03 +03:00
flash_qwen2_modeling.py feat: Qwen2 (#1608) 2024-04-25 09:21:22 +03:00
flash_rw_modeling.py Revamp medusa implementation so that every model can benefit. (#1588) 2024-04-25 09:13:03 +03:00
flash_santacoder_modeling.py Revamp medusa implementation so that every model can benefit. (#1588) 2024-04-25 09:13:03 +03:00
flash_starcoder2_modeling.py feat: starcoder2 (#1605) 2024-04-25 09:18:55 +03:00
idefics_config.py chore: add pre-commit (#1569) 2024-04-24 15:32:02 +03:00
idefics_image_processing.py chore: formatting 2024-04-18 16:26:00 +03:00
idefics_modeling.py Revamp medusa implementation so that every model can benefit. (#1588) 2024-04-25 09:13:03 +03:00
idefics_perceiver.py feat: format code (#1070) 2023-09-27 12:22:09 +02:00
idefics_processing.py chore: add pre-commit (#1569) 2024-04-24 15:32:02 +03:00
idefics_vision.py feat: format code (#1070) 2023-09-27 12:22:09 +02:00
mamba_modeling.py Revamp medusa implementation so that every model can benefit. (#1588) 2024-04-25 09:13:03 +03:00
mpt_modeling.py Revamp medusa implementation so that every model can benefit. (#1588) 2024-04-25 09:13:03 +03:00
neox_modeling.py feat: Qwen2 (#1608) 2024-04-25 09:21:22 +03:00
opt_modeling.py feat: Qwen2 (#1608) 2024-04-25 09:21:22 +03:00
phi_modeling.py Revamp medusa implementation so that every model can benefit. (#1588) 2024-04-25 09:13:03 +03:00
t5_modeling.py Revamp medusa implementation so that every model can benefit. (#1588) 2024-04-25 09:13:03 +03:00