huggingface/text-generation-inference
5
0
Fork 0
mirror of https://github.com/huggingface/text-generation-inference.git synced 2025-04-22 15:32:08 +00:00
Code Issues Packages Projects Releases Wiki Activity
f1f28404e7
text-generation-inference/server/text_generation_server/models/custom_modeling
History
Daniël de Kok f1f28404e7 Add support for GPTQ Marlin (#2052) 
Add support for GPTQ Marlin kernels

GPTQ Marlin extends the Marlin kernels to support common GPTQ
configurations:

- bits: 4 or 8
- groupsize: -1, 32, 64, or 128
- desc_act: true/false

Using the GPTQ Marlin kernels requires repacking the parameters in the
Marlin quantizer format.

The kernels were contributed by Neural Magic to VLLM. We vendor them
here for convenience.
2024-09-24 03:43:30 +00:00
..
__init__.py feat(server): flash santacoder (#153) 2023-04-03 19:06:42 +02:00
bloom_modeling.py Refactor layers. (#1866) 2024-07-17 05:36:58 +00:00
clip.py Refactor layers. (#1866) 2024-07-17 05:36:58 +00:00
flash_cohere_modeling.py Add support for GPTQ Marlin (#2052) 2024-09-24 03:43:30 +00:00
flash_dbrx_modeling.py Add Phi-3 medium support (#2039) 2024-09-24 03:42:29 +00:00
flash_gemma_modeling.py Add support for Marlin-quantized models 2024-09-24 03:38:05 +00:00
flash_gpt2_modeling.py Add Phi-3 medium support (#2039) 2024-09-24 03:42:29 +00:00
flash_llama_modeling.py Add Phi-3 medium support (#2039) 2024-09-24 03:42:29 +00:00
flash_mistral_modeling.py Purely refactors paged/attention into layers/attention and make hardware differences more obvious with 1 file per hardware. (#1986) 2024-09-24 03:19:39 +00:00
flash_mixtral_modeling.py Add support for Marlin-quantized models 2024-09-24 03:38:05 +00:00
flash_neox_modeling.py feat: move allocation logic to rust (#1835) 2024-09-24 03:34:15 +00:00
flash_pali_gemma_modeling.py Pali gemma modeling (#1895) 2024-07-17 05:36:58 +00:00
flash_phi_modeling.py Add support for Marlin-quantized models 2024-09-24 03:38:05 +00:00
flash_qwen2_modeling.py Add support for Marlin-quantized models 2024-09-24 03:38:05 +00:00
flash_rw_modeling.py feat: move allocation logic to rust (#1835) 2024-09-24 03:34:15 +00:00
flash_santacoder_modeling.py Add support for GPTQ Marlin (#2052) 2024-09-24 03:43:30 +00:00
flash_starcoder2_modeling.py Add support for GPTQ Marlin (#2052) 2024-09-24 03:43:30 +00:00
idefics2.py MLPSpeculator. (#1865) 2024-07-17 05:36:58 +00:00
idefics_config.py chore: add pre-commit (#1569) 2024-04-24 15:32:02 +03:00
idefics_image_processing.py chore: formatting 2024-04-18 16:26:00 +03:00
idefics_modeling.py reenable xpu for tgi (#1939) 2024-07-17 05:36:58 +00:00
idefics_perceiver.py Refactor layers. (#1866) 2024-07-17 05:36:58 +00:00
idefics_processing.py chore: add pre-commit (#1569) 2024-04-24 15:32:02 +03:00
idefics_vision.py Refactor layers. (#1866) 2024-07-17 05:36:58 +00:00
llava_next.py Aligin the source code with main branch 2.0.4 2024-09-24 03:06:55 +00:00
mamba_modeling.py Refactor layers. (#1866) 2024-07-17 05:36:58 +00:00
mpt_modeling.py Refactor layers. (#1866) 2024-07-17 05:36:58 +00:00
neox_modeling.py Refactor layers. (#1866) 2024-07-17 05:36:58 +00:00
opt_modeling.py fix(server): fix OPT implementation (#2061) 2024-09-24 03:42:29 +00:00
phi_modeling.py Refactor layers. (#1866) 2024-07-17 05:36:58 +00:00
siglip.py Removing some unused code. (#1915) 2024-07-17 05:36:58 +00:00
t5_modeling.py Refactor layers. (#1866) 2024-07-17 05:36:58 +00:00
vlm.py Pali gemma modeling (#1895) 2024-07-17 05:36:58 +00:00