mirror of
https://github.com/huggingface/text-generation-inference.git
synced 2025-04-21 23:12:07 +00:00
47447ef017
- Always return the hidden states. - Create the output tensor inside the `attention` and `paged_attention` functions. This removes the difference between how the output is handled between attention (output parameter) and paged attention (return value). This also removes the assumption that the attention implementation can write to an output tensor (in preparation of FlashInfer). |
||
|---|---|---|
| .. | ||
| attention | ||
| awq | ||
| gptq | ||
| marlin | ||
| __init__.py | ||
| bnb.py | ||
| conv.py | ||
| eetq.py | ||
| exl2.py | ||
| fp8.py | ||
| layernorm.py | ||
| linear.py | ||
| lora.py | ||
| medusa.py | ||
| mlp.py | ||
| rotary.py | ||
| speculative.py | ||
| tensor_parallel.py | ||