mirror of
https://github.com/huggingface/text-generation-inference.git
synced 2025-05-03 07:52:06 +00:00
parent
40dfce644a
commit
6afe4307ab
@ -39,6 +39,6 @@ The custom kernel supports bf16 and fp16 data types, block size of 16, head size
|
|||||||
|
|
||||||
## Unsupported features
|
## Unsupported features
|
||||||
|
|
||||||
The following features are currently not supported in the ROCm version of TGI, and the supported may be extended in the future:
|
The following features are currently not supported in the ROCm version of TGI, and the support may be extended in the future:
|
||||||
* Loading [AWQ](https://huggingface.co/docs/transformers/quantization#awq) checkpoints.
|
* Loading [AWQ](https://huggingface.co/docs/transformers/quantization#awq) checkpoints.
|
||||||
* Kernel for sliding window attention (Mistral)
|
* Kernel for sliding window attention (Mistral)
|
||||||
|
Loading…
Reference in New Issue
Block a user