huggingface/text-generation-inference
5
0
Fork 0
mirror of https://github.com/huggingface/text-generation-inference.git synced 2025-10-08 22:45:23 +00:00
Code Issues Packages Projects Releases Wiki Activity
482 Commits 240 Branches 66 Tags 36 MiB
b2b5df0e94
Commit Graph

4 Commits

Author SHA1 Message Date
fxmarty
b2b5df0e94
Add RoCm support (#1243) 
This PR adds support for AMD Instinct MI210 & MI250 GPUs, with paged
attention and FAv2 support.

Remaining items to discuss, on top of possible others:
* Should we have a
`ghcr.io/huggingface/text-generation-inference:1.1.0+rocm` hosted image,
or is it too early?
* Should we set up a CI on MI210/MI250? I don't have access to the
runners of TGI though.
* Are we comfortable with those changes being directly in TGI, or do we
need a fork?

---------

Co-authored-by: Felix Marty <felix@hf.co>
Co-authored-by: OlivierDehaene <olivier@huggingface.co>
Co-authored-by: Your Name <you@example.com>
 2023-11-27 14:08:12 +01:00
OlivierDehaene
12590fdcce
feat: paged attention v2 (#1183) 2023-10-23 12:29:25 +02:00
OlivierDehaene
3b56d7669b
feat: add mistral model (#1071) 2023-09-28 09:55:47 +02:00
OlivierDehaene
3b71c38558
feat(server): flash attention v2 (#624) 2023-07-18 16:21:18 +02:00