huggingface/text-generation-inference
5
0
Fork 0
mirror of https://github.com/huggingface/text-generation-inference.git synced 2025-09-10 20:04:52 +00:00
Code Issues Packages Projects Releases Wiki Activity

Update docs/source/supported_models.md

Browse Source 
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
This commit is contained in:
Merve Noyan 2023-08-04 22:40:08 +03:00 committed by GitHub
parent 0c3f3cdb08
commit 801c7df18b
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
docs/source/supported_models.md
View File

@ -19,7 +19,7 @@ The following models are optimized and can be served with TGI:
- [MPT](https://huggingface.co/mosaicml/mpt-30b) - [MPT](https://huggingface.co/mosaicml/mpt-30b)
- [Llama V2](https://huggingface.co/meta-llama) - [Llama V2](https://huggingface.co/meta-llama)
If the above list lacks the model you would like to serve, depending on the model's pipeline type, you can try to initialize and serve the model on best-effort basis like below: If the above list lacks the model you would like to serve, depending on the model's pipeline type, you can try to initialize and serve the model anyways to see how well it performs but performance isn't guaranteed for non-optimized models:
`AutoModelForCausalLM.from_pretrained(<model>, device_map="auto")` `AutoModelForCausalLM.from_pretrained(<model>, device_map="auto")`