huggingface/text-generation-inference
5
0
Fork 0
mirror of https://github.com/huggingface/text-generation-inference.git synced 2025-09-11 04:14:52 +00:00
Code Issues Packages Projects Releases Wiki Activity

Update docs/source/basic_tutorials/non_core_models.md

Browse Source 
Co-authored-by: Omar Sanseviero <osanseviero@gmail.com>
This commit is contained in:
Merve Noyan 2023-08-22 23:44:13 +03:00 committed by GitHub
parent b4b52c6f32
commit abde90c493
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
docs/source/basic_tutorials/non_core_models.md
View File

@ -12,7 +12,7 @@ AutoModelForCausalLM.from_pretrained(<model>, device_map="auto")``
AutoModelForSeq2SeqLM.from_pretrained(<model>, device_map="auto") AutoModelForSeq2SeqLM.from_pretrained(<model>, device_map="auto")
``` ```
This means, you will be unable to use some of the features introduced by TGI, such as tensor-parallel sharding or flash attention. However, you can still get many benefits of TGI, such as continuous batching, or streaming outputs. This means you will be unable to use some of the features introduced by TGI, such as tensor-parallel sharding or flash attention. However, you can still get many benefits of TGI, such as continuous batching or streaming outputs.
You can serve these models using docker like below 👇 You can serve these models using docker like below 👇