huggingface/text-generation-inference
5
0
Fork 0
mirror of https://github.com/huggingface/text-generation-inference.git synced 2025-09-10 20:04:52 +00:00
Code Issues Packages Projects Releases Wiki Activity

Update docs/source/supported_models.md

Browse Source 
Co-authored-by: Nicolas Patry <patry.nicolas@protonmail.com>
This commit is contained in:
Merve Noyan 2023-09-12 12:27:16 +02:00 committed by GitHub
parent 9a0a4d926c
commit 62b05f2ccf
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 3 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
docs/source/supported_models.md
View File

@ -28,7 +28,9 @@ AutoModelForCausalLM.from_pretrained(<model>, device_map="auto")`
AutoModelForSeq2SeqLM.from_pretrained(<model>, device_map="auto")
```
If you wish to serve a supported model that already exists on a local folder, you can use `weight-cache-override` flag like below. Otherwise, it will be downloaded to Hugging Face Hub cache.
If you wish to serve a supported model that already exists on a local folder, Just point to the local folder.
text-generation-launcher --model-id <PATH-TO-LOCAL-BLOOM>
```bash
text-generation-launcher --model-id bigscience/bloom --weights-cache-override <PATH-TO-LOCAL-BLOOM>