huggingface/text-generation-inference
5
0
Fork 0
mirror of https://github.com/huggingface/text-generation-inference.git synced 2025-09-10 20:04:52 +00:00
Code Issues Packages Projects Releases Wiki Activity

Added note on weight-cache-override

Browse Source
This commit is contained in:
Merve Noyan 2023-09-07 16:20:56 +02:00 committed by GitHub
parent b03d2621a7
commit eb8f59083d
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 4 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
docs/source/supported_models.md
View File

@ -28,6 +28,10 @@ AutoModelForCausalLM.from_pretrained(<model>, device_map="auto")`
AutoModelForSeq2SeqLM.from_pretrained(<model>, device_map="auto")
```
If you wish to serve a different version of a model that exists in a local folder, you can use `weight-cache-override` flag like below 👇
```bash
text-generation-launcher --model-id bigscience/bloom --weights-cache-override <PATH-TO-LOCAL-BLOOM>
```
## Supported Hardware