huggingface/text-generation-inference
5
0
Fork 0
mirror of https://github.com/huggingface/text-generation-inference.git synced 2025-04-21 23:12:07 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat: update readme and container version

Browse Source
This commit is contained in:
drbh 2024-04-26 00:46:11 +00:00
parent d4077f70db
commit a6890cbea9
2 changed files with 13 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

12
README.md
View File

@ -68,6 +68,18 @@ Text Generation Inference (TGI) is a toolkit for deploying and serving Large Lan
## Get Started
### Quick Start ⚡️
The fastest way to get started is to use the quickstart script. This script simplifies the docker and nvidia container toolkit installation process. It also installs the latest version of the text-generation-inference container and runs it with a default model.
```bash
curl --proto '=https' --tlsv1.2 -sSf \
https://raw.githubusercontent.com/huggingface/text-generation-inference/quickstart.sh \
| bash
```
![best practice script review](https://img.shields.io/badge/Best_Practice-yellow) Always review the contents of a script before running it.
### Docker
For a detailed starting guide, please see the [Quick Tour](https://huggingface.co/docs/text-generation-inference/quicktour). The easiest way of getting started is using the official Docker container:

2
quickstart.sh
View File

@ -35,4 +35,4 @@ model="HuggingFaceH4/zephyr-7b-beta"
volume="$PWD/data"
# Run the Docker container in interactive mode to allow CTRL+C to stop the container
docker run -it --gpus all --shm-size 1g -p 8080:80 -v "$volume:/data" ghcr.io/huggingface/text-generation-inference:2.0 --model-id "$model"
docker run -it --gpus all --shm-size 1g -p 8080:80 -v "$volume:/data" ghcr.io/huggingface/text-generation-inference:latest --model-id "$model"