huggingface/text-generation-inference
5
0
Fork 0
mirror of https://github.com/huggingface/text-generation-inference.git synced 2025-09-10 20:04:52 +00:00
Code Issues Packages Projects Releases Wiki Activity

Add docker run --help flag + add curl example in docker_launch

Browse Source
This commit is contained in:
osanseviero 2023-08-09 23:14:23 +02:00
parent 38652c9da5
commit 744d78d88e
1 changed files with 12 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

14
docs/source/docker_launch.md
View File

@ -17,8 +17,18 @@ To use GPUs, you need to install the [NVIDIA Container Toolkit](https://docs.nvi
</Tip> </Tip>
Once TGI is running, you can use the `generate` endpoint by doing requests. To learn more about how to query the endpoints, check the [Consuming TGI](./basic_tutorials/consuming_tgi) section.
```shell
curl 127.0.0.1:8080/generate -X POST -d '{"inputs":"What is Deep Learning?","parameters":{"max_new_tokens":20}}' -H 'Content-Type: application/json'
```
<Tip>
To see all possible flags and options, you can use the `--help` flag. It's possible to configure the number of shards, quantization, generation parameters, and more. To see all possible flags and options, you can use the `--help` flag. It's possible to configure the number of shards, quantization, generation parameters, and more.
``` ```shell
docker run ghcr.io/huggingface/text-generation-inference:1.0.0 --help docker run ghcr.io/huggingface/text-generation-inference:1.0.0 --help
``` ```
</Tip>