From 744d78d88ecdf24b66da9a184c4fb4c5d018ccd2 Mon Sep 17 00:00:00 2001 From: osanseviero Date: Wed, 9 Aug 2023 23:14:23 +0200 Subject: [PATCH] Add docker run --help flag + add curl example in docker_launch --- docs/source/docker_launch.md | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/docs/source/docker_launch.md b/docs/source/docker_launch.md index 3ed675c4..623590a7 100644 --- a/docs/source/docker_launch.md +++ b/docs/source/docker_launch.md @@ -17,8 +17,18 @@ To use GPUs, you need to install the [NVIDIA Container Toolkit](https://docs.nvi +Once TGI is running, you can use the `generate` endpoint by doing requests. To learn more about how to query the endpoints, check the [Consuming TGI](./basic_tutorials/consuming_tgi) section. + +```shell +curl 127.0.0.1:8080/generate -X POST -d '{"inputs":"What is Deep Learning?","parameters":{"max_new_tokens":20}}' -H 'Content-Type: application/json' +``` + + + To see all possible flags and options, you can use the `--help` flag. It's possible to configure the number of shards, quantization, generation parameters, and more. -``` +```shell docker run ghcr.io/huggingface/text-generation-inference:1.0.0 --help -``` \ No newline at end of file +``` + + \ No newline at end of file