huggingface/text-generation-inference
5
0
Fork 0
mirror of https://github.com/huggingface/text-generation-inference.git synced 2025-09-11 12:24:53 +00:00
Code Issues Packages Projects Releases Wiki Activity

Update docs after rebase.

Browse Source
This commit is contained in:
Nicolas Patry 2024-02-09 11:44:42 +00:00
parent 4b06f318cb
commit 7143130ba4
1 changed files with 2 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
docs/source/basic_tutorials/launcher.md
View File

@ -198,23 +198,22 @@ Options:
[default: 20]
```
## MAX_BATCH_SIZE
```shell
--max-batch-size <MAX_BATCH_SIZE>
Enforce a maximum number of requests per batch Specific flag for hardware targets that do not support unpadded inference
[env: MAX_BATCH_SIZE=]
```
```
## ENABLE_CUDA_GRAPHS
```shell
--enable-cuda-graphs
Enable experimental support for cuda graphs
[env: ENABLE_CUDA_GRAPHS=]
```
```
## HOSTNAME
```shell
--hostname <HOSTNAME>