huggingface/text-generation-inference
5
0
Fork 0
mirror of https://github.com/huggingface/text-generation-inference.git synced 2025-09-11 12:24:53 +00:00
Code Issues Packages Projects Releases Wiki Activity

update doc

Browse Source
This commit is contained in:
OlivierDehaene 2024-02-08 17:12:14 +01:00
parent faaa9dfe0a
commit 9e042bd117
1 changed files with 8 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

8
docs/source/basic_tutorials/launcher.md
View File

@ -197,6 +197,14 @@ Options:
[env: MAX_WAITING_TOKENS=] [env: MAX_WAITING_TOKENS=]
[default: 20] [default: 20]
```
## MAX_BATCH_SIZE
```shell
--max-batch-size <MAX_BATCH_SIZE>
Enforce a maximum number of requests per batch Specific flag for hardware targets that do not support unpadded inference
[env: MAX_BATCH_SIZE=]
``` ```
## HOSTNAME ## HOSTNAME
```shell ```shell