huggingface/text-generation-inference
5
0
Fork 0
mirror of https://github.com/huggingface/text-generation-inference.git synced 2025-09-11 04:14:52 +00:00
Code Issues Packages Projects Releases Wiki Activity

Normalize a bit.

Browse Source
This commit is contained in:
Nicolas Patry 2023-10-04 12:39:39 +02:00
parent fcf0f890d2
commit 2469deedcc
2 changed files with 8 additions and 11 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

14
docs/source/basic_tutorials/launcher.md
View File

@ -77,7 +77,7 @@ Options:
[possible values: float16, bfloat16] [possible values: float16, bfloat16]
``` ```
## trust-remote-code ## TRUST_REMOTE_CODE
```shell ```shell
--trust-remote-code --trust-remote-code
Whether you want to execute hub modelling code. Explicitly passing a `revision` is encouraged when loading a model with custom code to ensure no malicious code has been contributed in a newer revision Whether you want to execute hub modelling code. Explicitly passing a `revision` is encouraged when loading a model with custom code to ensure no malicious code has been contributed in a newer revision
@ -251,7 +251,7 @@ Options:
[env: WEIGHTS_CACHE_OVERRIDE=] [env: WEIGHTS_CACHE_OVERRIDE=]
``` ```
## disable-custom-kernels ## DISABLE_CUSTOM_KERNELS
```shell ```shell
--disable-custom-kernels --disable-custom-kernels
For some models (like bloom), text-generation-inference implemented custom cuda kernels to speed up inference. Those kernels were only tested on A100. Use this flag to disable them if you're running on different hardware and encounter issues For some models (like bloom), text-generation-inference implemented custom cuda kernels to speed up inference. Those kernels were only tested on A100. Use this flag to disable them if you're running on different hardware and encounter issues
@ -291,7 +291,7 @@ Options:
[env: ROPE_FACTOR=] [env: ROPE_FACTOR=]
``` ```
## json-output ## JSON_OUTPUT
```shell ```shell
--json-output --json-output
Outputs the logs in JSON format (useful for telemetry) Outputs the logs in JSON format (useful for telemetry)
@ -323,7 +323,7 @@ Options:
[env: WATERMARK_DELTA=] [env: WATERMARK_DELTA=]
``` ```
## ngrok ## NGROK
```shell ```shell
--ngrok --ngrok
Enable ngrok tunneling Enable ngrok tunneling
@ -347,19 +347,19 @@ Options:
[env: NGROK_EDGE=] [env: NGROK_EDGE=]
``` ```
## env ## ENV
```shell ```shell
-e, --env -e, --env
Display a lot of information about your runtime environment Display a lot of information about your runtime environment
``` ```
## help ## HELP
```shell ```shell
-h, --help -h, --help
Print help (see a summary with '-h') Print help (see a summary with '-h')
``` ```
## version ## VERSION
```shell ```shell
-V, --version -V, --version
Print version Print version

5
update_doc.py
View File

@ -32,6 +32,7 @@ def main():
header = tokens[-1][:-1] header = tokens[-1][:-1]
else: else:
header = line.split("--")[-1] header = line.split("--")[-1]
header = header.upper().replace("-", "_")
block.append(line) block.append(line)
@ -39,10 +40,6 @@ def main():
final_doc += f"## {header}\n```shell\n{rendered_block}\n```\n" final_doc += f"## {header}\n```shell\n{rendered_block}\n```\n"
block = [] block = []
print(final_doc)
filename = "docs/source/basic_tutorials/launcher.md" filename = "docs/source/basic_tutorials/launcher.md"
if args.check: if args.check:
with open(filename, "r") as f: with open(filename, "r") as f: