huggingface/text-generation-inference
5
0
Fork 0
mirror of https://github.com/huggingface/text-generation-inference.git synced 2025-04-24 08:22:07 +00:00
Code Issues Packages Projects Releases Wiki Activity

FIxing the CI.

Browse Source
This commit is contained in:
Nicolas Patry 2024-08-16 14:21:29 +02:00 committed by yuanwu
parent 85df9fc2db
commit 11d25a4bd3
2 changed files with 4 additions and 4 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
docs/openapi.json
View File

@ -2094,4 +2094,4 @@
"description": "Hugging Face Text Generation Inference API" "description": "Hugging Face Text Generation Inference API"
} }
] ]
} }

6
docs/source/basic_tutorials/consuming_tgi.md
View File

@ -88,7 +88,7 @@ There is also an async version of the client, `AsyncInferenceClient`, based on `
You can directly use the OpenAI [Python](https://github.com/openai/openai-python) or [JS](https://github.com/openai/openai-node) clients to interact with TGI. You can directly use the OpenAI [Python](https://github.com/openai/openai-python) or [JS](https://github.com/openai/openai-node) clients to interact with TGI.
Install the OpenAI Python package via pip. Install the OpenAI Python package via pip.
```bash ```bash
pip install openai pip install openai
@ -145,7 +145,7 @@ def inference(message, history):
stream=True, stream=True,
max_tokens=1024, max_tokens=1024,
) )
for chunk in output: for chunk in output:
partial_message += chunk.choices[0].delta.content partial_message += chunk.choices[0].delta.content
yield partial_message yield partial_message
@ -196,4 +196,4 @@ To serve both ChatUI and TGI in same environment, simply add your own endpoints
} }
``` ```
![ChatUI](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/chatui_screen.png) ![ChatUI](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/chatui_screen.png)