huggingface/text-generation-inference
5
0
Fork 0
mirror of https://github.com/huggingface/text-generation-inference.git synced 2025-09-10 20:04:52 +00:00
Code Issues Packages Projects Releases Wiki Activity

Update consuming_tgi.md

Browse Source
This commit is contained in:
Merve Noyan 2023-08-18 17:14:52 +03:00 committed by GitHub
parent 3a2a13ecd5
commit 8f1d266e69
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 2 additions and 6 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

8
docs/source/basic_tutorials/consuming_tgi.md
View File

@ -143,13 +143,9 @@ You can try the demo directly here 👇
You can disable streaming mode using `return` instead of `yield` in your inference function, like below. You can disable streaming mode using `return` instead of `yield` in your inference function, like below.
```diff ```python
def inference(message, history): def inference(message, history):
partial_message = "" return client.text_generation(message, max_new_tokens=20)
for token in client.text_generation(message, max_new_tokens=20, stream=True):
partial_message += token
- yield partial_message
+ return partial_message
``` ```
You can read more about how to customize a `ChatInterface` [here](https://www.gradio.app/guides/creating-a-chatbot-fast). You can read more about how to customize a `ChatInterface` [here](https://www.gradio.app/guides/creating-a-chatbot-fast).