From 3a2a13ecd51c829497d2b5498b7aa9c474258e1d Mon Sep 17 00:00:00 2001 From: Merve Noyan Date: Fri, 18 Aug 2023 17:13:31 +0300 Subject: [PATCH] Added diff and dark/light mode for demo --- docs/source/basic_tutorials/consuming_tgi.md | 27 ++++++++++++++++++-- 1 file changed, 25 insertions(+), 2 deletions(-) diff --git a/docs/source/basic_tutorials/consuming_tgi.md b/docs/source/basic_tutorials/consuming_tgi.md index 60b0d203..b69cee2e 100644 --- a/docs/source/basic_tutorials/consuming_tgi.md +++ b/docs/source/basic_tutorials/consuming_tgi.md @@ -125,10 +125,33 @@ The UI looks like this 👇 You can try the demo directly here 👇 - +
+ +
+ -You can disable streaming mode using `return` instead of `yield` in your inference function. +You can disable streaming mode using `return` instead of `yield` in your inference function, like below. + +```diff +def inference(message, history): + partial_message = "" + for token in client.text_generation(message, max_new_tokens=20, stream=True): + partial_message += token +- yield partial_message ++ return partial_message +``` + You can read more about how to customize a `ChatInterface` [here](https://www.gradio.app/guides/creating-a-chatbot-fast). ## API documentation