From 2aa7e073bc182f720e70f148b06b9f2e79d23ef8 Mon Sep 17 00:00:00 2001 From: dr3s Date: Thu, 18 Apr 2024 16:58:07 -0400 Subject: [PATCH] Update guidance.md to reflect grammar support https://github.com/huggingface/text-generation-inference/blob/main/router/src/server.rs#L654 confirms that support for grammar is TGI only at this time. --- docs/source/conceptual/guidance.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/source/conceptual/guidance.md b/docs/source/conceptual/guidance.md index 0a3bbd60..da289652 100644 --- a/docs/source/conceptual/guidance.md +++ b/docs/source/conceptual/guidance.md @@ -2,7 +2,9 @@ Text Generation Inference (TGI) now supports [JSON and regex grammars](#grammar-and-constraints) and [tools and functions](#tools-and-functions) to help developer guide LLM responses to fit their needs. -These feature are available starting from version `1.4.3`. They are accessible via the [text_generation](https://pypi.org/project/text-generation/) library and is compatible with OpenAI's client libraries. The following guide will walk you through the new features and how to use them! +These feature are available starting from version `1.4.3`. They are accessible via the [text_generation](https://pypi.org/project/text-generation/) library. The tool support is compatible with OpenAI's client libraries. The following guide will walk you through the new features and how to use them! + +> The Grammar guidance support is currently only available in the TGI API due to lack of support in Open AI API. ## Quick Start