huggingface/text-generation-inference
5
0
Fork 0
mirror of https://github.com/huggingface/text-generation-inference.git synced 2025-09-11 12:24:53 +00:00
Code Issues Packages Projects Releases Wiki Activity

Update guidance.md to reflect grammar support

Browse Source 
https://github.com/huggingface/text-generation-inference/blob/main/router/src/server.rs#L654 confirms that support for grammar is TGI only at this time.
This commit is contained in:
dr3s 2024-04-18 16:58:07 -04:00 committed by GitHub
parent 2d0a7173d4
commit 2aa7e073bc
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 3 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
docs/source/conceptual/guidance.md
View File

@ -2,7 +2,9 @@
Text Generation Inference (TGI) now supports [JSON and regex grammars](#grammar-and-constraints) and [tools and functions](#tools-and-functions) to help developer guide LLM responses to fit their needs.
These feature are available starting from version `1.4.3`. They are accessible via the [text_generation](https://pypi.org/project/text-generation/) library and is compatible with OpenAI's client libraries. The following guide will walk you through the new features and how to use them!
These feature are available starting from version `1.4.3`. They are accessible via the [text_generation](https://pypi.org/project/text-generation/) library. The tool support is compatible with OpenAI's client libraries. The following guide will walk you through the new features and how to use them!
> The Grammar guidance support is currently only available in the TGI API due to lack of support in Open AI API.
## Quick Start