huggingface/text-generation-inference
5
0
Fork 0
mirror of https://github.com/huggingface/text-generation-inference.git synced 2025-09-12 12:54:52 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'vb/update-consuming-tgi-docs' of https://github.com/Vaibhavs10/text-generation-inference into vb/update-consuming-tgi-docs

Browse Source
This commit is contained in:
Vaibhav Srivastav 2024-08-14 11:19:03 +02:00
commit a590b2f548
1 changed files with 2 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
docs/source/basic_tutorials/consuming_tgi.md
View File

@ -4,7 +4,7 @@ There are many ways to consume Text Generation Inference (TGI) server in your ap
For more information on the API, consult the OpenAPI documentation of `text-generation-inference` available [here](https://huggingface.github.io/text-generation-inference).
You can make the requests using any tool of your preference, such as curl, Python or TypeScript. For an end-to-end experience, we've open-sourced ChatUI, a chat interface for open-source models.
You can make the requests using any tool of your preference, such as curl, Python, or TypeScript. For an end-to-end experience, we've open-sourced ChatUI, a chat interface for open-access models.
## curl
@ -68,7 +68,7 @@ for message in chat_completion:
### Inference Client
[`huggingface-hub`](https://huggingface.co/docs/huggingface_hub/main/en/index) is a Python library to interact with the Hugging Face Hub, including its endpoints. It provides a high-level class, [`huggingface_hub.InferenceClient`](https://huggingface.co/docs/huggingface_hub/package_reference/inference_client#huggingface_hub.InferenceClient), which makes it easy to make calls to TGI's Messages API. `InferenceClient` also takes care of parameter validation and provides a simple to-use interface.
[`huggingface_hub`](https://huggingface.co/docs/huggingface_hub/main/en/index) is a Python library to interact with the Hugging Face Hub, including its endpoints. It provides a high-level class, [`huggingface_hub.InferenceClient`](https://huggingface.co/docs/huggingface_hub/package_reference/inference_client#huggingface_hub.InferenceClient), which makes it easy to make calls to TGI's Messages API. `InferenceClient` also takes care of parameter validation and provides a simple-to-use interface.
Install `huggingface_hub` package via pip.
@ -87,7 +87,6 @@ You can now use `InferenceClient` the exact same way you would use `OpenAI` clie
base_url="http://localhost:3000/v1/",
)
output = client.chat.completions.create(
model="tgi",
messages=[