From e4fa84ba2679fd80ee5912d77a820074407da3d6 Mon Sep 17 00:00:00 2001 From: drbh Date: Tue, 23 Jan 2024 17:20:28 +0000 Subject: [PATCH] fix: prefer message api naming --- docs/source/openai_compatible.md | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/docs/source/openai_compatible.md b/docs/source/openai_compatible.md index 0a22f852..895ca6b8 100644 --- a/docs/source/openai_compatible.md +++ b/docs/source/openai_compatible.md @@ -1,10 +1,12 @@ -# Open AI Compatible +# Messages API -Text Generation Inference (TGI) is compatible with OpenAI's API. This means you can use OpenAI's client libraries to interact with TGI. Here are some examples of how to do this. +_Messages API is compatible to OpenAI Chat Completion API_ + +Text Generation Inference (TGI) now supports the Message API which is fully compatible with the OpenAI Chat Completion API. This means you can use OpenAI's client libraries to interact with TGI's Messages API. Below are some examples of how to utilize this compatibility. ## Making a Request -You can make a request to TGI using `curl`. Here's an example: +You can make a request to TGI's Messages API using `curl`. Here's an example: ```bash curl localhost:3000/v1/chat/completions \ @@ -87,7 +89,7 @@ TGI can be deployed on various cloud providers for scalable and robust text gene Amazon SageMaker allows two routes: `/invocations` and `/ping` (or `/health`) for health checks. By default, we map `/generate` to `/invocations`. However, SageMaker does not allow requests to any other routes. -To provide the new feature of chat completion API, we have introduced an environment variable `OAI_ENABLED`. If `OAI_ENABLED=true`, the `chat_completions` method is used when `/invocations` is called, otherwise it defaults to `generate`. This allows users to opt in for the OAI format. +To provide the new feature of Messages API, we have introduced an environment variable `OAI_ENABLED`. If `OAI_ENABLED=true`, the `chat_completions` method is used when `/invocations` is called, otherwise it defaults to `generate`. This allows users to opt in for the OAI format. Here's an example of running the router with `OAI_ENABLED` set to `true`: