From 6808b2de7ec75a4cf29ff70b04c20b7d8ddcc70b Mon Sep 17 00:00:00 2001 From: Ikram Ul Haq <44068298+ulhaqi12@users.noreply.github.com> Date: Mon, 30 Sep 2024 09:56:20 +0300 Subject: [PATCH] Update architecture.md (#2577) --- docs/source/architecture.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/architecture.md b/docs/source/architecture.md index 28c84f624..6660630d0 100644 --- a/docs/source/architecture.md +++ b/docs/source/architecture.md @@ -10,7 +10,7 @@ This diagram shows well there are these separate components: - **The router**, also named `webserver`, that receives the client requests, buffers them, creates some batches, and prepares gRPC calls to a model server. - **The model server**, responsible of receiving the gRPC requests and to process the inference on the model. If the model is sharded across multiple accelerators (e.g.: multiple GPUs), the model server shards might be synchronized via NCCL or equivalent. -- **The launcher** is a helper thar will be able to launch one or several model servers (if model is sharded), and it launches the router with the compatible arguments. +- **The launcher** is a helper that will be able to launch one or several model servers (if model is sharded), and it launches the router with the compatible arguments. The router and the model server can be two different machines, they do not need to be deployed together.