From 0af0315b781764a8794b54a3c4cca68bbeebd00c Mon Sep 17 00:00:00 2001 From: Merve Noyan Date: Wed, 23 Aug 2023 15:45:24 +0300 Subject: [PATCH] Update docs/source/conceptual/tensor_parallelism.md Co-authored-by: Omar Sanseviero --- docs/source/conceptual/tensor_parallelism.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/conceptual/tensor_parallelism.md b/docs/source/conceptual/tensor_parallelism.md index 9aceeb7c..278c0832 100644 --- a/docs/source/conceptual/tensor_parallelism.md +++ b/docs/source/conceptual/tensor_parallelism.md @@ -4,7 +4,7 @@ Tensor parallelism is a technique used to fit a large model in multiple GPUs. I ![Image courtesy of Anton Lozkhov](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/tgi/TP.png) -In TGI, tensor parallelism is implemented under the hood by sharding weights and placing them in different ranks. The matrix multiplications then take place in different ranks and are then gathered into a single tensor. +In TGI, tensor parallelism is implemented under the hood by sharding weights and placing them in different GPUs. The matrix multiplications then take place in different GPUs and are then gathered into a single tensor.