From ee19513cf71e36626468f4388c37acbed3ea0002 Mon Sep 17 00:00:00 2001 From: Merve Noyan Date: Tue, 22 Aug 2023 23:02:17 +0300 Subject: [PATCH] Initial commit --- docs/source/_toctree.yml | 2 ++ docs/source/conceptual/safetensors.md | 7 +++++++ 2 files changed, 9 insertions(+) create mode 100644 docs/source/conceptual/safetensors.md diff --git a/docs/source/_toctree.yml b/docs/source/_toctree.yml index 5ba470bd..e26695e4 100644 --- a/docs/source/_toctree.yml +++ b/docs/source/_toctree.yml @@ -21,4 +21,6 @@ - sections: - local: conceptual/streaming title: Streaming + - local: conceptual/safetensors + title: Safetensors title: Conceptual Guides diff --git a/docs/source/conceptual/safetensors.md b/docs/source/conceptual/safetensors.md new file mode 100644 index 00000000..fcc31bac --- /dev/null +++ b/docs/source/conceptual/safetensors.md @@ -0,0 +1,7 @@ +# Safetensors + +Safetensors is a model serialization format for deep learning models. It is [faster](https://huggingface.co/docs/safetensors/speed) and safer compared to other serialization formats like pickle (which is used under the hood in many deep learning libraries). + +TGI depends on safetensors format mainly to enable [tensor parallelism sharding](./tensor_parallelism). For a given model repository during serving, TGI looks for safetensors weights. If there are no safetensors weights, TGI converts the PyTorch weights to safetensors format. + +You can learn more about safetensors by reading the [safetensors documentation](https://huggingface.co/docs/safetensors/index). \ No newline at end of file