From 2d8b31a4b4081f54317fe51cb275940abd4645c7 Mon Sep 17 00:00:00 2001 From: osanseviero Date: Thu, 10 Aug 2023 11:14:17 +0200 Subject: [PATCH] Fix gated docs --- .../basic_tutorials/gated_model_access.md | 21 ++++++++++++++++++- 1 file changed, 20 insertions(+), 1 deletion(-) diff --git a/docs/source/basic_tutorials/gated_model_access.md b/docs/source/basic_tutorials/gated_model_access.md index f5858dc4..2df2a8b6 100644 --- a/docs/source/basic_tutorials/gated_model_access.md +++ b/docs/source/basic_tutorials/gated_model_access.md @@ -2,4 +2,23 @@ If the model you wish to serve is behind gated access or the model repository on Hugging Face Hub is private, and you have access to the model, you can provide your Hugging Face Hub access token. You can generate and copy a read token from [Hugging Face Hub tokens page](https://huggingface.co/settings/tokens) -If you're using the CLI, set the `HUGGING_FACE_HUB_TOKEN` environment variable. +If you're using the CLI, set the `HUGGING_FACE_HUB_TOKEN` environment variable. For example: + +``` +export HUGGING_FACE_HUB_TOKEN= +``` + +If you would like to do it through Docker, you can provide your token by specifying `HUGGING_FACE_HUB_TOKEN` as shown below. + +```bash +model=meta-llama/Llama-2-7b-chat-hf +volume=$PWD/data +token= + +docker run --gpus all \ + --shm-size 1g + -e HUGGING_FACE_HUB_TOKEN=$token \ + -p 8080:80 \ + -v $volume:/data ghcr.io/huggingface/text-generation-inference:1.0.0 \ + --model-id $model +``` \ No newline at end of file