From b226e469c95d5abe4a67ca0aff758b49338654b0 Mon Sep 17 00:00:00 2001 From: Nicolas Patry Date: Tue, 21 Nov 2023 10:39:18 +0100 Subject: [PATCH 1/2] Update README.md (#1272) --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 4fcef3dc..e4f90d53 100644 --- a/README.md +++ b/README.md @@ -59,7 +59,7 @@ Text Generation Inference (TGI) is a toolkit for deploying and serving Large Lan For a detailed starting guide, please see the [Quick Tour](https://huggingface.co/docs/text-generation-inference/quicktour). The easiest way of getting started is using the official Docker container: ```shell -model=tiiuae/falcon-7b-instruct +model=HuggingFaceH4/zephyr-7b-beta volume=$PWD/data # share a volume with the Docker container to avoid downloading weights every run docker run --gpus all --shm-size 1g -p 8080:80 -v $volume:/data ghcr.io/huggingface/text-generation-inference:1.1.1 --model-id $model From 91111a0dc2936741631bdd3c2be8fd354ac4c0e3 Mon Sep 17 00:00:00 2001 From: Diwank Singh Tomer Date: Thu, 23 Nov 2023 03:41:05 -0800 Subject: [PATCH 2/2] Fix missing `trust_remote_code` flag for AutoTokenizer in utils.peft (#1270) Peft loading function was missing the `trust_remote_code=trust_remote_code` argument causing the custom tokenizer code to be not found. ## Before submitting - [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case). - [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests), Pull Request section? - [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link to it if that's the case. - [ ] Did you make sure to update the documentation with your changes? Here are the [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation). - [ ] Did you write any new necessary tests? ## Who can review? Anyone in the community is free to review the PR once the tests have passed. Feel free to tag members/contributors who may be interested in your PR. @Narsil --- server/text_generation_server/utils/peft.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/server/text_generation_server/utils/peft.py b/server/text_generation_server/utils/peft.py index e37447dc..d37e8940 100644 --- a/server/text_generation_server/utils/peft.py +++ b/server/text_generation_server/utils/peft.py @@ -38,7 +38,7 @@ def download_and_unload_peft(model_id, revision, trust_remote_code): os.makedirs(model_id, exist_ok=True) cache_dir = model_id logger.info(f"Saving the newly created merged model to {cache_dir}") - tokenizer = AutoTokenizer.from_pretrained(base_model_id) + tokenizer = AutoTokenizer.from_pretrained(base_model_id, trust_remote_code=trust_remote_code) model.save_pretrained(cache_dir, safe_serialization=True) model.config.save_pretrained(cache_dir) tokenizer.save_pretrained(cache_dir)