huggingface/text-generation-inference
5
0
Fork 0
mirror of https://github.com/huggingface/text-generation-inference.git synced 2025-09-12 04:44:52 +00:00
Code Issues Packages Projects Releases Wiki Activity

Launcher tgi

Browse Source
This commit is contained in:
Nicolas Patry 2024-10-02 17:09:45 +02:00
parent 4e120c064b
commit 8e4ba60e6b
No known key found for this signature in database
GPG Key ID: E939E8CC91A1C674
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
.github/workflows/build.yaml vendored
View File

@ -189,7 +189,7 @@ jobs:
make install-integration-tests make install-integration-tests
- name: Run tests - name: Run tests
run: | run: |
docker run busybox sleep infinity docker run --device /dev/kfd --device /dev/dri --env ROCR_VISIBLE_DEVICES -e HUGGING_FACE_HUB_TOKEN=$HF_TOKEN -e ROCM_USE_FLASH_ATTN_V2_TRITON=1 -v /home/nicolas/.cache/huggingface/:/data -p 3000:80 registry.internal.huggingface.tech/api-inference/community/text-generation-inference:sha-318bbc7-rocm --model-id mistralai/Mistral-7B-Instruct-v0.1
# export DOCKER_VOLUME=/mnt2/ # export DOCKER_VOLUME=/mnt2/
# export DOCKER_IMAGE=${{ needs.build-and-push.outputs.docker_image }} # export DOCKER_IMAGE=${{ needs.build-and-push.outputs.docker_image }}
# export DOCKER_DEVICES=${{ needs.build-and-push.outputs.docker_devices }} # export DOCKER_DEVICES=${{ needs.build-and-push.outputs.docker_devices }}