mirror of
https://github.com/huggingface/text-generation-inference.git
synced 2025-04-19 22:02:06 +00:00
584b4d7a68
* nix: experimental support for building a Docker image Run using something like: ``` docker run \ --device nvidia.com/gpu=all \ -it --rm -p 8080:80 \ -v $PWD/data:/data \ -v $PWD/tmp:/tmp \ tgi-docker:latest \ --model-id <model_id> ``` * Example of building the Docker image using Nix inside Docker * Stream to make the builder image smaller This avoids storing a Docker image tarball in the image. Instead, stream the layers while doing `docker run`. * Don't spam journalctl on Linux * Other dockerfile. --------- Co-authored-by: Nicolas Patry <patry.nicolas@protonmail.com>
8 lines
106 B
Plaintext
8 lines
106 B
Plaintext
aml
|
|
target
|
|
server/transformers
|
|
server/flash-attention
|
|
cmake-build-debug/
|
|
cmake-build-release/
|
|
Dockerfile*
|