huggingface/text-generation-inference
5
0
Fork 0
mirror of https://github.com/huggingface/text-generation-inference.git synced 2025-04-24 08:22:07 +00:00
Code Issues Packages Projects Releases Wiki Activity

Make the Nix-based Docker container work on non-NixOS

Browse Source 
On NixOS, the CUDA driver shim gets mounted on /run/opengl-driver,
where Nix packages expect the shim to be. However, on other
distributions, some FHS paths are mounted. This is a small change
to make the dynamic loader find the shim.
This commit is contained in:
Daniël de Kok 2025-03-13 12:36:56 +00:00
parent 8b91f92978
commit 2158aaa3d9
2 changed files with 10 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

8
flake.nix
View File

@ -176,11 +176,15 @@
''; '';
}; };
dockerImage = pkgs.callPackage nix/docker.nix { # Use plain nixpkgs without overlays for dockerTools. dockerTools
# uses a Python package for computing the layers from the transitive
# closure. However, this needs a lot of rebuilds due to our overlay.
dockerImage = nixpkgs.legacyPackages.${system}.callPackage nix/docker.nix {
text-generation-inference = default; text-generation-inference = default;
}; };
dockerImageStreamed = pkgs.callPackage nix/docker.nix { dockerImageStreamed = nixpkgs.legacyPackages.${system}.callPackage nix/docker.nix {
text-generation-inference = default; text-generation-inference = default;
stream = true; stream = true;
}; };

4
nix/docker.nix
View File

@ -18,6 +18,10 @@ build {
Env = [ Env = [
"HF_HOME=/data" "HF_HOME=/data"
"PORT=80" "PORT=80"
# The CUDA container toolkit will mount the driver shim into the
# container. We just have to ensure that the dynamic loader finds
# the libraries.
"LD_LIBRARY_PATH=/usr/lib64"
]; ];
}; };