huggingface/text-generation-inference
5
0
Fork 0
mirror of https://github.com/huggingface/text-generation-inference.git synced 2025-04-24 16:32:12 +00:00
Code Issues Packages Projects Releases Wiki Activity

Make the Nix-based Docker container work on non-NixOS

Browse Source 
On NixOS, the CUDA driver shim gets mounted on /run/opengl-driver,
where Nix packages expect the shim to be. However, on other
distributions, some FHS paths are mounted. This is a small change
to make the dynamic loader find the shim.
This commit is contained in:
Daniël de Kok 2025-03-13 12:36:56 +00:00
parent 8b91f92978
commit 2158aaa3d9
2 changed files with 10 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

8
flake.nix
View File

@ -176,11 +176,15 @@
'';
};
dockerImage = pkgs.callPackage nix/docker.nix {
# Use plain nixpkgs without overlays for dockerTools. dockerTools
# uses a Python package for computing the layers from the transitive
# closure. However, this needs a lot of rebuilds due to our overlay.
dockerImage = nixpkgs.legacyPackages.${system}.callPackage nix/docker.nix {
text-generation-inference = default;
};
dockerImageStreamed = pkgs.callPackage nix/docker.nix {
dockerImageStreamed = nixpkgs.legacyPackages.${system}.callPackage nix/docker.nix {
text-generation-inference = default;
stream = true;
};

4
nix/docker.nix
View File

@ -18,6 +18,10 @@ build {
Env = [
"HF_HOME=/data"
"PORT=80"
# The CUDA container toolkit will mount the driver shim into the
# container. We just have to ensure that the dynamic loader finds
# the libraries.
"LD_LIBRARY_PATH=/usr/lib64"
];
};