nix: add default package

The default package wraps the launcher and puts the server/router in the
path.

As a result, TGI can be started using something like:

```
nix run .# -- \
  --model-id hugging-quants/Meta-Llama-3.1-8B-Instruct-AWQ-INT4 \
  --port 8080
```
This commit is contained in:
Daniël de Kok 2024-08-23 06:22:09 +00:00
parent 358ceb67dd
commit dd89e0d24c

View File

@ -99,6 +99,17 @@
''; '';
}; };
}; };
packages.default = pkgs.writeShellApplication {
name = "text-generation-inference";
runtimeInputs = [
server
router
];
text = ''
${launcher}/bin/text-generation-launcher "$@"
'';
};
} }
); );
} }