mirror of
https://github.com/huggingface/text-generation-inference.git
synced 2025-04-26 12:32:10 +00:00
nix: add default package (#2453)
The default package wraps the launcher and puts the server/router in the path. As a result, TGI can be started using something like: ``` nix run .# -- \ --model-id hugging-quants/Meta-Llama-3.1-8B-Instruct-AWQ-INT4 \ --port 8080 ```
This commit is contained in:
parent
b7d1adc3e9
commit
92ac02e4f2
Loading…
Reference in New Issue
Block a user