huggingface/text-generation-inference
5
0
Fork 0
mirror of https://github.com/huggingface/text-generation-inference.git synced 2025-05-24 12:32:11 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,399 Commits 234 Branches 61 Tags 29 MiB
ee0fab4916
Commit Graph

2 Commits

Author SHA1 Message Date
Daniël de Kok
e32528792c
Switch to punica-sgmv kernel from the Hub (#3236) 
* Switch to punica-sgmv kernel from the Hub

This also switches (temporarily) to the tgi-nix/kernel-builder merge
branch, bumping up to CUDA 12.8 (same as non-Nix Torch).

* nix: client depends on aiohttp

This probably worked before the nixpkgs bump because a dependency
propagated aiohttp.
 2025-05-21 15:44:15 +02:00
Nicolas Patry
f512021e77
Stream options. (#2533) 
* Stream options.

* Fetch stuff from nix integration test for easier testing.

* Adding the assert.

* Only send the usage when asked for.

* Update the docs.

* Impure test because we need network.

* develop.

* Optional usage.

* Fixes.

* Workflow
 2024-09-19 20:50:37 +02:00