huggingface/text-generation-inference
5
0
Fork 0
mirror of https://github.com/huggingface/text-generation-inference.git synced 2025-04-19 22:02:06 +00:00
Code Issues Packages Projects Releases Wiki Activity
2003d8be0c
text-generation-inference/server
History
Daniël de Kok 2003d8be0c
Sync (most) server dependencies with Nix (#2782) 
* Sync (most) server dependencies with Nix

Skipped most grpcio packages, because of protobuf version
incompatibility with the opentelemetry packages.

* Add a primitive script to generate Poetry commands to sync with Nix

This is not fully automated, since getting the Nix versions may be
unresolvable. However, it does take most of the work out of doing
this manually.

* Upgrade eetq ?

* Fmt.

---------

Co-authored-by: Nicolas Patry <patry.nicolas@protonmail.com>
2024-12-03 04:04:06 +01:00
..
custom_kernels All integration tests back everywhere (too many failed CI). (#2428) 2024-08-16 21:19:46 +02:00
exllama_kernels Update ROCM libs and improvements (#2579) 2024-09-30 10:54:32 +02:00
exllamav2_kernels Update ROCM libs and improvements (#2579) 2024-09-30 10:54:32 +02:00
tests feat: prefill chunking (#2600) 2024-10-16 12:49:33 +02:00
text_generation_server Sync (most) server dependencies with Nix (#2782) 2024-12-03 04:04:06 +01:00
.gitignore Impl simple mamba model (#1480) 2024-02-08 10:19:45 +01:00
bounds-from-nix.py Sync (most) server dependencies with Nix (#2782) 2024-12-03 04:04:06 +01:00
Makefile Remove vLLM dependency for CUDA (#2751) 2024-11-17 17:34:50 +01:00
Makefile-awq chore: add pre-commit (#1569) 2024-02-16 11:58:58 +01:00
Makefile-eetq Sync (most) server dependencies with Nix (#2782) 2024-12-03 04:04:06 +01:00
Makefile-exllamav2 Upgrading exl2. (#2415) 2024-08-14 11:58:08 +02:00
Makefile-flash-att Hotfixing make install. (#2008) 2024-06-04 23:34:03 +02:00
Makefile-flash-att-v2 Update ROCM libs and improvements (#2579) 2024-09-30 10:54:32 +02:00
Makefile-flashinfer Prefix test - Different kind of load test to trigger prefix test bugs. (#2490) 2024-09-11 18:10:40 +02:00
Makefile-lorax-punica Enable multiple LoRa adapters (#2010) 2024-06-25 14:46:27 -04:00
Makefile-selective-scan chore: add pre-commit (#1569) 2024-02-16 11:58:58 +01:00
Makefile-vllm Remove vLLM dependency for CUDA (#2751) 2024-11-17 17:34:50 +01:00
poetry.lock Sync (most) server dependencies with Nix (#2782) 2024-12-03 04:04:06 +01:00
pyproject.toml Sync (most) server dependencies with Nix (#2782) 2024-12-03 04:04:06 +01:00
README.md chore: add pre-commit (#1569) 2024-02-16 11:58:58 +01:00
requirements_cuda.txt Sync (most) server dependencies with Nix (#2782) 2024-12-03 04:04:06 +01:00
requirements_intel.txt Sync (most) server dependencies with Nix (#2782) 2024-12-03 04:04:06 +01:00
requirements_rocm.txt Sync (most) server dependencies with Nix (#2782) 2024-12-03 04:04:06 +01:00

README.md

Text Generation Inference Python gRPC Server

A Python gRPC server for Text Generation Inference

Install

make install

Run

make run-dev