mirror of
https://github.com/huggingface/text-generation-inference.git
synced 2025-05-02 23:42:06 +00:00
52e48739a5
* Remove vLLM dependency for CUDA This change adds `attention-kernels` as a dependency for paged attention and cache reshaping. With that, we don't use vLLM anywhere for CUDA. Tested run (since we don't have paged attention in CI): ``` ❯ ATTENTION=paged python -m pytest integration-tests -k "llama and awq" --release [...] 5 snapshots passed. ``` * Fix clippy warning |
||
|---|---|---|
| .. | ||
| infer | ||
| config.rs | ||
| kserve.rs | ||
| lib.rs | ||
| logging.rs | ||
| sagemaker.rs | ||
| server.rs | ||
| usage_stats.rs | ||
| validation.rs | ||
| vertex.rs | ||