mirror of
https://github.com/huggingface/text-generation-inference.git
synced 2025-04-27 04:52:07 +00:00
571ac9b507
* Use Hub kernels for Marlin and cutlass quantization kernels * Use hub kernels for MoE/GPTQ-Marlin MoE * Use attention kernels from the Hub * Cache the kernels in the Docker image * Update moe kernels * Support loading local kernels for development * Support latest moe kernels * Update to moe 0.1.1 * CI: download locked kernels for server tests * Fixup some imports * CI: activate venv * Fix unused imports * Nix: add attention/moe/quantization kernels * Update hf-kernels to 0.1.5 * Update kernels * Update tgi-nix flake for hf-kernels * Fix EOF * Take `load_kernel` out of a frequently-called function * Hoist another case of kernel loading out of a somewhat hot function * marlin-kernels -> quantization * attention -> paged-attention * EOF fix * Update hf-kernels, fixup Docker * ipex fix * Remove outdated TODO |
||
|---|---|---|
| .. | ||
| autodocs.yaml | ||
| build_documentation.yaml | ||
| build_pr_documentation.yaml | ||
| build.yaml | ||
| ci_build.yaml | ||
| client-tests.yaml | ||
| integration_tests.yaml | ||
| load_test.yaml | ||
| nix_cache.yaml | ||
| nix_tests.yaml | ||
| stale.yaml | ||
| tests.yaml | ||
| trufflehog.yaml | ||
| upload_pr_documentation.yaml | ||