mirror of
https://github.com/huggingface/text-generation-inference.git
synced 2025-04-21 23:12:07 +00:00
a9c7d2e3b6
* Basic flashinfer 0.2 support This change does not use any of the new features yet, but makes some small compatibility changes. * Update to flashinfer 0.2.0.post1 * flashinfer: remove `contiguous` calls * Fix flashinfer install * flashinfer: fixup kv cache dtype * Fix some annoying perturbations * More output changes
6 lines
219 B
Plaintext
6 lines
219 B
Plaintext
install-flashinfer:
|
|
# We need fsspec as an additional dependency, but
|
|
# `pip install flashinfer` cannot resolve it.
|
|
pip install fsspec
|
|
pip install flashinfer==0.2.0.post1 -i https://flashinfer.ai/whl/cu124/torch2.4
|