mirror of https://github.com/huggingface/text-generation-inference.git synced 2025-04-22 15:32:08 +00:00

History

Nicolas Patry 1da07e85aa feat(server): Add Non flash MPT. (#514 ) # What does this PR do? This adds a non flash version of MPT. Flash is harder because we need to create a bias ready cuda kernel of flash attention. Fixes https://github.com/huggingface/text-generation-inference/issues/361 Fixes https://github.com/huggingface/text-generation-inference/issues/491 Fixes https://github.com/huggingface/text-generation-inference/issues/290		2023-07-03 13:01:46 +02:00
..
custom_kernels	feat(server): Rework model loading (#344 )	2023-06-08 14:51:52 +02:00
tests	feat(server): Rework model loading (#344 )	2023-06-08 14:51:52 +02:00
text_generation_server	feat(server): Add Non flash MPT. (#514 )	2023-07-03 13:01:46 +02:00
.gitignore	feat(clients): Python client (#103 )	2023-03-07 18:52:22 +01:00
Makefile	feat(server): Rework model loading (#344 )	2023-06-08 14:51:52 +02:00
Makefile-flash-att	feat(server): pre-allocate past key values for flash causal LM (#412 )	2023-06-12 18:30:29 +02:00
Makefile-vllm	feat(server): add paged attention to flash models (#516 )	2023-06-30 19:09:59 +02:00
poetry.lock	feat(server): Add Non flash MPT. (#514 )	2023-07-03 13:01:46 +02:00
pyproject.toml	feat(server): Add Non flash MPT. (#514 )	2023-07-03 13:01:46 +02:00
README.md	feat(router): refactor API and add openAPI schemas (#53 )	2023-02-03 12:43:37 +01:00
requirements.txt	feat(server): Add Non flash MPT. (#514 )	2023-07-03 13:01:46 +02:00

Text Generation Inference Python gRPC Server

A Python gRPC server for Text Generation Inference

Install

make install

make run-dev