huggingface/text-generation-inference
5
0
Fork 0
mirror of https://github.com/huggingface/text-generation-inference.git synced 2025-09-12 04:44:52 +00:00
Code Issues Packages Projects Releases Wiki Activity

Fixing docs + causal_lm batch_class.

Browse Source
This commit is contained in:
Nicolas Patry 2024-07-04 12:59:59 +02:00
parent e2edf2beb2
commit 8ecee7283c
No known key found for this signature in database
GPG Key ID: E939E8CC91A1C674
2 changed files with 2 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
docs/source/supported_models.md
View File

@ -10,6 +10,7 @@ Text Generation Inference enables serving optimized models on specific hardware
- [Llama](https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct)
- [Phi 3](https://huggingface.co/microsoft/Phi-3-mini-4k-instruct)
- [Gemma](https://huggingface.co/google/gemma-7b)
- [PaliGemma](https://huggingface.co/google/paligemma-3b-pt-224)
- [Gemma2](https://huggingface.co/google/gemma2-9b)
- [Cohere](https://huggingface.co/CohereForAI/c4ai-command-r-plus)
- [Dbrx](https://huggingface.co/databricks/dbrx-instruct)

2
server/text_generation_server/models/causal_lm.py
View File

@ -642,7 +642,7 @@ class CausalLM(Model):
@property
def batch_type(self) -> Type[CausalLMBatch]:
return self.batch
return self.batch_class
def forward(
self, input_ids, attention_mask, position_ids, past_key_values: Optional = None