huggingface/text-generation-inference
5
0
Fork 0
mirror of https://github.com/huggingface/text-generation-inference.git synced 2025-09-12 04:44:52 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix(server): fix cohere

Browse Source
This commit is contained in:
OlivierDehaene 2024-07-18 15:59:36 +02:00
parent da82c63a4f
commit e952e4cfd3
No known key found for this signature in database
GPG Key ID: BB104D67809DA93C
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
server/text_generation_server/models/custom_modeling/flash_cohere_modeling.py
View File

@ -259,8 +259,8 @@ class FlashCohereAttention(torch.nn.Module):
cu_seqlen_prefill,
kv_cache,
block_tables,
input_lengths,
slots,
input_lengths,
max_s,
):
qkv = self.query_key_value(hidden_states)