huggingface/text-generation-inference
5
0
Fork 0
mirror of https://github.com/huggingface/text-generation-inference.git synced 2025-10-13 08:55:24 +00:00
Code Issues Packages Projects Releases Wiki Activity
852 Commits 240 Branches 66 Tags 36 MiB
84153702d2
Commit Graph

28 Commits

Author SHA1 Message Date
Morgan Funtowicz
6300bab8b4 make sure executor_worker is provided 2024-07-19 11:57:10 +00:00
Morgan Funtowicz
97723d1458 add logging in case of decoding error 2024-07-18 22:19:25 +00:00
Morgan Funtowicz
9ea7f9e950 remove logging 2024-07-18 22:08:46 +00:00
Morgan Funtowicz
e82dc30e8a expose information about potential error happening while decoding 2024-07-18 22:07:59 +00:00
Morgan Funtowicz
a19d318947 define a shared struct to hold the result of a decoding step 2024-07-18 21:33:04 +00:00
Morgan Funtowicz
a036574a86 add some more validation about grammar not supported 2024-07-18 20:57:29 +00:00
Morgan Funtowicz
b643a436f3 forward tgi parameters rep/freq penalty 2024-07-18 20:56:58 +00:00
Morgan Funtowicz
95847c6587 expose the internal missing start/queue timestamp 2024-07-18 15:57:33 +00:00
Morgan Funtowicz
fd021e5461 refactor Stream impl for Generation to factorise code 2024-07-18 14:21:43 +00:00
Morgan Funtowicz
b56c43ec30 remove unneeded scope variable for now 2024-07-18 12:57:10 +00:00
Morgan Funtowicz
0212b1774a correctly forward back the log probabilities 2024-07-17 22:33:10 +00:00
Morgan Funtowicz
69674a3a2d add all the necessary plumbery to return the generated content 2024-07-17 22:12:49 +00:00
Morgan Funtowicz
ce715c76f8 remove unnecessary log 2024-07-17 22:09:50 +00:00
Morgan Funtowicz
e983ee5bb8 make sure the context is not dropped in the middle of the async decoding. 2024-07-17 21:56:50 +00:00
Morgan Funtowicz
9220340ff7 compute the number of maximum new tokens for each request independently 2024-07-17 13:55:29 +00:00
Morgan Funtowicz
7784a21d48 impl RwLock scenario for TensorRtLllmBackend 2024-07-16 20:08:10 +00:00
Morgan Funtowicz
31d9f4d5dc expose shutdown function at ffi layer 2024-07-15 07:36:01 +00:00
Morgan Funtowicz
b291be64a0 impl the rust backend which currently cannot move the actual computation in background thread 2024-07-12 19:26:32 +00:00
Morgan Funtowicz
344f33f398 end to end ffi flow working 2024-07-12 19:25:40 +00:00
Morgan Funtowicz
50e9fc89c8 working setup of the ffi layer 2024-07-11 21:24:32 +00:00
Morgan Funtowicz
42748d5960 allow converting huggingface::tokenizers error to TensorRtLlmBackendError 2024-07-10 13:56:57 +00:00
Morgan Funtowicz
40fe2ec0ff add auth_token CLI argument to provide hf hub authentification token 2024-07-10 13:50:28 +00:00
Morgan Funtowicz
68a0247a2c unconditionally call InitializeBackend on the FFI layer 2024-07-08 22:09:09 +00:00
Morgan Funtowicz
f57f2a4521 First version loading engines and making it ready for inference 2024-07-03 21:12:24 +00:00
Morgan Funtowicz
f8a1463915 Enable end to end CMake build 2024-07-03 10:27:53 +02:00
Morgan Funtowicz
6dc98abe46 Remove unused parameters annd force tokenizer name to be set 2024-07-01 16:11:59 +02:00
Morgan Funtowicz
47ac5c654d Working FFI call for TGI and TRTLLM backend 2024-07-01 15:53:23 +02:00
Morgan Funtowicz
dc402dc9ac Initial setup for CXX binding to TRTLLM 2024-06-30 23:37:20 +02:00