huggingface/text-generation-inference
5
0
Fork 0
mirror of https://github.com/huggingface/text-generation-inference.git synced 2025-10-15 09:55:23 +00:00
Code Issues Packages Projects Releases Wiki Activity
843 Commits 240 Branches 66 Tags 36 MiB
95847c6587
Commit Graph

12 Commits

Author SHA1 Message Date
Morgan Funtowicz
0212b1774a correctly forward back the log probabilities 2024-07-17 22:33:10 +00:00
Morgan Funtowicz
e983ee5bb8 make sure the context is not dropped in the middle of the async decoding. 2024-07-17 21:56:50 +00:00
Morgan Funtowicz
9220340ff7 compute the number of maximum new tokens for each request independently 2024-07-17 13:55:29 +00:00
Morgan Funtowicz
7784a21d48 impl RwLock scenario for TensorRtLllmBackend 2024-07-16 20:08:10 +00:00
Morgan Funtowicz
31d9f4d5dc expose shutdown function at ffi layer 2024-07-15 07:36:01 +00:00
Morgan Funtowicz
344f33f398 end to end ffi flow working 2024-07-12 19:25:40 +00:00
Morgan Funtowicz
50e9fc89c8 working setup of the ffi layer 2024-07-11 21:24:32 +00:00
Morgan Funtowicz
68a0247a2c unconditionally call InitializeBackend on the FFI layer 2024-07-08 22:09:09 +00:00
Morgan Funtowicz
f57f2a4521 First version loading engines and making it ready for inference 2024-07-03 21:12:24 +00:00
Morgan Funtowicz
f8a1463915 Enable end to end CMake build 2024-07-03 10:27:53 +02:00
Morgan Funtowicz
47ac5c654d Working FFI call for TGI and TRTLLM backend 2024-07-01 15:53:23 +02:00
Morgan Funtowicz
dc402dc9ac Initial setup for CXX binding to TRTLLM 2024-06-30 23:37:20 +02:00