19 Commits

Author	SHA1	Message	Date
Morgan Funtowicz	cdac4b0058	chore(looper): cleanup a bit more	2024-10-22 09:52:05 +02:00
Morgan Funtowicz	04c6f51258	feat(trtllm): rewrite health to not account for current state	2024-10-22 09:52:05 +02:00
Morgan Funtowicz	d73401ac73	chore(rebase): fix invalid references	2024-10-21 21:44:28 +02:00
Morgan Funtowicz	8d1c3c8ad4	feat(trtllm): do not tokenize twice	2024-10-21 15:06:54 +02:00
Morgan Funtowicz	1a3da05f34	misc(router): remove SchedulingError	2024-10-21 14:57:19 +02:00
Morgan Funtowicz	6687c06a21	feat(looper): minor optimizations to avoid growing too much the containers	2024-10-21 10:00:27 +02:00
Morgan Funtowicz	629153b44b	feat(looper): check engine and executorWorker paths exist before creating the backend	2024-10-21 10:00:27 +02:00
Morgan Funtowicz	c8a99af6c9	(fix): do not recreate the stateful hashmap at every it	2024-10-21 10:00:27 +02:00
Morgan Funtowicz	213acc6e34	(misc) move to latest trtllm	2024-10-21 10:00:27 +02:00
Morgan Funtowicz	984ae9798f	(post) impl postprocessing	2024-10-21 10:00:27 +02:00
Morgan Funtowicz	fa63db0d07	(scheduler) rework submit/pull logic	2024-10-21 10:00:27 +02:00
Morgan Funtowicz	b1846fb4e6	(backend) refactor & cleanup	2024-10-21 10:00:27 +02:00
Morgan Funtowicz	8e648ce425	(ffi) fix usage of wrong vector constructor making a capacity fill call	2024-10-21 10:00:27 +02:00
Morgan Funtowicz	291eaa99fb	use blocking_recv in looper to consume awaiting_requests at max before pulling in a single step	2024-10-21 10:00:27 +02:00
Morgan Funtowicz	7bebc629af	(misc) missing Result types for Rust	2024-10-21 10:00:27 +02:00
Morgan Funtowicz	c2e21d8725	(backend) implement the post_processor background thread	2024-10-21 10:00:27 +02:00
Morgan Funtowicz	0dca168bcb	(misc) change scope identifiers	2024-10-21 10:00:27 +02:00
Morgan Funtowicz	933ab67aa1	(ffi) encode the provided user prompt within each request thread	2024-10-21 10:00:27 +02:00
Morgan Funtowicz	fb759bdd2a	(looper) new looper initial implementation	2024-10-21 10:00:27 +02:00