12 Commits

Author	SHA1	Message	Date
OlivierDehaene	1539d3cbbe	feat(router): Remove second lock from batcher hot path (#27) ... @njhill	2023-01-26 16:29:13 +01:00
Nick Hill	60472f9d2b	feat(router): Add const parameters to validation logic (#15) ... I noticed some opportunity to collapse some of the logic, in case you are interested.	2023-01-03 10:41:22 +01:00
OlivierDehaene	32a253063d	feat: Return logprobs (#8)	2022-12-15 17:03:56 +01:00
OlivierDehaene	718096f695	feat: Support stop sequences (#7)	2022-12-12 18:25:22 +01:00
OlivierDehaene	427d7cc444	feat(server): Support AutoModelForSeq2SeqLM	2022-11-04 18:03:04 +01:00
OlivierDehaene	3cf6368c77	feat(server): Support all AutoModelForCausalLM on a best effort basis	2022-10-28 19:24:00 +02:00
OlivierDehaene	c837893370	feat(router): Add max_waiting_tokens	2022-10-21 16:40:05 +02:00
Olivier Dehaene	f16f2f5ae1	v0.1.0	2022-10-20 19:14:44 +02:00
Olivier Dehaene	5e5d8766a2	feat: Improve error handling	2022-10-17 14:59:00 +02:00
Olivier Dehaene	4c693e6524	Refactored gRPC interface ... Added validation logic	2022-10-11 16:50:54 +02:00
Olivier Dehaene	fa9a088467	Add load testing	2022-10-11 10:36:51 +02:00
Olivier Dehaene	295831a481	Init	2022-10-08 12:30:12 +02:00