8 Commits

Author	SHA1	Message	Date
OlivierDehaene	610bb1f978	feat(benchmark): tui based benchmarking tool (#149)	2023-03-30 15:26:27 +02:00
OlivierDehaene	b49dbf2d88	fix(server): use server tokenizer as gt (#128)	2023-03-16 12:12:26 +01:00
OlivierDehaene	1a2d68250a	feat: support typical sampling (#114) ... closes #112	2023-03-09 11:33:57 +01:00
OlivierDehaene	cd5961b5da	feat: allow local models (#101) ... closes #99	2023-03-06 14:39:36 +01:00
OlivierDehaene	9b8ea6a6c7	feat(server): add logits watermark (#90)	2023-03-02 12:30:41 +01:00
OlivierDehaene	439fcaf810	feat(router): add prometheus metrics scrape endpoint (#71)	2023-02-16 17:18:53 +01:00
OlivierDehaene	9af454142a	feat: add distributed tracing (#62)	2023-02-13 13:02:45 +01:00
OlivierDehaene	7b870e1e18	feat(router): use background task to manage request queue (#52) ... Co-authored-by: Nick Hill <nickhill@us.ibm.com>	2023-02-02 14:59:27 +01:00