text-generation-inference/router/src/infer
drbh 6489f85269
feat: return streaming errors as an event formatted for openai's client (#2668)
* feat: return streaming errors as an event formatted for openai's client

* fix: propagate completions error events to stream

* fix: improve stream api error format and add status code

* fix: improve streamin error to include error_type

* Revert "fix: improve streamin error to include error_type"

This reverts commit 2b1a360b1511d94ea9a24e5432e498e67939506a.

* Reworked the implementation.

* Revert "Reworked the implementation."

This reverts commit 7c3f29777f17411ae4ade57e2f88e73cde704ee5.

* Small lifting.

---------

Co-authored-by: Nicolas Patry <patry.nicolas@protonmail.com>
2024-11-15 14:49:19 +01:00
..
chat_template.rs Mllama flash version (#2585) 2024-10-02 11:22:13 +02:00
mod.rs feat: return streaming errors as an event formatted for openai's client (#2668) 2024-11-15 14:49:19 +01:00
tool_grammar.rs feat: allow tool calling to respond without a tool (#2614) 2024-10-10 09:28:25 -04:00