Commit Graph

15 Commits

Author SHA1 Message Date
Daniël de Kok
289aa48554
Move JSON grammar -> regex grammar conversion to the router ()
* Move JSON grammar -> regex grammar conversion to the router

This change moves the JSON grammar -> regex grammar conversion to the
router by adding a dependency on the `outlines-core` Rust crate. In
contrast to the Python implementation, the conversions are not LRU-cached
since they seem to be fast enough:

simple schema           time:   [5.8293 µs 5.8307 µs 5.8320 µs]
                        change: [-13.166% -12.884% -12.641%] (p = 0.00 < 0.05)
                        Performance has improved.

complex schema          time:   [14.875 µs 14.881 µs 14.887 µs]
                        change: [-2.1637% -1.9914% -1.7852%] (p = 0.00 < 0.05)
                        Performance has improved.

Using the schemas from:
https://github.com/dottxt-ai/outlines-core/blob/main/benchmarks/bench_json_schema.py
2024-11-25 18:47:34 +01:00
OlivierDehaene
780531ec77
chore: prepare 2.4.1 release ()
* chore: prepare 2.4.1 release

* fix tests

* fmt
2024-11-22 17:26:15 +00:00
drbh
5489406c4a
PR 2634 CI - Fix the tool_choice format for named choice by adapting OpenAIs scheme ()
* add OpenAI like tool_choice for named choice

* add tests

* fix: run linter and bump api docs

* fix: consolidate changes and remove old tool type

* feat: improve, simplify and rename tool choice struct add required support and refactor

* fix: simplify tool choice logic, improve tests, openapi and rust docs

* fix: refactor away prepare_chat_input and improve tool grammar apply control flow

* feat: update docs and add tool choice configuration section

* fix: simplify naming, tool choice default and improve test

* fix: adjust tool choice none logic, add test and small refactors

* fix: add missing snapshot file

* fix: adjust tool choice type in test

* fix: adjust default when json tool choice is

* fix: remove trailing space lint after rebase

* fix: remove mostly mocked unit test

---------

Co-authored-by: Linus Bierhoff <linus.bierhoff@icloud.com>
2024-11-19 13:31:59 -05:00
OlivierDehaene
a6b02da971
chore: prepare 2.4.0 release () 2024-10-25 21:10:49 +00:00
drbh
e36dfaa8de
feat: allow tool calling to respond without a tool ()
* feat: process token stream before returning to client

* fix: expect content in test

* fix: improve comparison via ruff lint

* fix: return event in all cases

* fix: always send event on error, avoid unwraps, refactor and improve tests

* fix: prefer no_tool over notify_error to improve reponse

* fix: adjust chat input test for no_tool

* fix: adjust test expected content

---------

Co-authored-by: System administrator <root@ip-10-90-0-186.ec2.internal>
2024-10-10 09:28:25 -04:00
drbh
3011639ff7
Revert "Unroll notify error into generate response" ()
Revert "Unroll notify error into generate response ()"

This reverts commit d22b0c1fbe.
2024-10-03 17:56:40 -04:00
drbh
d22b0c1fbe
Unroll notify error into generate response ()
* feat: unroll notify_error if no tool is choosen

* fix: expect simple message when no tool is selected

* fix: improve test to avoid notify_error

* fix: improve docs and indicate change in expected response

* fix: adjust linting in test file
2024-10-02 11:34:57 -04:00
OlivierDehaene
2d0a7173d4 v2.0.1 2024-04-18 17:20:36 +02:00
drbh
7276d43495
feat: improve tools to include name and add tests ()
This PR makes tool calling aware of the name of the function selected. 

Fixes:
https://github.com/huggingface/text-generation-inference/issues/1657

Thank you @puppetm4st3r for the helpful snippets, large parts of this PR
are simply refactors of the code shared 🙏

**opening draft PR because small tweaks are needed before merging
2024-04-16 09:02:46 -04:00
OlivierDehaene
c38a7d7ddd
v2.0.0 () 2024-04-12 18:38:34 +02:00
OlivierDehaene
4ee0a0c401
v1.4.5 () 2024-03-29 19:17:24 +01:00
OlivierDehaene
6c4496a1a3
v1.4.4 () 2024-03-22 18:44:05 +01:00
drbh
de6cb15fa5
fix: improve tool type, bump pydantic and outlines ()
This PR resolves a couple 

- [X] adjusts the tool response to align with openai's tools response
type
- [X] bumps pydantic to `2.6.4` in all apps (resolves dependency issue
when running tests)
- [X] bump `outlines` version and fix import for new name
2024-03-21 12:45:56 -04:00
OlivierDehaene
e6bb3ff81f
v1.4.3 () 2024-02-28 16:12:14 +01:00
drbh
9b6db5f793
Support tools ()
This work in progress PR begins to add support for tools. Tools relies
on grammar support and still has some unsolved challenges. Opening the
PR for visibility and feedback
2024-02-28 11:10:27 +01:00