text-generation-inference/docs/source/basic_tutorials
drbh 55acb86f42 Outlines guided generation ()
This WIP PR starts to add grammar support via outlines, currently this
PR supports very simple regex grammars and does not optimize for
precompiling or caching grammar fsm's.

todo:
- [X] add simple outlines guidance to `NextTokenChooser`
- [X] update protos for grammar
- [X] update generation params API
- [X] constrain simple grammar
- [ ] support parsing more complex grammar into fsm
- [ ] support all outline support grammar types
- [ ] explore optimizations to avoid recompiling grammars

guided request
```bash
curl -s 'http://localhost:3000/generate' \
--header 'Content-Type: application/json' \
--data-raw '{
    "inputs": "make an email for david: \n",
    "parameters": {
        "max_new_tokens": 6,
        "grammar": "[\\w-]+@([\\w-]+\\.)+[\\w-]+"
    }
}' | jq
```
response
```json
{
  "generated_text": "david@example.com"
}
```

unguided request
```bash
curl -s 'http://localhost:3000/generate' \
--header 'Content-Type: application/json' \
--data '{
    "inputs": "make an email for david: \n",
    "parameters": {
        "max_new_tokens": 6
    }
}' | jq
```
response
```json
{
  "generated_text": "    email = 'david"
}
```
2024-04-24 14:57:37 +03:00
..
consuming_tgi.md Added gradio example to docs () 2023-08-23 23:50:12 +02:00
gated_model_access.md v1.4.0 () 2024-04-22 15:47:42 +03:00
launcher.md Outlines guided generation () 2024-04-24 14:57:37 +03:00
non_core_models.md Unsupported model serving docs () 2023-09-12 15:55:14 +02:00
preparing_model.md Fix link in quantization guide () 2023-11-08 17:34:38 +01:00
using_cli.md [docs] Fix link to Install CLI () 2024-04-23 09:24:43 +03:00