mirror of
https://github.com/huggingface/text-generation-inference.git
synced 2025-04-21 23:12:07 +00:00
56670398f3
This PR correctly handles batches with a mixture of constrained and non constrained generations. Currently if batch contains mixed generations the generation will throw an error because it will incorrectly attempt to constrain a request with an empty grammar. We now handled `None` grammars and only apply the mask if needed Fixes: https://github.com/huggingface/text-generation-inference/issues/1643 |
||
|---|---|---|
| .. | ||
| models | ||
| pb | ||
| utils | ||
| __init__.py | ||
| cache.py | ||
| cli.py | ||
| habana_quantization_env.py | ||
| interceptor.py | ||
| server.py | ||
| tgi_service.py | ||
| tracing.py | ||