huggingface/text-generation-inference
5
0
Fork 0
mirror of https://github.com/huggingface/text-generation-inference.git synced 2025-04-24 00:12:08 +00:00
Code Issues Packages Projects Releases Wiki Activity
fb939370a3
text-generation-inference/server/text_generation_server
History
Daniël de Kok fb939370a3 Support different image sizes in prefill in VLMs (#2065) 
When a batch contained images if different sizes during prefill, the
server would fail (see e.g. #2056). Images were processed separately and
then concatenated. However, this can fail for images with different sizes.

Fix this by preprocessing all images in the batch together, so that the
image processor can ensure that all image tensors have compatible sizes.
2024-09-24 03:43:31 +00:00
..
layers Add support for GPTQ Marlin (#2052) 2024-09-24 03:43:30 +00:00
models Support different image sizes in prefill in VLMs (#2065) 2024-09-24 03:43:31 +00:00
pb chore: add pre-commit (#1569) 2024-04-24 15:32:02 +03:00
utils Add support for GPTQ Marlin (#2052) 2024-09-24 03:43:30 +00:00
__init__.py feat(clients): Python client (#103) 2023-03-07 18:52:22 +01:00
cache.py fix(server): decrease memory fragmentation (#557) 2023-07-06 14:28:33 +02:00
cli.py ROCm and sliding windows fixes (#2033) 2024-09-24 03:42:29 +00:00
interceptor.py Aligin the source code with main branch 2.0.4 2024-09-24 03:06:55 +00:00
server.py ROCm and sliding windows fixes (#2033) 2024-09-24 03:42:29 +00:00
tracing.py feat(clients): Python client (#103) 2023-03-07 18:52:22 +01:00