mirror of
https://github.com/huggingface/text-generation-inference.git
synced 2025-04-26 20:42:06 +00:00
fca2218fa9
Fix runtime error when Qwen2-VL model is prompted with prompt with more
than one image. The runtime error was:
File "text-generation-inference/server/text_generation_server/models/custom_modeling/qwen2_vl.py", line 459, in get_position_ids
text_pos_ids = torch.arange(text_length, device=d)
RuntimeError: upper bound and larger bound inconsistent with step sign
The error was caused by text_length variable going to negative value
when multiple images caused multiple loops in the get_position_ids
function's main loop.
The error is a simple logic mistake where next_image_pos is initialized
as relative offset from current_pos, but was used like it was absolute
position from zero.
|
||
|---|---|---|
| .. | ||
| custom_modeling | ||
| __init__.py | ||
| bloom.py | ||
| causal_lm.py | ||
| flash_causal_lm.py | ||
| galactica.py | ||
| globals.py | ||
| idefics_causal_lm.py | ||
| mamba.py | ||
| metadata_kernels.py | ||
| mllama_causal_lm.py | ||
| model.py | ||
| pali_gemma.py | ||
| seq2seq_lm.py | ||
| types.py | ||
| vlm_causal_lm.py | ||