drbh
|
da5ab46705
|
Improve vlm support (add idefics3 support) (#2437)
* feat: expand vlm support and add image token logic and tests
* fix: avoid unused perceiver config
* feat: integrate image tokens into inputs embeds
* feat: add simple idefics3 test
* feat: update docs, image token logic and weight names
* fix: improve image processing
* feat: improve prefix for idefics3
* fix: bump idefics3 tests and snapshots
* fix: improve text model loading
* feat: consolidate changes with existing vlms and add support and test for smolvlm
* fix: create new idefic3 file, simplify logic and adjust llama weight loading
* fix: lint with ruff
* fix: clean up idefics 3 and improve prefix handling
* fix: improve typing
* fix: improve prompt_split_image with ref to original impl
* fix: adjust ruff lints and small refactors
* fix: adjust FlashLlamaModel prefix logic
|
2025-01-09 10:35:32 -05:00 |
|