router: send the input as chunks to the backend

Before this change, the generation input was sent to the backend as a
single string, encoding images as Base64 and packing them in
Markdown-style links.

This change adds a new chunked input representation that separates text
chunks from images chunks. Image chunks contain binary data (for smaller
message sizes) and the image's MIME type.

The stringly-typed inputs are still sent to support backends that do not
support chunked inputs yet.
This commit is contained in:
Daniël de Kok 2024-05-30 12:31:35 +00:00
parent fc52ba61ab
commit ebeea9daf8

View File

@ -714,7 +714,7 @@ pub enum ValidationError {
#[cfg(test)]
mod tests {
use super::*;
use crate::config::{Idefics2, LlavaNext, PaliTextConfig, Paligemma, TextConfig, VisionConfig};
use crate::config::{PaliTextConfig, Paligemma};
use crate::default_parameters;
use crate::tests::get_tokenizer;