text-generation-inference/router/src/infer/v3/mod.rs

mod block_allocator;
mod queue;
mod scheduler;

pub(crate) use scheduler::SchedulerV3;
feat: move allocation logic to rust (#1835) Close #2007 2024-06-05 10:18:38 +00:00			`mod block_allocator;`
feat: add SchedulerV3 (#1996) - Refactor code to allow supporting multiple versions of the generate.proto at the same time - Add v3/generate.proto (ISO to generate.proto for now but allow for future changes without impacting v2 backends) - Add Schedule trait to abstract queuing and batching mechanisms that will be different in the future - Add SchedulerV2/V3 impl 2024-06-04 13:56:56 +00:00			`mod queue;`
			`mod scheduler;`

			`pub(crate) use scheduler::SchedulerV3;`