17 Commits

Author	SHA1	Message	Date
Morgan Funtowicz	02cd6fe427	chore(backend): minor improvements	2024-11-14 08:42:01 +01:00
Morgan Funtowicz	363d5e45de	feat(backend): use std::ranges to map uint32_t to llama_token	2024-11-14 08:42:01 +01:00
Morgan Funtowicz	6915fa3441	feat(backend): remove reinterpret_cast converting from uint32_t to llama_token(int32_t)	2024-11-14 08:42:01 +01:00
Morgan Funtowicz	86d30aea43	feat(backend): simplify overall cpp structure	2024-11-14 08:42:01 +01:00
Morgan Funtowicz	a1154b17ec	feat(backend): avoid copy constructor	2024-11-14 08:42:01 +01:00
Morgan Funtowicz	588421833c	misc(backend): missing header <variant>	2024-11-14 08:42:01 +01:00
Morgan Funtowicz	1473259f84	feat(backend): add early stopping criteria from TGI stream callback	2024-11-14 08:42:01 +01:00
Morgan Funtowicz	5b7a951389	feat(backend): refactor the callback to handle intermediate and end inference message	2024-11-14 08:42:01 +01:00
Morgan Funtowicz	05ff551950	feat(backend): add number of generated tokens in the callback	2024-11-14 08:42:01 +01:00
Morgan Funtowicz	31d9254776	feat(backend): remove static from inner_fw visitor as it leads to invalid memory locations	2024-11-14 08:42:01 +01:00
Morgan Funtowicz	b50dcddbb8	feat(backend): avoid dropping the boxed stream at the end of the callback	2024-11-14 08:42:01 +01:00
Morgan Funtowicz	d4aee42fd8	feat(backend): add logit parameter in the callback fn	2024-11-14 08:42:01 +01:00
Morgan Funtowicz	d52b4c4978	feat(backend): full rework of the backend internal to safer c++	2024-11-14 08:42:01 +01:00
Morgan Funtowicz	0c1dd0ed2b	feat(llamacpp): wip explosion	2024-11-14 08:42:01 +01:00
Morgan Funtowicz	a316c53255	feat(llamacpp): expose number of threads for the backend when constructing the model	2024-11-14 08:42:01 +01:00
Morgan Funtowicz	e4d803c94e	feat(backend): build and link through build.rs	2024-11-14 08:42:01 +01:00
Morgan Funtowicz	355d8a55b4	feat(backend): wip Rust binding	2024-11-14 08:42:01 +01:00