text-generation-inference/backends/v3/src
Nicolas Patry 82c24f7420
Using both value from config as they might not be correct. (#2817)
* Using both value from config as they might not be correct.

* Fixing max_position_embeddings for falcon.

* Simple attempt to fix the healthcheck block allocation.

* Much simpler solution.

* Default value for Backend start_health
2024-12-10 19:37:09 +01:00
..
client Using both value from config as they might not be correct. (#2817) 2024-12-10 19:37:09 +01:00
backend.rs Using both value from config as they might not be correct. (#2817) 2024-12-10 19:37:09 +01:00
block_allocator.rs Lots of improvements (Still 2 allocators) (#2449) 2024-08-29 16:29:01 +02:00
lib.rs Choosing input/total tokens automatically based on available VRAM? (#2673) 2024-10-28 04:59:49 +01:00
main.rs feat: add payload limit (#2726) 2024-11-21 18:20:15 +00:00
queue.rs feat: auto max_new_tokens (#2803) 2024-12-06 05:50:35 +01:00
radix.rs Adding a test for FD. (#2516) 2024-09-16 17:00:54 +02:00