text-generation-inference/router/src
Nicolas Patry 82c24f7420
Using both value from config as they might not be correct. (#2817)
* Using both value from config as they might not be correct.

* Fixing max_position_embeddings for falcon.

* Simple attempt to fix the healthcheck block allocation.

* Much simpler solution.

* Default value for Backend start_health
2024-12-10 19:37:09 +01:00
..
infer Using both value from config as they might not be correct. (#2817) 2024-12-10 19:37:09 +01:00
config.rs Auto max prefill (#2797) 2024-12-06 05:52:00 +01:00
kserve.rs fix: simplify kserve endpoint and fix imports (#2119) 2024-06-25 19:30:10 -04:00
lib.rs Auto max prefill (#2797) 2024-12-06 05:52:00 +01:00
logging.rs Rebase TRT-llm (#2331) 2024-07-31 10:33:10 +02:00
sagemaker.rs feat: allow any supported payload on /invocations (#2683) 2024-10-23 11:26:01 +00:00
server.rs Auto max prefill (#2797) 2024-12-06 05:52:00 +01:00
usage_stats.rs feat: allow any supported payload on /invocations (#2683) 2024-10-23 11:26:01 +00:00
validation.rs feat: auto max_new_tokens (#2803) 2024-12-06 05:50:35 +01:00
vertex.rs Auto max prefill (#2797) 2024-12-06 05:52:00 +01:00