text-generation-inference/backends
Nicolas Patry 967e67111d Max token capacity metric (#2595)
* adding max_token_capacity_metric

* added tgi to name of metric

* Adding max capacity metric.

* Add description for the metrics

---------

Co-authored-by: Edwinhr716 <Edandres249@gmail.com>
2024-10-27 04:03:57 +00:00
..
client Pass the max_batch_total_tokens to causal_lm 2024-10-23 08:28:26 +00:00
grpc-metadata Rebase TRT-llm (#2331) 2024-09-25 05:55:39 +00:00
trtllm More fixes trtllm (#2342) 2024-09-25 06:08:00 +00:00
v2 Hotfixing main (#2556) 2024-10-25 08:53:47 +00:00
v3 Max token capacity metric (#2595) 2024-10-27 04:03:57 +00:00