huggingface/text-generation-inference
5
0
Fork 0
mirror of https://github.com/huggingface/text-generation-inference.git synced 2025-06-03 21:22:08 +00:00
Code Issues Packages Projects Releases Wiki Activity

Small update to docs (#2816)

Browse Source
This commit is contained in:
Nicolas Patry 2024-12-10 15:16:26 +05:30 committed by GitHub
parent b2fac5d947
commit a2d878fa0f
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
2 changed files with 2 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

BIN
assets/v3_benchmarks.png
View File

Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 201 KiB

After

Width:  |  Height:  |  Size: 209 KiB

4
docs/source/conceptual/chunking.md
View File

@ -80,7 +80,7 @@ Raw results
| | | | | |
|---|---|---|---|---|
|2nd run ||**TGI v3**|**vLLM**|**Amount of req**|
|2nd run ||**TGI v3** (time in s)|**vLLM** (s)|**Amount of req**|
|**Llama 3.1 8b**|Small test - L4 - 8B|17.5|19.9|200|
|**Llama 3.1 8b**|Long test* - L4 - 8B|53|57|10|
|**Llama 3.1 8b**|Small test - 4xL4 - 8B|4.8|6|200|
@ -88,7 +88,7 @@ Raw results
|**Llama 3.1 70b**|Small test - 8XH100 - 70B|6.2|7.4|200|
|**Llama 3.1 70b**|Long test - 8H100 - 70B|2|27.5|20|
||||||
|1st run ||TGI|vLLM|Amount of req|
|1st run ||TGI (s)|vLLM (s)|Amount of req|
|**Llama 3.1 8b**|Small test - L4|19.9|19.9|200|
|**Llama 3.1 8b**|Long test (10) - L4|49.8|55|10|
|**Llama 3.1 8b**|Small test - 4xL4|13|12.6|200|