huggingface/text-generation-inference
5
0
Fork 0
mirror of https://github.com/huggingface/text-generation-inference.git synced 2025-09-11 20:34:54 +00:00
Code Issues Packages Projects Releases Wiki Activity

Odd.

Browse Source
This commit is contained in:
Nicolas Patry 2024-04-22 21:28:08 +00:00
parent f65d06cfb6
commit 0b20661cb7
1 changed files with 2 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
docs/source/conceptual/speculation.md
View File

@ -1,5 +1,6 @@
## Speculation ## Speculation
Speculative decoding, assisted generation, Medusa, and others are a few different names for the same idea. Speculative decoding, assisted generation, Medusa, and others are a few different names for the same idea.
The idea is to generate tokens *before* the large model actually runs, and only *check* if those tokens where valid. The idea is to generate tokens *before* the large model actually runs, and only *check* if those tokens where valid.