mirror of
https://github.com/huggingface/text-generation-inference.git
synced 2025-09-12 12:54:52 +00:00
Doc review from Nico. x2
This commit is contained in:
parent
bb2b93e7a3
commit
0e09eeacfc
@ -31,6 +31,20 @@ curl localhost:8080/v1/chat/completions \
|
|||||||
-H 'Content-Type: application/json'
|
-H 'Content-Type: application/json'
|
||||||
```
|
```
|
||||||
|
|
||||||
|
For non-chat use-cases, you can also use the `/generate` and `/generate_stream` routes.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl 127.0.0.1:8080/generate \
|
||||||
|
-X POST \
|
||||||
|
-d '{
|
||||||
|
"inputs":"What is Deep Learning?",
|
||||||
|
"parameters":{
|
||||||
|
"max_new_tokens":20
|
||||||
|
}
|
||||||
|
}' \
|
||||||
|
-H 'Content-Type: application/json'
|
||||||
|
```
|
||||||
|
|
||||||
## Python
|
## Python
|
||||||
|
|
||||||
### Inference Client
|
### Inference Client
|
||||||
@ -46,11 +60,9 @@ pip install huggingface_hub
|
|||||||
You can now use `InferenceClient` the exact same way you would use `OpenAI` client in Python
|
You can now use `InferenceClient` the exact same way you would use `OpenAI` client in Python
|
||||||
|
|
||||||
```python
|
```python
|
||||||
- from openai import OpenAI
|
from huggingface_hub import InferenceClient
|
||||||
+ from huggingface_hub import InferenceClient
|
|
||||||
|
|
||||||
- client = OpenAI(
|
client = InferenceClient(
|
||||||
+ client = InferenceClient(
|
|
||||||
base_url="http://localhost:8080/v1/",
|
base_url="http://localhost:8080/v1/",
|
||||||
)
|
)
|
||||||
|
|
||||||
|
Loading…
Reference in New Issue
Block a user