mirror of
https://github.com/huggingface/text-generation-inference.git
synced 2025-04-26 04:22:08 +00:00
8863f3728c
- Fixes CPU affinity when running inference on CPU, and when CPUs are externally managed using taskset, numactl, cgroups, Kubernetes CPU manager, NRI resource policy plugins, for instance. - Detect external CPU management and trust the external CPU manager completely. It is more likely that external manager has the big picture of all other tasks running on the system, their QoS, hardware characteristics, etc. - For instance, do not modify even memory affinity, because the external manager may know better which NUMA node has fastest memory, or which NUMA nodes have enough free memory for this inference. Fixes: #3011 Signed-off-by: Antti Kervinen <antti.kervinen@intel.com> |
||
|---|---|---|
| .. | ||
| adapters | ||
| layers | ||
| models | ||
| pb | ||
| utils | ||
| __init__.py | ||
| cache.py | ||
| cli.py | ||
| interceptor.py | ||
| server.py | ||
| tracing.py | ||