ivampiresp/Spdk
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: update scheduler documentation

Browse Source 
Updated scheduler documentation with information
about thread and core workloads, scheduling process
and dynamic scheduler parameters.

Added descriptions of new dynamic scheduler parameters
in jsonrpc.md.

Change-Id: I567efe21c425c13c16235d7976aad6ae961381bb
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11886
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This commit is contained in:
Krzysztof Karas 2022-03-11 08:29:51 +00:00 committed by Jim Harris
parent 8ec34933e9
commit 835494b53a
2 changed files with 31 additions and 5 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
doc/jsonrpc.md
View File

@ -713,6 +713,9 @@ Name | Optional | Type | Description
----------------------- | -------- | ----------- | ----------- ----------------------- | -------- | ----------- | -----------
name | Required | string | Name of a scheduler name | Required | string | Name of a scheduler
period | Optional | number | Scheduler period period | Optional | number | Scheduler period
load_limit | Optional | number | Thread load limit in % (dynamic only)
core_limit | Optional | number | Load limit on the core to be considered full (dynamic only)
core_busy | Optional | number | Indicates at what load on core scheduler should move threads to a different core (dynamic only)
#### Response #### Response

33
doc/scheduler.md
View File

@ -48,10 +48,11 @@ core has to be managed by the administrator.
## Scheduler implementations ## Scheduler implementations
The scheduler in use may be controlled by JSON-RPC. Please use the The scheduler in use may be controlled by JSON-RPC. Please use the
[framework_set_scheduler](jsonrpc.md/#rpc_framework_set_scheduler) RPC to [framework_set_scheduler](jsonrpc.html#rpc_framework_set_scheduler) RPC to
switch between schedulers or change their options. switch between schedulers or change their options. Currently only dynamic
scheduler supports changing its parameters.
[spdk_top](spdk_top.md#spdk_top) is a useful tool to observe the behavior of [spdk_top](spdk_top.html#spdk_top) is a useful tool to observe the behavior of
schedulers in different scenarios and workloads. schedulers in different scenarios and workloads.
### static [default] ### static [default]
@ -65,11 +66,27 @@ the previous behavior of the SPDK event/application framework.
The `dynamic` scheduler is designed for power saving and reduction of CPU The `dynamic` scheduler is designed for power saving and reduction of CPU
utilization, especially in cases where workloads show large variations over utilization, especially in cases where workloads show large variations over
time. time. In SPDK thread and core workloads are measured in CPU ticks. Those
values are then compared with all the ticks since the last check, which allows
to calculate `busy time`.
`busy time = busy ticks / (busy tick + idle tick) * 100 %`
The thread is considered to be active, if its busy time is over the `load limit`
parameter.
Active threads are distributed equally among reactors, taking cpu_mask into Active threads are distributed equally among reactors, taking cpu_mask into
account. All idle threads are moved to the main core. Once an idle thread becomes account. All idle threads are moved to the main core. Once an idle thread becomes
active, it is redistributed again. active, it is redistributed again. Dynamic scheduler monitors core workloads and
redistributes SPDK threads on cores in a way that none of them is over `core limit`.
In case a core utilization surpasses this threshold, scheduler should move threads
out of it until this condition no longer applies. Cores might also be in overloaded
state, which indicates that moving threads out of this core will not decrease its
utilization under the `core limit` and the threads are unable to process all the I/O
they are capable of, because they share CPU ticks with other threads. The threshold
to decide if a core is overloaded is called `core busy`. Note that threads residing
on an overloaded core will not perform as good as other threads, because the CPU ticks
intended for them are limited by other threads on the same core.
When a reactor has no scheduled `spdk_thread`s it is switched into interrupt When a reactor has no scheduled `spdk_thread`s it is switched into interrupt
mode and stops actively polling. After enough threads become active, the mode and stops actively polling. After enough threads become active, the
@ -80,3 +97,9 @@ not exceed the sum of all idle threads. When no active threads are present on
the main core, the frequency of that CPU core will decrease as the load the main core, the frequency of that CPU core will decrease as the load
decreases. All CPU cores corresponding to the other reactors remain at maximum decreases. All CPU cores corresponding to the other reactors remain at maximum
frequency. frequency.
The dynamic scheduler is currently the only one that allows manual setting of
its parameters.
Current values of scheduler parameters can be displayed by using
[framework_get_scheduler](jsonrpc.html#rpc_framework_get_scheduler) RPC.