ivampiresp/Spdk
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

scheduler/dynamic: don't put dst core over limit when moving thread

Browse Source 
We have _is_core_over_limit() which determines if a core is
currently over its busy:total tsc ratio.  We use this to determine
if we need to move threads off of a core that is too busy.

But when we pick a core to move a thread *to* we were allowing the
dst core to fill to 100%, rather than the SCHEDULER_CORE_LIMIT.

This patch fixes that, which has the nice effect of keeping
thread-to-core assignments much more stable when running
I/O workloads.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Id98b08803939d2a25104082e6436bb8d4727d7c2

Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9578
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This commit is contained in:
Jim Harris 2021-09-22 09:29:45 -07:00 committed by Keith Lucas
parent ff66aa1ae2
commit 0ba8d51052
1 changed files with 11 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

14
module/scheduler/dynamic/scheduler_dynamic.c
View File

@ -169,6 +169,7 @@ static bool
_can_core_fit_thread(struct spdk_scheduler_thread_info *thread_info, uint32_t dst_core) _can_core_fit_thread(struct spdk_scheduler_thread_info *thread_info, uint32_t dst_core)
{ {
struct core_stats *dst = &g_cores[dst_core]; struct core_stats *dst = &g_cores[dst_core];
uint64_t new_busy_tsc, new_idle_tsc;
/* Thread can always fit on the core it's currently on. */ /* Thread can always fit on the core it's currently on. */
if (thread_info->lcore == dst_core) { if (thread_info->lcore == dst_core) {
@ -186,10 +187,17 @@ _can_core_fit_thread(struct spdk_scheduler_thread_info *thread_info, uint32_t ds
return true; return true;
} }
if (thread_info->current_stats.busy_tsc <= dst->idle) { /* Core doesn't have enough idle_tsc to take this thread. */
return true; if (dst->idle < thread_info->current_stats.busy_tsc) {
return false;
} }
return false;
new_busy_tsc = dst->busy + thread_info->current_stats.busy_tsc;
new_idle_tsc = dst->idle - thread_info->current_stats.busy_tsc;
/* Core cannot fit this thread if it would put it over the
* SCHEDULER_CORE_LIMIT. */
return _busy_pct(new_busy_tsc, new_idle_tsc) < SCHEDULER_CORE_LIMIT;
} }
static uint32_t static uint32_t