We want to avoid an active for_each_reactor operation
while the reactors are being shut down.
Fixes issue #1766
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I30bc258c5b22545320080d269a1ed8cb0b4e12f2
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6104
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Using reactor once insteads of target to fix this warning.
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Change-Id: I30cd6be26669fe1b2c5d694c15bfda223efd3b85
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6086
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
spdk_reactor_set_interrupt_mode will send event to set
reactor into interrupt mode or poll mode, also set every
notify_cpuset on all reactors for consistency.
It can be used by RPC method or scheduler to set
reactor to interrupt mode while workload is lightweight.
Currently, this function is limited that the specific
reactor should have no attached spdk_thread.
Change-Id: I7e8f449bff1184b9a7948f80b9572066a19da60f
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5853
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Currently, spdk_thread can't get executed on reactor
which is in interrupt mode but interrupt of spdk_thread
is not enabled. So avoid schedule spdk_thread on it.
Change-Id: Ib37d1585f173ed3be8def83cd18aef65e287361c
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5852
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Each bit of the cpuset indicates whether a reactor
is going to be in interrupt mode.
Each spdk_cpuset is allocated to each reactor. So it
can only be touched by its reactor.
Change-Id: Ic186de341588b701d7471bf09336309d28b1bf4e
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5850
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Those spdk threads were neither used or tracked in event framework.
Needed to edit hardcoded cpumask value in spdkcli test to accept any string.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ieffea1bdf467f50ec0bd3af4b17a1d259fd876b4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5817
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Added rpc to get name and period of currently set spdk scheduler
and name of currently set spdk governor.
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Change-Id: I5562a81a7f9e4879bd48a765c9467f70b43f73ab
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5917
Community-CI: Broadcom CI
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This field will be used in upcoming patches
by scheduler to keep a history of threads
statistics. Base on those scheduler will
be able to calculate how busy a thread is.
Also we have to save the value of this field
while rescheduling thread.
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: Iffbd1f946e1d159ec7277a39bc30161e71722456
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5972
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Size of g_reactors allocated is g_reactor_count, so
valid lcore should be less than g_reactor_count.
Change-Id: I9d5cde7856d6b400a7188631812b53a7e5701d0f
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5870
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Record result of spdk_env_get_last_core() to
g_reactor_count, since spdk_env_get_last_core()
iterates through all core ids in a loop, it is
not so efficient.
Change-Id: I6bbbfec0bc25490bab33436c84fca3da6380cbe8
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5955
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
That is more inline with other places we
do things like this in SPDK. Also change
argument type to uint64_t.
Change-Id: Ie98f6c37bf53d583431ea9edd7d977075aeaa7a0
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5876
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
This will be used by a scheduler while balancing threads
to get information how much busy a thread is.
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: Ib0e897a9bb5d2a71173f60cce7e1d442e67a57f9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5253
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
This fixes issue where scheduler is not being
initialized on first set.
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I881438955f08852fb72187a2b92c7acfb12cd662
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5029
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
master_core will still be available but deprecated.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I6fe9eb7fecb2919795c439a02cb7d9d06a2774d8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5358
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
A round of _reactors_scheduler_gather_metrics should be stopped
when there is calloc failure.
Change-Id: Ic2220c561abb07a849ea37d3c88af3f6d5d1ffa1
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4882
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Features:
- does not delay thread creation
- does delay thread deletion
- singlethreaded, but can be made multithreaded.
By being singlethreaded, we don't waste time,
because reactors are not paused during rescheduling,
but we do make statistics less up to date
Change-Id: Ie5a7e8569bc32b3fd4bb887804dfbc3f5c2ea858
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3899
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Move current scheduler to it's own module
and make use of new API.
Change-Id: I4928aed82603d51de01194c9650709e814f7f61b
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4054
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Add structures and function for new scheduler API.
Change-Id: I8e379db62189eb7c36092265c137f07368650e9b
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4053
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Each reactor and each thread is assigned with one fd group.
At the same time, each thread is treated as one interrupt source
registered into its corresponding reactor.
The egrp function for reacotr is the only block point waiting
for events.
Change-Id: Id092e66591b07b445342f8ae16dc218d28887427
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4269
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
It's not needed. Add it to one of the UT where it
is needed though.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I06628184c22df68deffce3ca0561878569ccf3f4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4717
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
There is nothing left here, so remove it.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ib947d42bc577dbebb4650b1be885e05a80f8f8cf
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4541
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Anil Veerabhadrappa <anil.veerabhadrappa@broadcom.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Community-CI: Broadcom CI
This patch removes the string from register component.
Removed are all instances in libs or hardcoded in apps.
Starting with this patch literal passed to register,
serves as name for the flag.
All instances of SPDK_LOG_* were replaced with just *
in lowercase.
No actual name change for flags occur in this patch.
Affected are SPDK_LOG_REGISTER_COMPONENT() and
SPDK_*LOG() macros.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I002b232fde57ecf9c6777726b181fc0341f1bb17
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4495
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Mellanox Build Bot
Reviewed-by: Anil Veerabhadrappa <anil.veerabhadrappa@broadcom.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Broadcom CI
cpumask can be changed by spdk_thread_set_cpumask()
during the time that event takes before it arrives
on _schedule_thread() function, which would make the
function assert(false), even though that is ~ok~.
Currently, that can happen right after thread is created
or between two successive calls to spdk_thread_set_cpumask().
But most importantly, it will constantly happen if we
introduce rescheduler.
This patch just disables the check for now.
Change-Id: Ie6dfe22d6eff2c908c367d1311436cc6769a6960
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3905
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
In practice, the reactor pointers will never be NULL
in these cases, but some static analysis tools don't
realize that.
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I0657959e5572df2741398b179907f2bbf0b02b3b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1984
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Mellanox Build Bot
Extract _spdk_thread_exit() from spdk_thread_exit() and
_spdk_thread_poll() calls _spdk_thread_exit() if the thread is in
the exiting state. spdk_thread_exit() changes to move the state to
the exiting state. The spdk_thread_poll() loop will end after the
thread moves to the exited state because the caller of
spdk_thread_poll() will check if the thread is in the exited state,
and break the loop if true.
If the user does not call spdk_thread_exit() explicitly, the reactor
has to terminate all existing threads at its shutdown. In this case,
multiple threads may have some dependency to release I/O channels or
unregister pollers. So the reactor has the large two loops, the first
loop calls spdk_thread_exit() on all threads, the second loop calls
spdk_thread_destroy() if exited or spdk_thread_poll() otherwise for
each thread until all threads are destroyed.
Besides, change the return value of spdk_thread_exit() to return
always 0. Keep it for ABI compatibility. Change ERRLOG to INFOLOG
for _spdk_thread_exit() because it is called repeatedly now. Remove
the check of I/O reference count from _spdk_thread_exit() because
_free_thread() cannot free I/O channel. Refine the unit test
accordingly.
Fixes issue #1288.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Iee5fb984a96bfac53110fe991dd994ded31dffa4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1423
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Following patches will require thread termination so that thread_exit()
will move thread to exiting, thread_poll() will move thread from
exiting to exited, thread_is_exited() will detect the threadis exited,
and then call thread_destroy().
So change all places as a preparation.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I6b2e8aee5ed7cd160a88b4c9aaed7d90bd9dac07
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1640
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Previously the caller had to check if thread is not exited when
it calls spdk_thread_exit().
Subsequent patches will change return type of spdk_thread_exit()
to void, and so include the check int spdk_thread_exit() in this
patch.
If spdk_thread_exit() is called when the thread is already exited,
collect INFOLOG and return normally.
This will make the next patch a little easier.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I8b94261575e770485b33c0b37e76e770b77b417c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1639
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Following the idea of thread CPU stats, add reactor CPU stats.
Reactor CPU stats accumulates run time of spdk_thread_poll() calls
to idle TSC or busy TSC according to their return codes.
Add necessary unit tests.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I1a1391e79d74387c68f1651a61c8900e4c6faf66
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1501
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Collect elapsed time of each SPDK thread and add it to output of
framework_get_reactors RPC.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I9e2f0487de81720327428cda5738284a4ce2c557
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1278
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
This patch updates reactor to count thread run time correctly
on multiple SPDK threads per CPU core configuration by using
the refined spdk_thread_poll().
Add tsc_last to struct spdk_reactor to use the end time of the
last thread as the start time of the next thread.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I21042867885d289ff0c23bf2a9ba6a8076a59673
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1256
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Add thread_count to struct spdk_reactor to count number of threads
per reactor. This number will be used in the next patch to know
if all threads are idle or not for each reactor to support CPU
power saving.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I4f7cc5a6b78d85e9f8d0b539c60058c13e282759
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1169
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
It was too difficult for us to add unit test for _spdk_reactor_run()
because the main polling loop in _spdk_reactor_run() was infinite
if g_reactor_state is SPDK_REACTOR_STATE_RUNNING.
Factor out the main polling loop and rusage update in _spdk_reactor_run()
into a helper function reactor_run().
This improves the code readability as well.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ia26db14f44026fefd696443227c16d2be4166832
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1186
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
This is a preparation to the next patch which factors out the main
polling loop of _spdk_reactor_run() into a helper function reactor_run().
One of the subsequent patches will support CPU power saving by
adding sleep into reactor_run(). We should not insert sleep between
the main polling loop and getrusage() because now is got before
entering the main polling loop.
To put getrusage() into reactor_run(), we need to maintain last_rusage
in struct spdk_reactor and maintain g_rusage_period as a global
variable. This patch does these changes.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I61bf50de6a170ac73c8fe17e85077b90171dd9c0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1185
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Add a flag resched to check if reschedule operation is requested
to struct spdk_lw_thread. Add _reactor_resquest_thread_reschedule()
to set the resched flag, and add it to the case SPDK_THREAD_OP_RESCHED
in spdk_reactor_thread_op(), and return true in the case
SPDK_THREAD_OP_RESCHED in spdk_reactor_thread_op_supported().
Then _spdk_reactor_run() checks if the resched flag is true for each
thread. If true, set the resched flag to false, and remove the
thread and call _reactor_schedule_thread(). Add continue to avoid
use-after-free issue for both reschedule and terminate cases.
This idea follows voluntary thread termination and will remove our
worries for all complicated rare cases.
Add unit test case to verify this update.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I656872d32dbb469ae70f771cd0419a77236bfe18
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/500
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Remove the prefix "spdk" from spdk_reactor_schedule_thread(), add
spdk_reactor_thread_op() and spdk_reactor_thread_op_supported().
For SPDK_THREAD_OP_NEW, spdk_reactor_thread_op() calls
_reactor_schedule_thread() and spdk_reactor_thread_op_supported()
returns true.
Then replace spdk_thread_lib_init() by spdk_thread_lib_init_ext()
with spdk_reactor_thread_op() and spdk_reactor_thread_op_supported().
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I232a3b2c6bcaf4d86b0dd3cefacd3e47eadda6d8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/968
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
In the reactor main loop, _spdk_reactor_run(), check if both
spdk_thread_is_exited() and spdk_thread_is_idle() return true,
and if they are true, then remove it from the queue and call
spdk_thread_destroy() to delete it.
By previous patches, the exited thread does not accept any
new message or poller, all pending messages are processed, and
all unregistering pollers are completed. Hence we only need to
have very simple check in the reactor main loop.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ib678245de98a5c050843e6435026bdcf8b6c75c7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/507
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
This is a preparation to support voluntary thread termination by
calling spdk_thread_exit().
Change spdk_thread_exit() to return -EINVAL if the thread is already
marked as exited. This will be helpful to detect wrong call sequence
of voluntary thread termination.
Besides, update reactor shutdown and unit test framework shutdown
to incorporate this change accordingly.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I2296c61e273bf4d9580656dcbc2da0e8a8f3bcf7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/671
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
spdk_reactor_enable_framework_monitor_context_switch and
spdk_reactor_framework_monitor_context_switch_enabled had been
a little long and not easy to get the meaning.
spdk_framework_enable_context_switch_monitor and
spdk_framework_context_switch_monitor_enabled will be a little
more concise, and hence change the names accordingly.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I5f1d50e8b62846cbd0f91b94f94cbaf16fefa39b
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478538
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Following the last patch, this is also a preparation to add
reactor_get_stats RPC.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I18c260bb10bdf1c7aa5e00aa81a171f2ff50c7d5
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478026
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Following the last patch, this is also a preparation to add
reactor_get_stats RPC.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ia6dc29f591f7112512be8a67b180b056150f467b
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478025
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
This API will be used in the upcoming reactor_get_stats RPC first.
This API is not public but internal in SPDK. Add necessary unit test
together.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I761ffe3c8d6d8da4594c0d728b479f9f40275fc9
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478024
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
Add check if reactor scheduled the thread to one of the allowed
cores correctly to _schedule_thread().
This check will be useful in the next patch and may be helpful
when we schedule SPDK thread dynamically.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ibea8e8315187ae8a3a421007d8865bbee2d7e037
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478156
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
struct spdk_cpuset has been declared in the header file and
so we can allocate struct spdk_cpuset statically. This change will
avoid potential out of memory failure for reactor too.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ic5f4fdb493d73b4ae11cddd2638eeb84ebbb792e
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478150
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
spdk_thread_poll returns only non-negative value now, and the return
code check is not reasonable now. For just in case we may want to add
assert(rc >= 0) but compiler gives warning because rc is not used anywhere.
So just remove checking return code check here.
We noticed first that spdk_thread_destroy was called without spdk_thread
exit but this change removes spdk_thread_destroy. So the issue is also
fixed together.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I4e7f5c09faf5eda68ea8d42320dbefdca88bbd21
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478237
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
For now we don't provide #ifdef to users, SPDK_CACHE_LINE_SIZE is defined with 64 bytes.
Change-Id: I8ddb01a382008a3e333967650200c5aa8b93e09e
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/477385
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
spdk_reactors_fini() is unconditionally called in spdk_app_fini()
at the end of every application and it currently throws a ton
of warning messages if the reactors weren't initialized yet [1].
Let's silence those warnings.
[1] $ spdk_tgt -c invalid.conf
[...]
*WARNING*: Called spdk_reactor_get() while the g_reactors array was NULL!
*WARNING*: Called spdk_reactor_get() while the g_reactors array was NULL!
*WARNING*: Called spdk_reactor_get() while the g_reactors array was NULL!
*WARNING*: Called spdk_reactor_get() while the g_reactors array was NULL!
*WARNING*: Called spdk_reactor_get() while the g_reactors array was NULL!
(Apparently SPDK_ENV_FOREACH_CORE iterates through 128 cores
if the dpdk env framework wasn't initialized. SPDK calls
spdk_reactor_get() on each core and that's what generates the
warnings)
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Change-Id: Ic3a2355ef6d2e0d0e1cc125ba21cc6a802b355bc
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/470736
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Get all of the important stuff into the first cache line.
Change-Id: I5bbfb031bb1d693019abb9e5145579d0b867eaf5
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/465994
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Begin organizing file so setup operations appear
at the top.
Change-Id: I7411b4bf20480c8aeb40bc21b521e5d359f8da1f
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/465991
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
There is a case in vhost where this occurs. Until that is
sorted out, just make this a warning.
Change-Id: Id021791e8cbddf3023e0cb1b8c52a733b3578a7d
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/466075
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Seth Howell <seth.howell@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This was unused. The rusage polling is done in another way.
Change-Id: I478ddb2d664647e922f3049a64199fdc61f25ce1
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/465989
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
DPDK rte_ring_enqueue_bulk() has free_space parameter to return
the amount of space in the ring after enqueue operation has finished.
This parameter can be used to wait when the ring is almost full and
wake up when there is enough space available in the ring.
Hence we add free_space to spdk_ring_enqueue() and spdk_ring_enqueue()
passes it to rte_ring_enqueue_bulk() simply.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I9b9d6a5a097cf6dc4b97dfda7442f2c4b0aed4d3
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456734
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Lightweight threads may now be exited by calling
spdk_thread_exit() within the thread. The framework
polling the thread can release the resources associated
with that lightweight thread by calling
spdk_thread_destroy().
Change-Id: I6586b9d22556b3874fb113ce5402c6b1f371786e
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455319
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Individual threads can have their own CPU mask. Attempt to
respect that in the scheduler.
Change-Id: I2bd08d4249bdae32a459ed8770b88090346be5dc
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452258
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
We're about to drop legacy event messages from SPDK libs
and for that we'll replace various lcore numbers with
spdk_thread objects. For now SPDK libs can't spawn their
own threads, so they must use spdk_for_each_thread() and
spdk_get_thread() to retrieve different thread pointers.
The vhost library offers API to construct a vhost device
to be polled on specified cores, and in order to keep that
functionality we'll need to know which core each thread
is polling on. We would like to achieve that with
spdk_thread_get_cpumask(), but right now it always returns
all cores, so this patch changes it to return just
a single cpu on which the thread is actually pinned. It's
only a stop gap, eventually the vhost library will spawn
its own threads with custom cpumasks.
Change-Id: I15947727123c51b23f63727d52079770bfb2e07b
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452204
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Before enabling SPDK thread scheduler,
each POSIX thread name was reactor_% (% was the core number).
After enabling SPDK thread scheduler, the name of the master POSIX
thread is reactor_% (% is the max core number), and the name of the
slave posix threads are lcore-slave-% (% is the current core number).
SPDK threads are light-weight threads - sometimes also called
green threads or fibers, and so are independent from posix
threads.
But reactor is tied to the POSIX thread in the SPDK event library.
So SPDK thread doesn't rename the POSIX thread at its creation
but reactor renames the POSIX thread at start instead.
This change makes POSIX thread name compatible between before and
after enabling SPDK thread scheduler.
Change-Id: I26e8dabc73e163c9f74e18b3640cf54954603b1f
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/451712
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Allow users to optionally specify an affinity mask when
creating a thread. This isn't currently used, but we want
the API to be in its final form for the next release.
Change-Id: I7bd05e921ece6d8d5f61775bd14286f6a58f267f
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/451683
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Limit the thread scheduler to put spdk_threads on
lcores < last_lcore instead of lcores < lcore_count,
which was probably the original intent.
When the cpumask was not a contiguous cpu range, the
thread scheduler failed to schedule any spdk_threads on
the last cores. There was one hardcoded thread created
for each reactor, but the scheduler could squash some
of those into a single reactor. This broke the legacy
lcore-based messages as those expect there to be at least
one spdk_thread per lcore. Any spdk_poller_register()
or spdk_get_io_channel() called from such a legacy
message would fail an assertion, as spdk_get_thread()
returned NULL.
Fixes#743
Change-Id: I81a3f76d9c4788596c697df6ff51b264b99ce10b
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/450353
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Events are the mechanism by which threads are scheduled,
so events need to be processed even if there are no
threads.
Change-Id: I2e908e6a948709f2122b1c7385b6fd771827b9aa
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/447111
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Just use a function pointer and a context.
Change-Id: I2d41ed2572d892f3328aadf7f22d8696816bf4d1
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/446995
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
It iterates over the list and polls each one. However,
in practice the list still contains just one thread for
now.
Change-Id: I9bac7eb5ebf9b4edc6409caaf26747470b65e336
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/440763
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Schedulers can use this region to store required information.
Change-Id: I93efb44f1a534596f6285bbe014579311fe011e7
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/444454
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This is much simpler and avoids the problems with requiring
it to run on a thread.
Change-Id: I811444c5a15d292356703beccc17e505d55d7678
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/443645
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
The thread scheduling mechanism is being rewritten and this
won't be used in the new system.
Change-Id: I829e8118ed0a10480bd86934b45e68fcb810931a
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/444453
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Keep all of the thread library interactions in one file.
Change-Id: Iecb20d3767190b5da105a29670ead9e192d03257
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/440761
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
spdk_thread_poll()
This is an optimization if the calling function already knows the
current time.
Change-Id: I1645e08e7475ba6345a44e0f9d4b297a79f6c3c2
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/443634
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
This name more closely resembles pthread_exit, which is a
closer analogy to how the new threading library works.
Change-Id: I68b04509f3ff8e94b8688804a7e5661155a3ecd1
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/440597
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
This mirrors pthread_create, which works more closely
to the new style where SPDK libraries can spawn their
own threads.
Change-Id: Ic524c4c35bcf7c1611e4f261ebb64b98ac5a5a1b
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/440596
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Instead of implicitly grabbing the thread from the thread
local variable, make it explicit.
Change-Id: I733fad06181439e12b1e71a4829b84e7b64e2468
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/440595
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
These are no longer used by anything.
Change-Id: I0db6bc88e4dc945ff4f64df2ac410e1d00a669c1
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/437601
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Previously, a pool of events per socket was created for NUMA
locality. However, when passing a message between threads,
the event would be taken from one pool and put back into another,
resulting in imbalances.
At this time, I do not see an efficient way to allocate the events
so that they remain NUMA local. We'll work on that over time.
However, for correctness right now, go down to a single global
pool of events.
Change-Id: I09f8b0c5c928777e8274c53c6dce21b9c346e2a5
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/433519
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Move the message ring and the polling into the thread
itself.
Change-Id: I5bc53c68bb1314fd69ed80cc98c5b86ce8be880f
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/423768
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
The user of the thread library is now only responsible for
periodically calling spdk_thread_poll. Pollers are handled
internally.
In order to avoid changing all of the unit tests, the ability
to provide function pointers to change the behavior of
the poller registration is still in the code. This should
only be used from tests until they are all converted.
Change-Id: Ie2c00ce1d57ca3710ed2c469cd711924768e23ef
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/417784
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Per the definition of the poller returned value
* return 0 to indicate that polling took place but no events were found;
* return positive to indicate that polling took place and some events were processed;
* return negative if the poller does not provide spin-wait information.
Have a change to compare whether positive.
Change-Id: Ic01fe7a663afeccbb145979670234e023dac1d07
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.gerrithub.io/428576
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This sleep functionality is only really used by the stub
app currently. nvmf target enables it, but it never
gets exercised since there is always a poller running on
each core. So don't bother trying to count how long
the reactor didn't take action - try to sleep any time
where the loop did not take action.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: If48cc3f989347811190de67a6423932d0b77cf45
Reviewed-on: https://review.gerrithub.io/423577
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
This was defined in two places, so consolidate
the definitions.
Change-Id: I0bbb262b97e90d1064bcc50ee201928f6ca9518a
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/423182
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This adds tsc statistics (busy, idle or unknown) to the reactor.
It will help measure actual time we are busy vs. idle during
an event run or whenever a poller kicks in.
Change-Id: Ife556a27a30dab842488bacb5fbee0d4297745c3
Signed-off-by: Vishal Verma <vishal4.verma@intel.com>
Reviewed-on: https://review.gerrithub.io/412695
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I6babd4cf990bf19b510db88bdfb0ca81e29d9252
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/414700
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Madhu Pai <mpai@netapp.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
This removes checks for timer poll iterations
within a reactor.
Change-Id: Id8eea26dab201064123575998b3b24e1f609ac5a
Signed-off-by: Vishal Verma <vishal4.verma@intel.com>
Reviewed-on: https://review.gerrithub.io/412694
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
