Commit Graph

5465 Commits

Author SHA1 Message Date
Tomasz Zawadzki
e65310624b lib/event: accumulate idle_tsc when reactor has no threads
Before this patch idle_tsc was sum of all idle tsc of all
threads running on a reactor.

There are cases when no threads are present on the reactor,
and _reactor_run() spins doing nothing.

To give more accurate representation of the reactors state,
the idle_tsc now adds time spent doing idle spinning.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: If797b2a03507d17b07367d56d5f6c40cefbbbd49
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7900
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2021-05-25 07:11:53 +00:00
Tomasz Zawadzki
7438c388a0 lib/event: fix tsc_last for first thread on reactor
tsc_last value is used to update thread stats
during _reactor_run(). See:
spdk_thread_poll(thread, 0, reactor->tsc_last);

If no threads were present on the reactor,
this value got outdated and resulted in
adding time reactor spent with no threads to
stats of the first thread placed on that reactor.

This patch fixes thread stats by making sure
that argument to spdk_thread_poll() is up to date.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I0c35fdba1b63b6ee19a5a2b34751090839cb2438
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7845
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2021-05-25 07:11:53 +00:00
Tomasz Zawadzki
320fdaa325 ut/event: add current reactor tsc accounting
This patch does not change the UT in functional way.

Added accounting of last_tsc and spdk_get_ticks() similar
to the real application.
First the reactor_run() [not _reactor_run()] starts
reactor->tsc_last to current time. UT now set it at the start
and never touch it again directly.

Second the spdk_get_ticks() is updated to the elapsed time
to simulate flow of time, and make sure that its further
usage is up to date to the current time.

Fixed typo in test case description for test_reactor_stats.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Iaa2eb00fa3bffc2f21f1692da0259f1d023086b8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7904
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Community-CI: Mellanox Build Bot
2021-05-25 07:11:53 +00:00
Ben Walker
af935f768e event: Shift subsystem initialization code to a separate library
This is useful for applications even if they elect not to use the SPDK
event framework.

This doesn't shift everything in one go - just the subsystem
initialization logic. Configuration file loading also needs to move
in a separate patch later.

Change-Id: Id419df1045442d416650ed90e5ee78adfdd623d7
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6641
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-05-24 10:12:50 +00:00
ChengqiangMeng
36cb16219e unittest/nvme_tcp: increase code coverage for some handle
nvme_tcp_c2h_data_payload_handle and nvme_tcp_c2h_term_req_payload_handle and nvme_tcp_icresp_handle code comprehensive coverage

Signed-off-by: ChengqiangMeng <chengqiangx.meng@intel.com>
Change-Id: Ib7ccc7dbab11990f851807c0b07e7c0e67e61bde
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7872
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2021-05-24 10:12:14 +00:00
Mao Jiang
f596245ce6 test/nvmf/ctrlr: cases for ctrlr creating and destruct
Change-Id: Ib4b3c124e6906e63409afc7e225d5afb89a28e76
Signed-off-by: Mao Jiang <maox.jiang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7612
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2021-05-24 10:11:52 +00:00
Michal Berger
cf091238f5 test/make: Ignore abidiff's exit status
Upon running abidiff to get the list of impacted interfaces it seems
to exit with != 0 status. This triggers errexit causing the script
to exit in the middle of processing the lib files so we don't get
the full picture of all potentially impacted files. Ignore the
exit status in this case and allow the loop to go through all the
.sos.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I7edc5122161b0927fdd9a918571419f32fd46dac
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7815
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-05-24 10:11:05 +00:00
Michal Berger
24e0f403b8 test/make: Fix LTS tag lookup
Replicating 00afb74669.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I19049e35ad959a34342cd714d805db6bdd933ea4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7814
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-05-24 10:11:05 +00:00
Michal Berger
ef68667a82 pkgdep/git: Remove 20.01 release workarounds
There's already a new LTS release in place, hence these workarounds
are not needed anymore.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Idf597a5cf8da18d75f2630d59a277d4545fadc0f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7987
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-05-24 10:10:41 +00:00
Michal Berger
022ecaa7f1 vm_setup: Don't fail on freebsd if etc_os-release package is missing
This package is not available under latest 13.0 release, however,
the /etc/os-release we are looking for is already installed there.
For older versions, and in case this package is missing, we still
can determine that we are running under freebsd as part of the
fallback check to uname().

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I6a04eb66392d91899a4b55d4eb508bef140e924a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7985
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-05-24 10:10:41 +00:00
Ben Walker
15ae31fb0c nvmf: Rearrange spdk_nvmf_requset for cache line locality
Get all of the hot stuff to the first cache line.

* Shrink the xfer enum to one byte (it only has 3 values).
* Pull out the dif enabled flag form the dif structure so it
  can be access separately
* Rearrange the members

Change-Id: Id4a2fe90a49c055a4672642faac0028671ebfae9
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7827
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-05-24 10:10:19 +00:00
Michal Berger
0c25e6fbbc test/openstack: Clean up the test
This is done as part of the work to fix currently existing CI job
using this particular test suite.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Id5e9e8a814fb3025f192871650b8aa317f2de910
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7440
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
2021-05-24 09:58:17 +00:00
Mao Jiang
bb27314c03 test/nvme_pcie: cases for mapping io cmd
Change-Id: Icf174668e9b1361bb232db4a0cc53921423a17a4
Signed-off-by: Mao Jiang <maox.jiang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7671
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-05-21 08:04:39 +00:00
Krishna Kanth Reddy
b73d3e6894 test/nvme: Simple Copy Command takes nlb as a 0's based value
Signed-off-by: Krishna Kanth Reddy <krish.reddy@samsung.com>
Change-Id: I10a33d6478ce10fd2786c8eca2f179a9809764dc
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7973
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-05-20 20:15:51 +00:00
Evgeniy Kochetov
d8eb6957a0 ut/nvme: Add tests for NVMe controller namespace handling
Signed-off-by: Evgeniy Kochetov <evgeniik@nvidia.com>
Change-Id: Id2d3d1a15df22512f8d0d4414b5b3d35e79149f4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6505
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-05-20 15:17:41 +00:00
Evgeniy Kochetov
f0e248586e nvme/ctrlr: Always access namespaces via getter function
Signed-off-by: Evgeniy Kochetov <evgeniik@nvidia.com>
Change-Id: I16e327c0c0485057dc90a87cae316c4d6b62720d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6504
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-05-20 15:17:41 +00:00
Evgeniy Kochetov
65ff07719d nvme/ctrlr: Retrieve active NS list before NS construct
This patch changes the order of IDENTIFY_ACTIVE_NS and CONSTRUCT_NS
controller states. It is required to further improve memory management
for namespaces by allocating memory only for active ones.

Signed-off-by: Evgeniy Kochetov <evgeniik@nvidia.com>
Change-Id: Ie540442b1bd9e897afcbaa4319c139109dd0c515
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6503
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-05-20 15:17:41 +00:00
Evgeniy Kochetov
aebbce2520 nvme: Refactor active namespace list retrieval
Previous implementation allocated memory just once at the beginning of
active NS list retrieval procedure. It allocated memory for maximum
possible number of active namespaces, i.e. 'cdata.nn'.

This patch changes allocation logic. One page is allocated at the
beginning. If more is needed, reallocation is done with one more
page.

This patch also removes SPDK_MALLOC_DMA flag from allocation since we
don't do RDMA directly into this buffer.

Signed-off-by: Evgeniy Kochetov <evgeniik@nvidia.com>
Change-Id: Iaa80c4d70c54daaf71dcbf755c63a01a1d83b772
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6502
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-05-20 15:17:41 +00:00
Michal Berger
fead05c025 test/nvmf: Rework the traps
Declare single trap from nvmftestinit() instead overriding main trap
depending on the use-case (veth vs phy).

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I23c66fc5728510d06eee9837843f0734b2d03dcf
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7850
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-05-20 15:16:21 +00:00
Michal Berger
a3a0651569 autotest: Run dd tests under SPDK_TEST_URING
As these tests were limited to SPDK_TEST_BLOCKBDEV the spdk_dd was
not being tested against liburing. Make sure it is the case now.
Also, fail the test if spdk_dd is not linked to liburing in case
where such tests were requested.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I5704577530048ed4775ea15a6cdacf23380b5072
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7653
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-05-20 15:15:33 +00:00
Shuhei Matsumoto
4e9adb3bf9 thread: Replace TAILQ by Red Black tree for timed pollers
Use the macros for red black tree provided by Free BSD to manage
timed pollers efficiently.

Allow RB_INSERT() to insert elements with duplicated keys by changing
the compare function to return 1 if two keys are equal.

Check the return code of RB_INSERT() because this is the first use case
for RB tree macros in SPDK. We did the same for RB_REMOVE() by
adding another temporary variable but we remove it from this patch
because it is not so important compared with RB_INSERT().

When a timed poller is inserted, update the cache for the closest (leftmost)
timed poller only if the tree was empty before or the closest (leftmost)
timed poller was actually changed. We do not have to use RB_MIN()
because all duplicated entries are inserted on the right side.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ibe253ca8eecc10116548b5eedbcdba8fb961b88d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7722
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-05-20 15:13:27 +00:00
Shuhei Matsumoto
3ca15e33e4 thread: Set poller to PAUSING even when it is in RUNNING
This change is a preparation to first dequeue the closest timed poller
always when it is expired. Previously the poller_remove_timer() calls
were not consistent and difficult to follow.

spdk_poller_pause() sets poller to PAUSING even when it in RUNNING
and move it to PAUSED after returning from its context.

If spdk_poller_pause() and spdk_poller_resume() are called while poller
runs, it is moved to WAITING. Hence thread_execute_poller() and
thread_execute_timed_poller() ignore such cases.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I29340613a2ec0c3529d0886f4d81c0a0fdf8745d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7908
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2021-05-20 15:13:27 +00:00
Shuhei Matsumoto
50487701b1 ut/thread: Test if multiple timed pollers are expired simultaneously
Add a little complex test cases to avoid regression by the following
patches.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I1c94edd913f0a582868e355be085a1faf9bd1a94
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7913
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2021-05-20 15:13:27 +00:00
Jim Harris
f651e22a98 bdev: remove all references to spdk_bdev_open
This API was removed previously, so remove remaining
references in map file and unit tests.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iba2f6a5f5ba590d3996dc133c8181083a33d7405
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7963
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-05-20 15:02:59 +00:00
Changpeng Liu
f4d2ba4371 test/nvmf: use recommended nvmf_create_subsystem rpc
There are logs like below in existing CI tests:
"nvmf_subsystem_create is deprecated, use nvmf_create_subsystem instead."

Change-Id: Id257ab13120a71ad3d0ae7ed22ed6e0a4f005cf0
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7670
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-05-20 15:00:22 +00:00
Ziye Yang
5c0160263c bdev/rbd: Revise bdev_rbd_create rpc function.
Revise bdev_rbd_create rpc call to add an optional
parameter "--cluster-name", e.g., "--cluster-name Rados".

Then users can create a rbd bdev with registered
Rados Cluster. This shared strategy can be used to
remove the thread creation overhead if multiple rbds
are connected to the same Ceph cluster.

Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: Ide5800f8fc6b2074805272a59731c666fe279b9a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7584
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-05-19 08:08:39 +00:00
Mao Jiang
4e4d865fa0 test/nvme_cuse: cases for getting cuse ns device
Change-Id: I117b81554b6cf0f6eb1af7bd1050ff06cc351af9
Signed-off-by: Mao Jiang <maox.jiang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7442
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-05-19 07:50:53 +00:00
Jim Harris
bef22f45e7 bdevperf: add -F option for zipf distribution
Currently this will only support a global setting
that must be set on the command line.  We can make
it per-job later, but will require adding float
support to the conf library.

Tested by running bdevperf with some malloc
bdevs.  Performance with low theta values (i.e. 0.2)
are almost identical to random w/o zipf.  But
higher zipf values start to show better performance,
because we get more hits on data that is already
in the CPU cache.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I55b1587cdec2919973b488786d361042ca210606
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7790
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
2021-05-19 07:50:28 +00:00
Jim Harris
760452eefe bdevperf: store the rand_r seed in the bdevperf_job
This eliminates the thread-local seed variable.  But
we're also adding zipf distributions in an upcoming
patch, and we'll want to store that context in the
job rather than making it thread local.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: If8079682e7d3da8f989ee6b880edc8d3fcb4fdd8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7789
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
2021-05-19 07:50:28 +00:00
Rui Chang
d424e2449a test/json_config: Fix json_config test fail for multiple nvme disk config
In json_config test, when there are multiple nvme disks, we see errors like:

ERROR: expected event 'bdev_register:Nvme0n1p1' but got 'bdev_register:Nvme1n1' (whole event line: bdev_register:Nvme1n1:1)

The reason is when there are multiple nvme disks in the system, when
start nvmf_tgt, it will register all the disks in bdev, so there would
be multiple bdev_register events for these disks before we run
create_bdev_subsystem_config. So in create_bdev_subsystem_config we need
add these events to expected events list first.

Signed-off-by: Rui Chang <rui.chang@arm.com>
Change-Id: Ice8f5464975e68ae63fec891ef18dbc7590a9ab5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7858
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-05-18 12:44:35 +00:00
Alexey Marchuk
019a5361a5 nvmf/rdma: Use contig buffers for multi SGL payload
Currently we allocate buffers perf each SGL descriptor.
That can lead to a problem when we use NVME bdev with
PRP controller and length of the 1st SGL descriptor is
not multiple of block size, i.e. the initiator may send
PRP1 (which is SGL[0]) which end address is page aligned
while start address is not aligned. This is allowed by
the spec. But when we read such a data to a local buffer,
start of the buffer is page aligned when its end is not.
That violates PRP requirements and we can't handle such
request. However if we use contig buffer to write both
PRP1 and PRP2 (SGL[0] and SGL[1]) then we won't meet
this problem.

Some existing unit tests were updated, 1 new was added.

Fixes github issue #1853

Change-Id: Ib2d56112b7b25e235d17bbc6df8dce4dc556e12d
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7259
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2021-05-18 12:44:18 +00:00
Krishna Kanth Reddy
861bf2d1c3 test/nvme: Add test application to verify Simple Copy Command support
Signed-off-by: Krishna Kanth Reddy <krish.reddy@samsung.com>
Change-Id: I7bf61d98c444b473ec73f6afaf31ac408af6e409
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7696
Community-CI: Broadcom CI
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-05-18 12:43:24 +00:00
Mao Jiang
75f9a150a6 test/nvme_pcie_common: cases for creating and deleting IO queue
Change-Id: I79237a63b0f3b414cb3300bf41ee6193ce35e60f
Signed-off-by: Mao Jiang <maox.jiang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7632
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-05-18 12:42:52 +00:00
Shuhei Matsumoto
4748ebef40 thread: Cache the closest timed poller into thread
When we introduce RB tree, getting the closest timed poller is not
O(1) but O(log N). To mitigate such delay, cache the closest timed
poller into thread, and update the cache when its content is changed.

Add unit test cases for this change. They will also clarify the current
behavior of spdk_poller_unregister() and spdk_poller_pause() for
timed pollers.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ibb98a54c261859a3210034038d3953e5c93ef8aa
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7720
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Community-CI: Mellanox Build Bot
2021-05-14 10:08:02 +00:00
Krishna Kanth Reddy
f795dd6e66 test/unit: Add unit test for Simple Copy Command
Signed-off-by: Krishna Kanth Reddy <krish.reddy@samsung.com>
Change-Id: Ia43f5d0588133027e80dc5707fd5940c44605c68
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7695
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-05-14 09:59:54 +00:00
Alexey Marchuk
3ebf25e020 test/unit/nvme: Use page size in nvme_pcie_qpair_construct_destroy
Some systems may have page size other than 4096 bytes

Change-Id: Id56bb1566fff480edbe1a293121fc7a68c74252a
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7820
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: <dongx.yi@intel.com>
2021-05-13 10:09:45 +00:00
Tomasz Zawadzki
1c9125e78e test: add bdev_wait_for_examine to static JSON configs
Runtime RPCs such as bdev creation has no chance to wait for
bdev examination to finish. To handle this case commit below
introduced bdev_wait_for_examine RPC, and built it into all newly
saved JSON configurations:
(e57bb1af)lib/bdev: build bdev_wait_for_examine into subsystem

Some tests generate the configuration by hand, rather than
saving it from an existing application.
This patch embeds this RPC into the test configs.

Fixes #1760

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I79f998d722a2d19aa98b78333c64dbd2c1151444
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7861
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-05-13 10:07:07 +00:00
Tomasz Zawadzki
d69b16dac8 test/iscsi: remove comma from initiator_json_config
Function initiator_json_config is never called with
additional arguments that would fill up the subsystem
JSON.

Handling for the comma is removed.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I6ce5648a0eac29cb0f45d210696edf7977dfe89d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7860
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-05-13 10:07:07 +00:00
Shuhei Matsumoto
6edc4abe4b thread: Change remaining direct accesses outside lib/thread to spdk_thread to helper functions
Use spdk_thread_get_last_tsc() and spdk_thread_get_stats() in
unit tests for reactor. As a result, we can change one inclusion
from spdk_internal/thread.h to spdk/thread.h.

This is the same effort as spdk_poller.

The following patches will move the definition of struct spdk_thread and
enum spdk_thread_state from include/spdk_internal/thread.h to
lib/thread/thread.c.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I24d4a52ecf885b6e7db749616ee1f95b82574dc5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7801
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-05-13 10:03:44 +00:00
Jonathan Teh
be6a01efb3 nvmf: Allow configurable controller ID range
Add {min,max}_cntlid to spdk_nvmf_subsystem, defaulting to 1 and
0xFFEF, respectively, and add nvmf_subsystem_set_cntlid_range() to
allow the controller range to be configured in the range [min_cntlid,
max_cntlid].

Also add {min,max}_cntlid to the nvmf_create_subsystem RPC to allow
the controller ID range to be specified when creating an nvmf
subsystem.

Signed-off-by: Jonathan Teh <jonathan.teh@mayadata.io>
Change-Id: I936db3bb0c9a38569063a6fd3c11df262dfad776
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7322
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-05-12 09:18:48 +00:00
ChengqiangMeng
744f401cc9 test/nvme_fabric: Remove priority assert check for rdma.
It's only for tcp now, but the transport here is rdma. So there is no need to assert.

Signed-off-by: ChengqiangMeng <chengqiangx.meng@intel.com>
Change-Id: Ieffaac9d31a8f9c79a0d9fd226d74268986b7a77
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7823
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
2021-05-12 08:48:28 +00:00
Michal Berger
19f0c9a0d0 autotest: Replace fio.py with a bash wrapper in tests
fio.py simply wraps itself around fio and doesn't do anything that
would require python to be in use. Having it in a simple bash form
makes it easier to integrate it with autotest's common sh tooling
and to debug any potential issues with the underlying tests.

This also fixes #1919 by making sure only proper nvme devices are
selected for the nvmf targets.

Fixes: #1919.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I111d00df3c7b2517f431cae865e258a665c2ecb3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7684
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-05-12 08:48:10 +00:00
Krishna Kanth Reddy
78a794b9a3 examples/identify: Identification of Simple Copy Command support
Signed-off-by: Krishna Kanth Reddy <krish.reddy@samsung.com>
Change-Id: I251f3b1a6c7e36b946ddc8bd8a07b0f588e23c9e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7693
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-05-11 12:04:54 +00:00
Tomasz Zawadzki
a8fe601853 lib/blob: delay executing blob persist completions
This helps in next patch in series where multiple
completions will be executing.

UT is adjusted since one additional poll is required.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Id72377ddef91e40cdbc2bdea6f33c23309b0ca3d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7642
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-05-11 12:03:57 +00:00
Tomasz Zawadzki
958364b4a0 lib/blob: only verify cluster allocation when I/O is not frozen
During snapshot creation the original blob becomes
a thin provisioned blob that will only the diff of data after
snapshot creation.

Despite the comment in the UT the number of polls before issuing
blob write was hitting blob BEFORE it swapped with new one.

Issuing I/O during this period shall check for io freeze
before checking cluster allocation.

Otherwise bs_io_unit_is_allocated() hits assert for thin
provisioned blob. This is because cluster map of blob is
empty, but properties have not been updated yet.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I742e1a50b14d456ae1e6de13b5111caec3e8322c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7641
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-05-11 12:03:57 +00:00
Changpeng Liu
d0b4deab2b UT/bdev: add unmap UT to cover the split case
Change-Id: I902c9ce113cfbb2549ecb720951403496dec6d06
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7579
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-05-11 12:03:28 +00:00
Mao Jiang
980a139631 test/nvmf/ctrlr_bdev: cases for identify namespace
Change-Id: I315f48185268da696e4047e1c7ac5cf2a4221765
Signed-off-by: Mao Jiang <maox.jiang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7285
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-05-11 12:00:15 +00:00
Shuhei Matsumoto
646f41476d ut/nvme_fabric: Fix access to local variable after returning from the function
Fix issue #1936

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Iaa2b8060053d3f798001a6b207a6fcb0335c4b23
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7833
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-05-11 11:58:46 +00:00
Jim Harris
53447f060f nvmf: remove a bdev_module.h #include
ctrlr_discovery.c doesn't need this #include.
Including it causes bdev_module.h types to be
emitted to the debug symbols at least with some
compilers, which can result in unwanted abidiff
errors.

The unit tests do need it, so just include it
there instead.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iad28f9778ce08b11b52325658583ae9032295f3a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7813
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-05-11 11:57:01 +00:00
Jim Harris
8a12e2e715 scsi: don't include bdev_module.h
The library itself doesn't need it. The unit tests
do need it, so just include it there.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I9aefd303ae12928d45141029436509f185105bd3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7812
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-05-11 11:57:01 +00:00