Commit Graph

2170 Commits

Author SHA1 Message Date
Shuhei Matsumoto
d4558c6132 bdev/nvme: Reduce conversion between spdk_bdev_io and nvme_bdev_io
We can hold bdev_io directly in nvme_bdev_ctrlr as an outstanding reset.

We can put spdk_bdev_io_from_ctx(bio) into a parameter for a few
functions because it is used only once in a function.

Passing not spdk_bdev_io but nvme_bdev_io to bdev_nvme_verify_pi_error()
remove unnecessary substitution.

This is a little more efficient and simplifies the implementation.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: If49ad9fa42abf27decf3afcd8c994f55faa3bc70
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8094
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-06-02 11:37:58 +00:00
Mao Jiang
f70180d50d test/nvme_cuse: cases for cuse submitting io
Change-Id: I21f640c1d633884543961b382c400449caba241b
Signed-off-by: Mao Jiang <maox.jiang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6935
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-06-02 11:37:33 +00:00
Ziye Yang
b85127cc6f util/crc32c: Add spdk_crc32c_iov_update.
Purpose: To support caculating crc32c for iovs.

Change-Id: I7ea6d8b71ea2cf6e8b81903439787870923b9bef
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8093
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2021-06-02 11:37:17 +00:00
ChengqiangMeng
6a49507c47 unittest/nvme_rdma: increase code coverage for nvme_rdma.c
nvme_rdma_parse_addr and nvme_rdma_qpair_init code comprehensive coverage.

Signed-off-by: ChengqiangMeng <chengqiangx.meng@intel.com>
Change-Id: I7474281a2b80c092f0ed516e1da3649473eb0fa3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7337
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2021-06-02 10:23:51 +00:00
Mao Jiang
0651ffe3f1 test/nvme_rdma: cases for rdma group connecting qpair
Change-Id: Ia2d76fb364671971d3c78555ab203c3c565ec2f3
Signed-off-by: Mao Jiang <maox.jiang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8029
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2021-06-01 07:42:55 +00:00
Jim Harris
a31e319fbb nvme: init discovery ctrlr using nvme_ctrlr_process_init
This ensures the discovery ctrlr initialization is
done the same as normal ctrlrs.  This will be
critical as we make the driver fully asynchronous.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I33c4fd7c82d241c30e7adb89abe79b8088c8776a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8090
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-05-28 08:14:06 +00:00
Jim Harris
df01076f70 nvme: add NVME_CTRLR_STATE_READ_VS
Read VS (Version) register as part of controller
initialization instead of controller construction.

This prepares for upcoming changes to make
controller attach fully asynchronous.  Since reading
fabrics registers is an asynchronous operation, it
will be easier to read the VS register as part of
controller initialization which operates as an
asynchronous state machine.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I771386dbdf5902633e0d9f91b3b20be98f26fdc3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8076
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2021-05-28 08:14:06 +00:00
Changpeng Liu
cfff04aea2 nvme/cuse: add subsystem reset IOCTL support
Change-Id: Id85989d0ae1246cb60fff4a41e5847a2d84e271f
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8084
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-05-28 07:20:23 +00:00
Shuhei Matsumoto
924bfd067e bdev/nvme: Add ANA state per namespace into JSON info dump
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Idc0e783a2a4373a4269276aab8b1d857ed73caae
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7509
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2021-05-26 09:22:55 +00:00
Shuhei Matsumoto
8c2738a8fe bdev/nvme: Add UUID and EUI64 comparison to check if two namespaces are identical
Following Linux NVMe host, add UUID and EUI64 comparison to
bdev_nvme_compare_ns().

Besides, previously the return value of memcmp() had been used as
the return value of bdev_nvme_compare_ns() and this was wrong.
Fix it in this patch together.

Add unit test cases for bdev_nvme_compare_ns().

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I069ab53e77741d6348b847d51e84a9338e2f3787
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7755
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-05-26 09:22:55 +00:00
Jiewei Ke
eabe783cc4 bdev: speed up bdev name lookup by using rbtree
Use the macros for red black tree provided by Free BSD to speed up bdev
name lookup in spdk_bdev_get_by_name().

In the bdev_multi_allocation test, we can get 3x ~ 5x speed up when
creating multiple bdevs for various bdev nums.

Signed-off-by: Jiewei Ke <jiewei@smartx.com>
Change-Id: I49a2fbcccf06d4c36cbd445ce59e0b0dd4ada31d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7837
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2021-05-26 09:22:06 +00:00
ChengqiangMeng
f90eaf1672 unittest/nvme_ctrlr: increase code coverage for nvme_ctrlr_identify_namespaces_iocs_specific_next
nvme_ctrlr_identify_namespaces_iocs_specific_next code comprehensive coverage.

Signed-off-by: ChengqiangMeng <chengqiangx.meng@intel.com>
Change-Id: Icf039e15a983b04fcd1b558559ba651e495a6cd1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8006
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Community-CI: Mellanox Build Bot
2021-05-26 09:21:10 +00:00
Jiewei Ke
d07c581b7d nvmf: nvmf_get_ana_log_page should use iov to store the log page
nvmf_get_ana_log_page used req->data to store the log page result.
While the req->data only contains the first iov, if req->iovcnt is
larger than 1, the req->data may not hold the complete log page; and
even worse, the log page result may be written to invalid address and
cause memory corruption.

The following patch will fix the same issue for other commands in
nvmf_ctrlr_get_log_page.

Fix #1946

Signed-off-by: Jiewei Ke <jiewei@smartx.com>
Change-Id: I495f3be05c82be5cd53609772c655c8924b9179f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7923
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Community-CI: Mellanox Build Bot
2021-05-26 09:20:30 +00:00
Ziye Yang
2250abaeca nvme/tcp: Raname send_pdu to pdu in tcp_req.
Since we will reuse send_pdu for other purpose in the next
patch.

Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: Iee5166131b70a25bc13aaa847bfc9066231f31a9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8028
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Mellanox Build Bot
2021-05-26 09:20:15 +00:00
Ben Walker
dde419084b event: Move json config loading to init
Loading subsystems and restoring state from a JSON config file is useful
outside of the SPDK application framework, so move it to lib/init.

Change-Id: I7dd3ceace2e7b1b28eef83c91ce6a4eedc85740e
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6645
Reviewed-by: Tom Nabarro <tom.nabarro@outlook.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2021-05-26 09:13:34 +00:00
Tomasz Zawadzki
e65310624b lib/event: accumulate idle_tsc when reactor has no threads
Before this patch idle_tsc was sum of all idle tsc of all
threads running on a reactor.

There are cases when no threads are present on the reactor,
and _reactor_run() spins doing nothing.

To give more accurate representation of the reactors state,
the idle_tsc now adds time spent doing idle spinning.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: If797b2a03507d17b07367d56d5f6c40cefbbbd49
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7900
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2021-05-25 07:11:53 +00:00
Tomasz Zawadzki
7438c388a0 lib/event: fix tsc_last for first thread on reactor
tsc_last value is used to update thread stats
during _reactor_run(). See:
spdk_thread_poll(thread, 0, reactor->tsc_last);

If no threads were present on the reactor,
this value got outdated and resulted in
adding time reactor spent with no threads to
stats of the first thread placed on that reactor.

This patch fixes thread stats by making sure
that argument to spdk_thread_poll() is up to date.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I0c35fdba1b63b6ee19a5a2b34751090839cb2438
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7845
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2021-05-25 07:11:53 +00:00
Tomasz Zawadzki
320fdaa325 ut/event: add current reactor tsc accounting
This patch does not change the UT in functional way.

Added accounting of last_tsc and spdk_get_ticks() similar
to the real application.
First the reactor_run() [not _reactor_run()] starts
reactor->tsc_last to current time. UT now set it at the start
and never touch it again directly.

Second the spdk_get_ticks() is updated to the elapsed time
to simulate flow of time, and make sure that its further
usage is up to date to the current time.

Fixed typo in test case description for test_reactor_stats.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Iaa2eb00fa3bffc2f21f1692da0259f1d023086b8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7904
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Community-CI: Mellanox Build Bot
2021-05-25 07:11:53 +00:00
Ben Walker
af935f768e event: Shift subsystem initialization code to a separate library
This is useful for applications even if they elect not to use the SPDK
event framework.

This doesn't shift everything in one go - just the subsystem
initialization logic. Configuration file loading also needs to move
in a separate patch later.

Change-Id: Id419df1045442d416650ed90e5ee78adfdd623d7
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6641
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-05-24 10:12:50 +00:00
ChengqiangMeng
36cb16219e unittest/nvme_tcp: increase code coverage for some handle
nvme_tcp_c2h_data_payload_handle and nvme_tcp_c2h_term_req_payload_handle and nvme_tcp_icresp_handle code comprehensive coverage

Signed-off-by: ChengqiangMeng <chengqiangx.meng@intel.com>
Change-Id: Ib7ccc7dbab11990f851807c0b07e7c0e67e61bde
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7872
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2021-05-24 10:12:14 +00:00
Mao Jiang
f596245ce6 test/nvmf/ctrlr: cases for ctrlr creating and destruct
Change-Id: Ib4b3c124e6906e63409afc7e225d5afb89a28e76
Signed-off-by: Mao Jiang <maox.jiang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7612
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2021-05-24 10:11:52 +00:00
Ben Walker
15ae31fb0c nvmf: Rearrange spdk_nvmf_requset for cache line locality
Get all of the hot stuff to the first cache line.

* Shrink the xfer enum to one byte (it only has 3 values).
* Pull out the dif enabled flag form the dif structure so it
  can be access separately
* Rearrange the members

Change-Id: Id4a2fe90a49c055a4672642faac0028671ebfae9
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7827
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-05-24 10:10:19 +00:00
Mao Jiang
bb27314c03 test/nvme_pcie: cases for mapping io cmd
Change-Id: Icf174668e9b1361bb232db4a0cc53921423a17a4
Signed-off-by: Mao Jiang <maox.jiang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7671
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-05-21 08:04:39 +00:00
Evgeniy Kochetov
d8eb6957a0 ut/nvme: Add tests for NVMe controller namespace handling
Signed-off-by: Evgeniy Kochetov <evgeniik@nvidia.com>
Change-Id: Id2d3d1a15df22512f8d0d4414b5b3d35e79149f4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6505
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-05-20 15:17:41 +00:00
Evgeniy Kochetov
f0e248586e nvme/ctrlr: Always access namespaces via getter function
Signed-off-by: Evgeniy Kochetov <evgeniik@nvidia.com>
Change-Id: I16e327c0c0485057dc90a87cae316c4d6b62720d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6504
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-05-20 15:17:41 +00:00
Evgeniy Kochetov
65ff07719d nvme/ctrlr: Retrieve active NS list before NS construct
This patch changes the order of IDENTIFY_ACTIVE_NS and CONSTRUCT_NS
controller states. It is required to further improve memory management
for namespaces by allocating memory only for active ones.

Signed-off-by: Evgeniy Kochetov <evgeniik@nvidia.com>
Change-Id: Ie540442b1bd9e897afcbaa4319c139109dd0c515
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6503
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-05-20 15:17:41 +00:00
Evgeniy Kochetov
aebbce2520 nvme: Refactor active namespace list retrieval
Previous implementation allocated memory just once at the beginning of
active NS list retrieval procedure. It allocated memory for maximum
possible number of active namespaces, i.e. 'cdata.nn'.

This patch changes allocation logic. One page is allocated at the
beginning. If more is needed, reallocation is done with one more
page.

This patch also removes SPDK_MALLOC_DMA flag from allocation since we
don't do RDMA directly into this buffer.

Signed-off-by: Evgeniy Kochetov <evgeniik@nvidia.com>
Change-Id: Iaa80c4d70c54daaf71dcbf755c63a01a1d83b772
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6502
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-05-20 15:17:41 +00:00
Shuhei Matsumoto
4e9adb3bf9 thread: Replace TAILQ by Red Black tree for timed pollers
Use the macros for red black tree provided by Free BSD to manage
timed pollers efficiently.

Allow RB_INSERT() to insert elements with duplicated keys by changing
the compare function to return 1 if two keys are equal.

Check the return code of RB_INSERT() because this is the first use case
for RB tree macros in SPDK. We did the same for RB_REMOVE() by
adding another temporary variable but we remove it from this patch
because it is not so important compared with RB_INSERT().

When a timed poller is inserted, update the cache for the closest (leftmost)
timed poller only if the tree was empty before or the closest (leftmost)
timed poller was actually changed. We do not have to use RB_MIN()
because all duplicated entries are inserted on the right side.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ibe253ca8eecc10116548b5eedbcdba8fb961b88d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7722
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-05-20 15:13:27 +00:00
Shuhei Matsumoto
3ca15e33e4 thread: Set poller to PAUSING even when it is in RUNNING
This change is a preparation to first dequeue the closest timed poller
always when it is expired. Previously the poller_remove_timer() calls
were not consistent and difficult to follow.

spdk_poller_pause() sets poller to PAUSING even when it in RUNNING
and move it to PAUSED after returning from its context.

If spdk_poller_pause() and spdk_poller_resume() are called while poller
runs, it is moved to WAITING. Hence thread_execute_poller() and
thread_execute_timed_poller() ignore such cases.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I29340613a2ec0c3529d0886f4d81c0a0fdf8745d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7908
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2021-05-20 15:13:27 +00:00
Shuhei Matsumoto
50487701b1 ut/thread: Test if multiple timed pollers are expired simultaneously
Add a little complex test cases to avoid regression by the following
patches.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I1c94edd913f0a582868e355be085a1faf9bd1a94
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7913
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2021-05-20 15:13:27 +00:00
Jim Harris
f651e22a98 bdev: remove all references to spdk_bdev_open
This API was removed previously, so remove remaining
references in map file and unit tests.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iba2f6a5f5ba590d3996dc133c8181083a33d7405
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7963
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-05-20 15:02:59 +00:00
Mao Jiang
4e4d865fa0 test/nvme_cuse: cases for getting cuse ns device
Change-Id: I117b81554b6cf0f6eb1af7bd1050ff06cc351af9
Signed-off-by: Mao Jiang <maox.jiang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7442
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-05-19 07:50:53 +00:00
Alexey Marchuk
019a5361a5 nvmf/rdma: Use contig buffers for multi SGL payload
Currently we allocate buffers perf each SGL descriptor.
That can lead to a problem when we use NVME bdev with
PRP controller and length of the 1st SGL descriptor is
not multiple of block size, i.e. the initiator may send
PRP1 (which is SGL[0]) which end address is page aligned
while start address is not aligned. This is allowed by
the spec. But when we read such a data to a local buffer,
start of the buffer is page aligned when its end is not.
That violates PRP requirements and we can't handle such
request. However if we use contig buffer to write both
PRP1 and PRP2 (SGL[0] and SGL[1]) then we won't meet
this problem.

Some existing unit tests were updated, 1 new was added.

Fixes github issue #1853

Change-Id: Ib2d56112b7b25e235d17bbc6df8dce4dc556e12d
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7259
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2021-05-18 12:44:18 +00:00
Mao Jiang
75f9a150a6 test/nvme_pcie_common: cases for creating and deleting IO queue
Change-Id: I79237a63b0f3b414cb3300bf41ee6193ce35e60f
Signed-off-by: Mao Jiang <maox.jiang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7632
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-05-18 12:42:52 +00:00
Shuhei Matsumoto
4748ebef40 thread: Cache the closest timed poller into thread
When we introduce RB tree, getting the closest timed poller is not
O(1) but O(log N). To mitigate such delay, cache the closest timed
poller into thread, and update the cache when its content is changed.

Add unit test cases for this change. They will also clarify the current
behavior of spdk_poller_unregister() and spdk_poller_pause() for
timed pollers.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ibb98a54c261859a3210034038d3953e5c93ef8aa
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7720
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Community-CI: Mellanox Build Bot
2021-05-14 10:08:02 +00:00
Krishna Kanth Reddy
f795dd6e66 test/unit: Add unit test for Simple Copy Command
Signed-off-by: Krishna Kanth Reddy <krish.reddy@samsung.com>
Change-Id: Ia43f5d0588133027e80dc5707fd5940c44605c68
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7695
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-05-14 09:59:54 +00:00
Alexey Marchuk
3ebf25e020 test/unit/nvme: Use page size in nvme_pcie_qpair_construct_destroy
Some systems may have page size other than 4096 bytes

Change-Id: Id56bb1566fff480edbe1a293121fc7a68c74252a
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7820
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: <dongx.yi@intel.com>
2021-05-13 10:09:45 +00:00
Shuhei Matsumoto
6edc4abe4b thread: Change remaining direct accesses outside lib/thread to spdk_thread to helper functions
Use spdk_thread_get_last_tsc() and spdk_thread_get_stats() in
unit tests for reactor. As a result, we can change one inclusion
from spdk_internal/thread.h to spdk/thread.h.

This is the same effort as spdk_poller.

The following patches will move the definition of struct spdk_thread and
enum spdk_thread_state from include/spdk_internal/thread.h to
lib/thread/thread.c.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I24d4a52ecf885b6e7db749616ee1f95b82574dc5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7801
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-05-13 10:03:44 +00:00
ChengqiangMeng
744f401cc9 test/nvme_fabric: Remove priority assert check for rdma.
It's only for tcp now, but the transport here is rdma. So there is no need to assert.

Signed-off-by: ChengqiangMeng <chengqiangx.meng@intel.com>
Change-Id: Ieffaac9d31a8f9c79a0d9fd226d74268986b7a77
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7823
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
2021-05-12 08:48:28 +00:00
Tomasz Zawadzki
a8fe601853 lib/blob: delay executing blob persist completions
This helps in next patch in series where multiple
completions will be executing.

UT is adjusted since one additional poll is required.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Id72377ddef91e40cdbc2bdea6f33c23309b0ca3d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7642
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-05-11 12:03:57 +00:00
Tomasz Zawadzki
958364b4a0 lib/blob: only verify cluster allocation when I/O is not frozen
During snapshot creation the original blob becomes
a thin provisioned blob that will only the diff of data after
snapshot creation.

Despite the comment in the UT the number of polls before issuing
blob write was hitting blob BEFORE it swapped with new one.

Issuing I/O during this period shall check for io freeze
before checking cluster allocation.

Otherwise bs_io_unit_is_allocated() hits assert for thin
provisioned blob. This is because cluster map of blob is
empty, but properties have not been updated yet.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I742e1a50b14d456ae1e6de13b5111caec3e8322c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7641
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-05-11 12:03:57 +00:00
Changpeng Liu
d0b4deab2b UT/bdev: add unmap UT to cover the split case
Change-Id: I902c9ce113cfbb2549ecb720951403496dec6d06
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7579
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-05-11 12:03:28 +00:00
Mao Jiang
980a139631 test/nvmf/ctrlr_bdev: cases for identify namespace
Change-Id: I315f48185268da696e4047e1c7ac5cf2a4221765
Signed-off-by: Mao Jiang <maox.jiang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7285
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-05-11 12:00:15 +00:00
Shuhei Matsumoto
646f41476d ut/nvme_fabric: Fix access to local variable after returning from the function
Fix issue #1936

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Iaa2b8060053d3f798001a6b207a6fcb0335c4b23
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7833
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-05-11 11:58:46 +00:00
Jim Harris
53447f060f nvmf: remove a bdev_module.h #include
ctrlr_discovery.c doesn't need this #include.
Including it causes bdev_module.h types to be
emitted to the debug symbols at least with some
compilers, which can result in unwanted abidiff
errors.

The unit tests do need it, so just include it
there instead.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iad28f9778ce08b11b52325658583ae9032295f3a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7813
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-05-11 11:57:01 +00:00
Jim Harris
8a12e2e715 scsi: don't include bdev_module.h
The library itself doesn't need it. The unit tests
do need it, so just include it there.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I9aefd303ae12928d45141029436509f185105bd3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7812
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-05-11 11:57:01 +00:00
Curt Bruns
b89ad84532 nvme: add support for NVME_IOCTL_IO_CMD for cuse
Nvme-cli uses NVME_IOCTL_IO_CMDs for "io-passthru"
commands to cuse devices.  This patch adds support
for that IOCTL.

Signed-off-by: Curt Bruns <curt.e.bruns@gmail.com>
Change-Id: I20e0ac91ba08fce91bc5da1f4a1e454058cdd1e7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7741
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-05-10 11:13:09 +00:00
Curt Bruns
f3edd7a359 nvme: rename "admin" to "passthru" for cuse
The nvme cuse IOCTLs are actually creating passthru commands
that can be either IO passthru commands or admin commands.
Renaming the routines to correctly reflect that should limit
the confusion when reading the code.  Passthru commands that
are admin commands will go to the spdk_nvme_ctrlr_cmd_admin_raw
interface and passthru commands that are IO will be sent to the
spdk_nvme_ctrlr_cmd_io_raw interface.

Signed-off-by: Curt Bruns <curt.e.bruns@gmail.com>
Change-Id: I8d427fe8b5f503fdb2d193236c77d410d5b13886
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7740
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-05-10 11:13:09 +00:00
Ziye Yang
4e0ca20a8a idxd: Do not present pci device info in accel_engine_idxd.c module
Purpose: We will also support the kernel idxd driver, so we do not
need export this feature in the module file.

Change-Id: I965e031497920f527962ba187bccd81de6977b8f
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7336
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-05-10 11:12:40 +00:00
Ziye Yang
82e4bfd346 nvme/tcp: Change the type of recv_pdu to pointer.
This is prepared for using the hardware offloading
engine in accel framework. And some fields in nvme_tcp_pdu
needs to be DMA addressable.

Change-Id: I75325e2cd7ff25fe938bea0ac9489a5027e3e0e9
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7770
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2021-05-07 11:41:24 +00:00