Commit Graph

1201 Commits

Author SHA1 Message Date
Tomasz Zawadzki
ccdbd2ed51 nvme/cuse: fix nblocks and block_size in rw path
nblocks is zero based, so read path was missing the increment.

NVMe device that cuse represents can be of any block_size,
so rather than hardcoding 512 - actually verify it.
Both paths didn't request enough of a buffer from cuse.

Reported-by: Niklas Cassel
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I228dc2572bc94ecbcb913e950d912a7ab5be9434
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4037
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-09-04 08:13:56 +00:00
Tomasz Zawadzki
ef3f0d97ab nvme/cuse: refactor read/write to pass cuse_device and block_size
This patch does not alter functionality, just moves
around where cuse_device and block_size is determined.

Next patch will fix both paths.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I5a827b5b4ab080b2aa0f76f5cdcbcb177b38b474
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4036
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-09-04 08:13:56 +00:00
Seth Howell
58216dd07e lib/nvme: fix mem leak in req submit.
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: If64c06177605a8f57d87ba22b86fe58ddebd6f7a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3921
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Michael Haeuptle <michaelhaeuptle@gmail.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2020-09-02 07:38:38 +00:00
Jacek Kalwas
477d9500a3 env: fix pci driver registration
Registration macro now generates function based on driver's name.
It allows to have multiple registration within single source file.
Similar pattern is used e.g. by SPDK_NVMF_TRANSPORT_REGISTER.

Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: Ied0887e8dae7fe9ca1517313be5eff8f218b7e98
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3895
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-08-27 08:36:43 +00:00
Seth Howell
309200661f lib/nvme: pass up rc from failed connect in reset.
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I8041728de7bb7bdb35fb1570f3c2fb20feef8664
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3877
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-08-26 09:47:24 +00:00
Seth Howell
316f92d118 lib/nvme: pass up fabric connect rc to app.
This will allow applications to understand why
they were unable to connect.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: Ic04c7e72098c6ec1823de7d6a07d90150ef5ac20
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3836
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2020-08-26 09:47:24 +00:00
Jim Harris
178291a516 nvme: read all uevents
Currently when the uevent processing code finds a non-uio/vfio
uevent, it just stops its loops and returns.  This means that if
there are a lot of non-uio/vfio uevents, the netlink socket buffer
can build up until its full because only one non-uio/vfio event
gets drained per spdk_nvme_probe() call (which may be very
infrequently).

So modify parse_event so that it does not indicate error when
a non-uio/vfio event is found.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ic8a40f71ee89d597ce46129eac889fe5b7ef5171

Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3876
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-08-24 07:38:33 +00:00
Ziye Yang
d4d2e317b5 nvme/tcp: Make the return value consistent.
We should make nvme_tcp_ctrlr_connect_qpair always return
negative value if this function fails.

Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I457e704e39d7a3acd298fd48e89e8ea51e2ed4ad
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3809
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-08-24 07:37:13 +00:00
Changpeng Liu
5adf099eed nvme: don't send Identify NS ID Descriptor List to inactive NS
The specification says it will return INVALID FIELD if the NS
is in inactive state.

Fix issue #1551.

Change-Id: I1b32f023ed665d410f4705e439068699e2b2f8de
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3860
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-08-21 08:45:31 +00:00
Alexey Marchuk
8bec9feb76 nvme/rdma: Remove unused spdk_nvme_send_wr_list
nvme_rdma_qpair::sends_to_post is not used, remove it and
spdk_nvme_send_wr_list structure

Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Change-Id: If9c42736d4e796a947bbfe80f59efd2fd7f77859
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3822
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-08-21 08:24:43 +00:00
Seth Howell
ce83fc2aff lib/nvme: remove qpair from ctrl list in connect fail path
This is an oversight that can cause issues with looping
through the list if we end up allocating the same qpair
twice.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I513ea35398f4b724366c21be144531fbfbdb4347
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3835
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-08-21 08:24:18 +00:00
yidong0635
361cddfd63 nvme/nvme_pcie: Remove unused codes.
pctrlr->cmb.mem_register_addr and pctrlr->cmb.mem_register_size
are assigned after spdk_mem_register.
if spdk_mem_register is failed , ctrlr_map_cmb hasn't been executed.
they are not be used.
So remove them.

Signed-off-by: yidong0635 <dongx.yi@intel.com>
Change-Id: I3d1996eee8b5260b79c4c3e0a2e1d376da2343b7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3856
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-08-21 08:24:00 +00:00
Changpeng Liu
5d5a9077a3 nvme: don't assert on custom transport
Change-Id: I2d425c127dc070f7bb508f5a61e6304f6042fdf7
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3857
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-08-21 08:23:19 +00:00
Ziye Yang
0d3cc15a62 nvme/tcp: Correct the incapsule data usage
According to page35 in recent NVMe-oF spec (
NVMe-over-Fabrics-1.1-2019.10.22-Ratified), ioccsz is used
to restrict the incapsule size of I/O command, so do not
restrict the NVMe-oF OPC command and also the admin command.

We accidently trigger an bug in kernel since we do not send
the fabrics command with the incapsule and make the kernel
coredump, though the kernel has bugs.

Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I869a2c8ab7b9c2ac1e5cc5b603920662591c2c64
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3837
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-08-20 09:26:06 +00:00
Ziye Yang
2031f8f70d nvme: set the error code if we cannot send keep alive command.
If the transport is broken, we should set errno code in
spdk_nvme_ctrlr_process_admin_completions instead of keeping silence.

Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: Ie73763e1329e12a8c82a0223d360991f86c39be3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3773
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-08-19 07:29:26 +00:00
Seth Howell
518a1e013a lib/nvme: make fabrics connect timeout configurable.
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: If829d399882ef948d95673c17e5689c91386c21d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3795
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-08-19 07:29:19 +00:00
Seth Howell
b3bb3a1bbf lib/nvme: change timeout in wait_for_completions to usec
This allows for much more granular control over the timeout.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: Ib23de21e60eec4207c55320579699edf284f4e16
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3794
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-08-19 07:29:19 +00:00
Tomasz Kulasek
549b9f31c6 lib/nvme: implement SPDK_NVME_DATA_NONE data transfer in CUSE
Change-Id: Ifb2a53bdbaabd74b7f412923a97d79b44afde861
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1744
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2020-08-12 10:39:29 +00:00
Seth Howell
a22f55658c lib/nvme: add timeout to fabric connect commands.
We can't afford to wait here blocking forever. Add a
reasonable timeout (1 second) to this command.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I7e373c5515d42a46403f03d7d09e72225f8910ff
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3563
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-08-10 11:27:16 +00:00
Ziye Yang
7bac9b06f1 nvme TCP: Make the control related pdu also allocated from the SPDK DMA memory
Purpose: To make the pdu management consistent with other PDUs, then
we can easily adapt our code into some hardware offloading solution.

Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: Ic4a2847fd1b6cacda4cbaa52ff12c338f0394805
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3588
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-08-04 18:28:08 +00:00
Seth Howell
0d8f86f842 lib/nvme: don't submit request if qpair is disconnected.
This becomes a problem when the qpair is reconnected.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I6677b396cf766684a4891ffbee93aa3e4e83374d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3391
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-07-31 08:23:44 +00:00
Seth Howell
0b1799cd98 nvme/transport: addd assert for transport.
Silences a KW error.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: Ifd8d6088a22de7c230d48751be2b3991d0649778
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3553
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-07-29 07:37:26 +00:00
Jim Harris
751e2812bc nvme: do not abort reqs in multi-process cleanup path
When a process cleans up IO qpairs from another crashed
process in a multi-process environment, we must not try to
abort reqs for that IO qpair.  Any reqs will contain callbacks
for the crashed process which we must not try to execute in
a different process.

Fixes issue #1509.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I5e58cce7bdb86e3feb4084733815c086901f867e

Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3536
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-07-27 22:42:07 +00:00
Darek Stojaczyk
cdffd22581 env_dpdk: move NVMe PCI driver definition to the nvme lib
Now that drivers can be registered from upper layers there's
no need to keep them centralized inside env.

(check_format.sh complains that spdk_pci_nvme_get_driver() shouldn't
start with the spdk_ prefix - to workaround that we move the function
declaration from one place in env.h to another - that's enough to
convince check_format it really is a public function)

Change-Id: If86aebd6c997349569c71430ec815b413eb44ef8
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3187
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-07-24 21:42:26 +00:00
Seth Howell
c9bc86c061 lib/nvme: add checks for new states in reconnect_io_qpair.
This function hasn't kept up properly with the states that
we use for tracking the qpair lifecycle.

Add checks for NVME_QPAIR_DISCONNECTING and NVME_QPAIR_DESTROYING.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I51607d4f00e94937b08fca28e766163580d46461
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3359
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-07-24 15:43:07 +00:00
Darek Stojaczyk
45528bfef6 env: add spdk_pci_id->class_id
This follows struct rte_pci_id which had class_id as well.
We'll need it to make some additional DPDK APIs public through
the env abstraction.

Change-Id: I794a6cd6b17e48daf53b48fa5abe3d3dcfeaa403
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3182
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-07-23 20:48:47 +00:00
Darek Stojaczyk
64c4fafd32 nvme/pcie: dont initiate hotremoval on SIGBUS for VFIO devices
This is an attempt to workaround a kernel data race spotted in #1275.
We have to delay our VFIO interraction until after the kernel handles
the PCIe hotremoval. Otherwise we risk having a deadlock inside the
kernel. We could put a dummy delay in SPDK hotremove routine, but we
don't know exactly how long we have to wait, so wait for the VFIO
hotremove notification to arrive instead. From what I understand it
is always sent after the kerner already releases the critical locks.

This patch breaks hotremove for VFIO devices when running with
DPDK version < 18.05, but those reached EOL already and their support
is removed from SPDK later within this series.

It would be nice to wait for this VFIO notification inside lib/env_dpdk
e.g. inside the detach function, but we don't know there why the
detach is happening. It could be triggered by a user RPC, not physical
device hotremoval.

Change-Id: I2727de923bb2e3e05a83202465330a32a77d7cfc
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1732
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-07-23 20:48:47 +00:00
Ziye Yang
1da44e0604 nvme_tcp: Move the default buffer factor size in nvme_tcp.h
1 Change the default factor from 4 to 8, which can be used
to improve the performance.

2 Change the base buffer size in nvme_tcp.c,
we should not use sizeof(struct spdk_nvme_tcp_cmd),
it is 72 bytes. Normally, the initiator will receive
C2h pdus and R2T Pdus by most, so set the size of using
sizeof(struct spdk_nvme_tcp_c2h_data_hdr) is enough.

Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I384f4cb026cb8d83e75b639f7256ee8cb8ed1df1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3283
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-07-22 12:21:07 +00:00
Shuhei Matsumoto
71da42e953 lib/nvme: Add spdk_nvme_ctrlr_cmd_abort_ext() to abort commands whose cb_arg matches
A new API spdk_nvme_ctrlr_cmd_abort_ext() gets cmd_cb_arg as a
parameter, and use it to abort requests whose cb_arg matches cmd_cb_arg.

The caller can set the parameter qpair to NULL if it wants to abort
requests on admin qpair.

Hold ctrlr->ctrlr_lock throughout because abort request is submitted
to admin qpair.

The API is not critical for performance, and so initialize parent
data first.

The API is for a specific qpair, and so hold SQID in the parent,
and it is copied to the children. On the other hand, CID is set
to child when request to abort is found.

Use an new helper function nvme_transport_qpair_iterate_requests()
to add abort request for each outstanding request which has
cmd_cb_arg as its callback context.

The case is possible such that the request to abort is not outstanding
yet but queued. Hence abort queued requests which has cmd_cb_arg
as its callback context too, but it is done only if there is no error
so far.

If only queued requests are aborted and there is no outstanding
request to abort, complete with success synchronously.

If there is no outstanding or queued request to abort, return -ENOENT.

When any abort request is submitted, the difference between success
and failure is only bit 0 of CDW0 according to the NVMe specification.

We cannot the existing helper functions nvme_request_add_child() and
nvme_cb_complete_child() but can use nvme_request_remove_child().
nvme_qpair_submit_request() may use only nvme_request_remove_child()
from these three helper functions. Hence we use req->parent as other
types of request do.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I3a271c6542f8e2e6b425b3bf6151f41e924bc200
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2039
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-07-14 07:54:42 +00:00
Shuhei Matsumoto
576a373d58 lib/nvme: Abort queued requests whose cb_arg matches
Use another list dedicated to hold queued requests being aborted
to avoid potential infinite recursive calls.

Add a helper function nvme_qpair_abort_queued_req() to move requests
whose cb_arg matches from qpair->queued_req to qpair->aborted_queued_req.
Then nvme_qpair_resubmit_requests() aborts all requests in
qpair->aborted_queued_req.

The first idea was that nvme_qpair_abort_queued_req() aborts queued
requests directly. However, this caused infinite recursive calls.
Hence separate requesting abort to queued requests and actually
aborting queued requests.

The detail of the infinite recursive calls is as follows:

Some SPDK tool submits the next request from the callback to the completion
of a request in the completion polling loop. For such tool, if the callback
submits a request and then aborts the request immediately, and the request
could not be submitted but queued, it will create infinite recursive calls
by request submit and abort, and it will not be able to get out of
completion polling loop.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I8196182b981bc52dee2074d7642498a5d6ef97d4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2891
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-07-13 08:40:42 +00:00
Alexey Marchuk
e137881e4e nvme/tcp: Insert free req at the head of the list
lifo model is more cache friendly

Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Change-Id: Id937ab0c1b8b4ce121136144c7d6013bbe5eb963
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3282
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-07-13 08:40:31 +00:00
Jacek Kalwas
4d9ab1e9a1 nvme: pretty print dptr
Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: I576878fbbafc3d17617ceeec99e40565be7d5d3d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3213
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-07-10 07:30:59 +00:00
Jacek Kalwas
64f05eb5c5 nvme: pretty print fabric cmd
Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: Ib4bc28026cab208d45c8b876714fa525e5bb38f3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3200
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-07-10 07:30:59 +00:00
Jacek Kalwas
9cd4723913 nvme: pretty print set/get features
Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: Ib6f1811da9a6294983bce04cff01ba1fb5e45607
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3179
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-07-10 07:30:59 +00:00
Jacek Kalwas
41b360d54e nvme: add missing cmds to admin opc string
Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: I6cdcf675ebc8ad31d88b5469f87e1eae066b2e3c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3178
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-07-10 07:30:59 +00:00
Jacek Kalwas
61668cc43e nvme: introduce new set of cmd/cpl printers
Having functions without qpair on the interface allows for wider usage
e.g. by nvmf layer.

Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: I3a51ad53f00eb29e2ba2681ef4ff0cc2a197b65d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3176
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
2020-07-10 07:30:59 +00:00
Ziye Yang
4c9aad0299 nvme/tcp: Report the free entries if sending_ack is set
Previous we fix the same issue in this commit:
cb98b2ab3e

But we forget to fix it here. And we also need to update here,
otherwise we will still face the same issue described in
commit: cb98b2ab3e

Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I3660dbb6e97c92ea4cb347cfce4bf23c6dfe97ab
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3242
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-07-09 07:23:19 +00:00
Jacek Kalwas
42f2f01a0e nvme: fix timeout log during ctrlr state change
Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: Icf6741f059ed92fdf0ee9fd0952acdd07711c69c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3226
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-07-08 07:54:29 +00:00
Shuhei Matsumoto
d80c9f6257 lib/nvme: Add underscore prefix to nvme_qpair_abort_queued_reqs()
This is a preparation to the next patch.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I15356c69e676dc41d3af69caa6d12c1fcb282152
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3071
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Michael Haeuptle <michaelhaeuptle@gmail.com>
2020-07-08 07:54:01 +00:00
Shuhei Matsumoto
f2bd635ecf lib/nvme: Add qpair_iterate_requests() to iterate the common operation among transports
To abort requests whose cb_arg matches, add child abort request greedily.
Iterating all outstanding requests is unique for each transport but
adding child abort is common among transports, and adding child abort
is replaceable by other operations.

Hence add qpair_iterate_requests() function to the function pointer table
of transport, and pass the operation done in the iteration by a
parameter of it.

In each transport, the implementation of qpair_iterate_requests() uses
TAILQ_FOREACH_SAFE() for potential future use cases.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ic70d1bf2613fce2566eade26335ceed731f66a89
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2038
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Michael Haeuptle <michaelhaeuptle@gmail.com>
2020-07-08 07:54:01 +00:00
Shuhei Matsumoto
aa2ea2bed5 nvme/rdma: Follow the fix in TCP transport and restore nvme_rdma_req_put()
Recently two patches were merged but we should have get more reviews.
The fix done in TCP transport will be better because we can keep
the existing functions and make the code change minimum.

Restore nvme_rdma_req_put() and move removing rdma_req from
rqpair->outstanding_reqs to nvme_rdma_req_complete(). One exception
is the case that only nvme_rdma_req_put() is called. For the case
remove rdma_req from rqpair->outstanding_reqs before calling
nvme_rdma_req_put().

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I3f68dbc88c60af6b8f4ecc3209fde9b763ac3189
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3073
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Michael Haeuptle <michaelhaeuptle@gmail.com>
2020-07-08 07:54:01 +00:00
Shuhei Matsumoto
ad69e739e1 nvme/tcp: Dequeue request from outstanding list before calling completion
Each request has a callback context as cb_arg, and the callback to
nvme_complete_request() for the completed request may reuse the context
to the new request.

On the other hand, TCP transport dequeues tcp_req from
tqpair->outstanding_reqs after calling nvme_complete_request() for
the request pointe by tcp_req.

Hence while nvme_complete_request() is executed, tqpair->outstanding_reqs
may have two requests which has the same callback context, the
completed request and the new submitted request.

The upcoming patch will search all requests whose cb_arg matches to
abort them. In the above case, the search may find two requests by
mistake.

To avoid such error, move dequeueing tcp_req from tqpair->outstanding_reqs
before calling nvme_request_complete(). One exception is the case that
only nvme_tcp_req_put() is called. For the case remove tcp_req from
tqpair->outstanding_reqs before calling nvme_tcp_req_put().

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I5f2ac292c60431ac1e27b8657db92b220860a0a8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2865
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Michael Haeuptle <michaelhaeuptle@gmail.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-07-08 07:54:01 +00:00
Shuhei Matsumoto
e060285ea6 nvme/tcp: Change nvme_tcp_req_complete() to take tcp_req instead of req
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ida0ee76015821d7db54b273d14383a245a18047b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3058
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-07-08 07:54:01 +00:00
Ziye Yang
449dee3563 nvme/tcp: Fix the sending conflict between cmd and h2c pdu.
As is well known, we may also handle the r2t data pdu receving earlier
before calling the cb function of send_cmd due to the outof order execution
of the lower layer uring socket interface.So we need to fix
this issue, otherwise the data of the sending_pdu will be placed with the
wrong data.

And it will cause the issue shown in
https://github.com/spdk/spdk/issues/1473

Fixes #1473

Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: Idac1ad65761695f3a655b85003861c1d1f4f3875
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3215
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-07-07 07:31:53 +00:00
Jin Yu
d76951c7ba nvme_rdma: fix the recvs_to_post.first to NULL
nvme_rdma_qpair_submit_recvs is not judged in
nvme_rdma_poll_group_process_completions path.
If we do not clean the recvs_to_post.first we
may get the wrong current_num_recvs when the rc
is non-zero and call it again.

Change-Id: If0046e711525dcfcb419132a01fed7a09db13ba0
Signed-off-by: Jin Yu <jin.yu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3163
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-07-06 07:20:33 +00:00
Jin Yu
05805e54a0 nvme:disconnected state then destroying state
Put the destroying state after the disconnected state.
Because nvme_transport_ctrlr_disconnect_qpair will modify the state
of qpair to disconnected, and in the path of rdma, it will postpone
the deletion of qpair until the release of pg by judging the
destroying state. So qpair is not deleted.

Change-Id: Ica606905cddf67d0ffda14bd48cc5f4e424f01ee
Signed-off-by: Jin Yu <jin.yu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3136
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-07-06 07:20:26 +00:00
Jin Yu
19228a0602 nvme_rdma:fix current_num_sends to current_num_recvs
Change-Id: I1a3067165c06db3fe7d7fd1c1ec149e845100b27
Signed-off-by: Jin Yu <jin.yu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3162
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-07-06 07:20:26 +00:00
Simon A. F. Lund
ae6c40a4a5 lib/nvme: fallback to NVM when then given command_set is not supported
When the user requests a non-default command set configuration, via the
probe_cb() supplied to spdk_nvme_probe(), which is not supported, then
the controller initialization will not proceed.

This patch changes that behavior into falling back to the NVM command
set and continue with the controller initialization. It is done by
assigning the NVM command set to opts.command_set such that the user
knows in attach_cb() with which command_set the controller is
configured/enabled.

The fallback is needed since the user does not have access to the
controller capabilities register. The strategy left for the user is thus
to try. However, this is an issue, as the user only has one attempt, as
subsequent calls to spdk_nvme_probe() will not trigger probe_cb() for
the controllers whose initialization did not proceed.

Signed-off-by: Simon A. F. Lund <simon.lund@samsung.com>
Change-Id: Ia414628fcd7d56956649647775462e62d98c0a90
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2931
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-07-03 07:31:31 +00:00
Jim Harris
7ff9609f3d nvme: add security send/receive quirk for Intel 900P SSDs
These SSDs set the oacs.security bit but do not actually
support OPAL.  So do not set the controller flag indicating
SECURITY_SEND_RECV support in this case.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I7fcfeafcc8d9439a1c53c60a1aea1801923a2ce5

Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3156
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-07-03 07:31:20 +00:00
Jim Harris
16364fdc14 nvme: copy 8 bytes at a time into virtualized CMB
QEMU won't emulate wide (>8 byte) instructions on an MMIO
region.  We tried to fix this previously but it wasn't sufficient.
So this version adds an explicit copy function that will only
copy 8 bytes at a time from the source cmd pointer into the
SQ cmd point in the virtualized CMB.

Fixes issue #1458.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I804005a2e8cf3f7365c1608674f82a3f843e59ba

Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3135
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-07-02 07:21:38 +00:00