For nvme/tcp connection, we use the synced manner
if the qpair is not fully connected. Thus without
the check, we will stuck here. And this patch
fixes this issue.
Change-Id: I72815bf5b4c0b31c4866bc1b9034b0e42b81d3f1
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8025
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Community-CI: Mellanox Build Bot
The header size is very small, which does not have too much value to
offload such calculation by hardware.
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: Iaa82f39312df7eef3282325a33677ea41ab735ab
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8011
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This patch changes the order of IDENTIFY_ACTIVE_NS and CONSTRUCT_NS
controller states. It is required to further improve memory management
for namespaces by allocating memory only for active ones.
Signed-off-by: Evgeniy Kochetov <evgeniik@nvidia.com>
Change-Id: Ie540442b1bd9e897afcbaa4319c139109dd0c515
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6503
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Previous implementation allocated memory just once at the beginning of
active NS list retrieval procedure. It allocated memory for maximum
possible number of active namespaces, i.e. 'cdata.nn'.
This patch changes allocation logic. One page is allocated at the
beginning. If more is needed, reallocation is done with one more
page.
This patch also removes SPDK_MALLOC_DMA flag from allocation since we
don't do RDMA directly into this buffer.
Signed-off-by: Evgeniy Kochetov <evgeniik@nvidia.com>
Change-Id: Iaa80c4d70c54daaf71dcbf755c63a01a1d83b772
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6502
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Nvme-cli uses NVME_IOCTL_IO_CMDs for "io-passthru"
commands to cuse devices. This patch adds support
for that IOCTL.
Signed-off-by: Curt Bruns <curt.e.bruns@gmail.com>
Change-Id: I20e0ac91ba08fce91bc5da1f4a1e454058cdd1e7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7741
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
The nvme cuse IOCTLs are actually creating passthru commands
that can be either IO passthru commands or admin commands.
Renaming the routines to correctly reflect that should limit
the confusion when reading the code. Passthru commands that
are admin commands will go to the spdk_nvme_ctrlr_cmd_admin_raw
interface and passthru commands that are IO will be sent to the
spdk_nvme_ctrlr_cmd_io_raw interface.
Signed-off-by: Curt Bruns <curt.e.bruns@gmail.com>
Change-Id: I8d427fe8b5f503fdb2d193236c77d410d5b13886
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7740
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Nvme-cli uses BLKSSZGET so support needs to be added for
nvme cuse devices.
Signed-off-by: Curt Bruns <curt.e.bruns@gmail.com>
Change-Id: Ic8316713b2d017c8ff32a225efff6bcb95842799
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7708
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
It is useful to have debug log information in the nvme_cuse
path when debugging IOCTls and flows.
Signed-off-by: Curt Bruns <curt.e.bruns@gmail.com>
Change-Id: Ifef1bb82c96438e2fcbb9ad2fafe3f3eb66bed51
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7707
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
The purpose is to prepared for implement the async crc32 caculation
in the future patch.
Change-Id: Ia75f28154c49f08b527d48c63b9da79a6bdfede8
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7794
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
This is prepared for using the hardware offloading
engine in accel framework. And some fields in nvme_tcp_pdu
needs to be DMA addressable.
Change-Id: I75325e2cd7ff25fe938bea0ac9489a5027e3e0e9
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7770
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
When payload_size is 0, we may get wrong cdw10 because of the calculate: 0 - 1,
add value check to fix value inversion bug.
Signed-off-by: sunshihao <sunshihao@huawei.com>
Change-Id: I3bcd38ba981c854ff917282341d32aac47d22b76
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7443
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
We have seen that dptr was not alligned to 4k using cuse. Added allignment of data in cuse ctx to 4k same as it is done in nvme_allocate_request_user_copy
Signed-off-by: jwyka <jakub.wyka@intel.com>
Change-Id: Ic5c2482eae20d64ba467016eb61f5255467f70a9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7453
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Broadcom CI
spdk_vtophys() takes a mapping_length parameter, so
it can return the length for which the returned
virtual address is valid.
But spdk_vtophys() will only return the max
between the valid length and the input mapping_length
parameter.
So the nvme SGL building code for contiguous buffers
was broken, since it would only set the mapping_length
once, before the loop started. Worst case, if a buffer
started just before (maybe 256 bytes) before a huge page
boundary, each time through the loop we would create
a new SGL for only 256 bytes at a time, very quickly
running out of SGL entries for a large buffer.
Fixes#1852.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ib1000d8b130e8e4bfeacccd6e60f8109428dfc1e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7659
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
The IDENTIFY_CNS quirk was applied as part of QEMU
OCSSD handling in commit 6442451b. But it was applied
not only to the OCSSD dev ID, but also the dev ID
for non-OCSSD NVMe controllers.
Starting with QEMU 5.2, QEMU will allocate a default
256 namespaces, but only some are active (associated
with the backing disks specified by the user). QEMU
supports IDENTIFY_CNS, but since this quirk was set,
we wouldn't send a real IDENTIFY_CNS and instead
would just populate a fake list where all namespaces
were considered active. This causes breakage in
a few places - mainly where we iterate through
the active namespaces, and then are surprised that
calling spdk_nvme_ns_is_active() returns false.
It was also breaking bdev_nvme_attach_controller RPC,
since by default we can only support returning 128
names, but since all of the namespaces were deemed
active, it was trying to return 256.
Fixes#1916.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I4fdd27e0e36f0ac07a95f9f29aa83357e8505a45
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7658
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
The io_msg qpair is allocated and managed by the
primary process, so don't try polling it from
secondary processes.
This fixes a bug where an SPDK target has configured
cuse, and we try to run fio (for example) as a
secondary process.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I48e2b89597196ce2ba1fc02ea3a7c76c5a33281a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7482
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: <dongx.yi@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
The P55XX serial SSDs can support dlfeat.read_value in the identify
namespace data structure, we don't need to add this quirk for it,
just remove it.
Change-Id: I165d89085e246a570e80dbaf05f41dc331b93f0c
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7526
Community-CI: Broadcom CI
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
When the format command is issued, the kioxia drives responds with "NS Attr change" notices.
In the callback function of the notice, the CQ Head Doorbell is updated twice with the same
value while issuing the Active NS list & identify NS commands.
Fixes: #1701
Signed-off-by: G.Balaji <gbalajieie@gmail.com>
Change-Id: I8cc80fba0a226c22753e605ef3129602a9313ce7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7149
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Also update the UT.
Change-Id: I6086bf4cafca8a917a467490955d7df0ba8930d5
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7495
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Previously we can only remap NVMe command using PRP, now we add
the SGL support.
Change-Id: Iec352d858a07bdd3d5f261336d6fa1167ba7aa79
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7279
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
The API `spdk_nvme_map_prps` is used in nvmf/vfio-user to
remap VM's NVMe command data buffer to local virtual address,
and for command using PRP, there maybe multiple pages, when
parsing the PRP list to local IOVs, we need a parameter to check
that the maximum number of vectors can't exceed the IOVs, this API
can't meet the requirement, while here, we add a new API `spdk_nvme_map_cmd`
and with a new parameter `max_iovcnt` to fix this case, and it can
also cover the command using SGL in the coming patches.
Change-Id: I71063524bed16ee3434103867a556d3741e55326
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7278
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Change spdk_nvme_map_prps to a internal fucntion with
a new parameter `max_iovcnt` to protect the IOVs. Also
for the purpose to keep API compatibility, we still leave
the API here.
Change-Id: I9a638beb87aab20bba5f8a4fa0a9396110d56aff
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7335
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
If the qpair is part of a poll group, the socket will get
flushed as part of polling that group already. We only need
to explicitly flush to handle the case where the qpair is
not in a poll group.
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: Ib2a510b6d26d1622950437d81e0a40f6b15d6b54
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7049
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
There was a fix for this that went into the posix layer, but the
underlying problem is the logic in the nvme/tcp transport. Attempt to
fix that instead.
Change-Id: I04dd850bb201641d441c8c1f88c7bb8ba1d09e58
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6751
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
This prepares for an upcoming patch to fix issue #1701 which
requires handling async events outside of the check
completions loop.
Fixes: #1701
Signed-off-by: G.Balaji <gbalajieie@gmail.com>
Change-Id: I4985d814903143511383172b1a443580db33a78f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7416
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Change-Id: I2e47b148209ce4c232dbdc5f20c90548be995e1a
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7334
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
NVME_MAX_PRP_LIST_ENTRIES has changed over time, so let's
just remove the reference to the exact value here. Also
explain a bit more why the max size isn't
(NUM_ENTRIES + 1) * page_size.
While here, do a small whitespace cleanup as well.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ib75813788abdd3dbb43192f9fdc27f99b33aeadf
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7328
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Check if qpair has a poll group during the connect process,
use poll group's statistics or allocate own structure per
qpair. That is done due to not all applications use poll
groups and we want to avoid "if (qpair->group)"
conditions in data path.
Admin qpair always allocates its own statistics
structure but the statistics are not reported
since this qpair is not attached to a poll group.
Statistics are reported by spdk_nvme_perf tool
if --transport-stats and in bdev_nvme_transport_statistics
RPC method.
Change-Id: I58765be161491fe394968ea65ea22db1478b219a
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6304
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
These are interface functions that can be used by
an application e.g. spdk_nvme_perf or bdev_nvme
library. The next patches will add usage of these
functions.
Change-Id: I33b88e0e713c2ea5967f9241885e3257c5070577
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6300
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
The new 2 API function allow to get and free stats
per poll group. New function to get transport name
have been added to report not only transport type but
also the name.
For now only RDMA transport reports statistics,
other transports will be added later.
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Change-Id: I2824cb474fde5fa859cf8196dabac2c48c05709c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6299
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
New statistics include number of poller calls,
number of idle polls and total number of completions.
These statistics allow to estimate % of idle polls
and the number of completions per poll.
Since nvme_rdma_cq_process_completions function
returns number of completed NVMF requests and each
NVMF request consumes 2 RDMA completions (send+recv),
this function was extended to return the number of
RDMA completions.
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Change-Id: Ifdc1e2e467f645adb5d66d39ff2a379e161fbd77
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6298
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
These statistics allow to estimate WRs batching
efficiency. The number of send WRs equals the total
number of submitted NVME commands.
Change-Id: I96c9836cd6b9070cf5f62e43b4d2738506866e94
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6297
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
The hotplug lib can be used for pcie devices
such as nvme, virtio_blk and virtio scsi.
For the sigbus handler, there is only one in a
process and it should handle all the devices.
And align nvme to the hotplug lib
Add the ADD uevent support for allowing the
device hotplug.
Change-Id: I82cd3b4af38ca24cee8b041a215a85c4a69e60f7
Signed-off-by: Jin Yu <jin.yu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5653
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Introduce new utilities NVME_CTRLR_ERRLOG, NVME_CTRLR_WARNLOG and so
on to output the ctrlr's identification at different log levels.
For RDMA and TCP, the subnqn will be output and for PCIe and custom,
the traddr will be ouptput.
Change-Id: I81a112463bf752999aa1fe4e0c867d88e09a2f64
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7057
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Vasuki Manikarnike <vasuki.manikarnike@hpe.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
When a request is submitted, it may have incorrect iov
alignment that doesn't fit PRP requirements. In the
current version an internal function fails such a request
and returns a NULL pointer. This is mapped to -ENOMEM
error which is returned to generic bdev layer where
such a request is queued in a "nomem_io" queue and
later can be resubmitted. That is incorrect and such
a request must be completed immediately. To fail the
request, we need to differentiate between -ENOMEM and
other cases, so we pass a pointer to a result to
local nvme functions
Change-Id: I7120d49114d801497a71fca5a23b172732d088de
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7036
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This function will be used the next patch, current
behaviour remains unchanged
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Change-Id: Ie74c7395f67a08b0cac018eb5114f358a6b583cb
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7092
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
The specification for Maximum Data Transfer Size (MDTS) says this field
should include the length of metadata, if metadata is interleaved with the
logical block data. However, some drives can support MDTS without counting
the interleaved metadata, so for this case SPDK will only use data length
without interleaved metadata length.
Change-Id: I29920a25885699e2689be043b87122367be0e416
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6813
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Remove the polling group check. Because at this moment,
the qpair is not added into a polling group. If we do
not remove it, we will never enable zcopy feature for
I/O qpair.
And in sock implementmentation, we already fixed the zero copy
handling if a socket is not in a polling group. See
posix_sock_flush function. So we can fix this issue if we directly
remove this check.
Reported by: Aleksey Marchuk <alexeymar@mellanox.com>
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I969936c4b6c7f13cbfa4d6eb479010c53f3e384a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7056
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
This API has been deprecated since SPDK 20.07,
see commit (b2947f52).
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Idb45906c81ea5682c6a67def0265910266d861b5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6629
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Community-CI: Mellanox Build Bot
This patch refactor the pdu sending logic with the async manner,
then if the group contains the accel engine, we can use it.
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I2d669c0a3255d7a8898441e406906add2f3a3556
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6759
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Purpose: To setup an accelerated function callback
for created spdk_nvme_poll_group. In this patch,
we just create the interface. The real usage of this
call back will be provided in the other patch.
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I0d936aa4eba4dbfcc0137942156b9f2919eb5b78
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6758
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
It is invalid to try to delete a NULL qpair, so do
not check for it in nvme_tcp_ctrlr_delete_io_qpair and
return an error when NULL. Just change it to an
assert instead. This makes it consistent with pcie
and rdma.
While here, add an assert in rdma as well.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ic2f76deecb21b78749dac85e33fb1fa0d14a1239
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6917
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: <dongx.yi@intel.com>
The shadow registers need to be zero when the qpair is
created. This happens automatically when a given qid
is used for the first time, since the page is allocated
with zmalloc. But if a qid is reused, we need to make
sure its shadow registers are cleared *before* we create
the qpair again with the same qid.
So clear the registers in nvme_pcie_ctrlr_delete_io_qpair,
just after the cq is deleted.
Fixes issue #1795.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I08c30d1ea248559a01b802cd132dd57199b491b5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6752
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Previously the callback parameter for this function is NULL, this will
cause segment fault, so pass the correct parameter here.
Fix#1817
Change-Id: Ie768b7bf4a72862d16a44742ab3032803d0939a2
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6690
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Community-CI: Mellanox Build Bot
For the following nvme controller statemachine states:
NVME_CTRLR_STATE_IDENTIFY_NS
NVME_CTRLR_STATE_IDENTIFY_ID_DESCS
NVME_CTRLR_STATE_IDENTIFY_NS_IOCS_SPECIFIC
The statemachine can either:
- Jump to succeeding state
- If active ns list is empty, jump directly to NVME_CTRLR_STATE_CONFIGURE_AER
- In the unlikely case if NVMe completion error, jump to NVME_CTRLR_STATE_ERROR
Simply this such that we either:
- Jump to succeeding state
- In the unlikely case if NVMe completion error, jump to NVME_CTRLR_STATE_ERROR
This will help to reduce the complexity of the nvme controller statemachine,
especially considering that there are new additional states
(NVME_CTRLR_STATE_IDENTIFY_NS_DIRECTIVE and
NVME_CTRLR_STATE_CONFIGURE_NS_STREAMS) currently on review that would continue
with the bad habit of having three possible jump states instead of just two.
Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Change-Id: I3242052b1108afcd8adbe6d0378b1358fef58ec8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6521
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: sunshihao <sunshihao@huawei.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Purpose: To get the optimal group, we need the socket information.
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I17b048a402fbf002307dd225f64b20a9f876d642
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3324
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Community-CI: Mellanox Build Bot
For a SGL using PRPs, there is always an alignment check of the start
address in the beginning of the loop. This is stored in start_valid.
If the start is indeed valid, we might fetch a new SGE,
and then perform a second alignment check on this new SGE.
However, this second alignment check is done unconditionally,
meaning that for the last SGE in a request, we check if the
same start address is aligned twice.
Only perform the second alignment check if we actually fetched
a new SGE.
Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Change-Id: I9df8038c650b0879f838d1d9d895e8dd7172840d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6493
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Add a function to get the number of max active zones for a zoned
namespace.
The value inside the identify namespace struct is a 0's based value,
where 0xffffffff means unlimited.
If unlimited, the addition will overflow and return 0,
which is the intended value to represent unlimited for this API.
Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Change-Id: Ia09e3db157ca0afadbd3ca4032eedd7bcd88248c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6443
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: sunshihao <sunshihao@huawei.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Add a function to get the number of max open zones for a zoned
namespace.
The value inside the identify namespace struct is a 0's based value,
where 0xffffffff means unlimited.
If unlimited, the addition will overflow and return 0,
which is the intended value to represent unlimited for this API.
Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Change-Id: I4223146bc1ddf90486892a0af5fe5ce006dc5fd3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6442
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: sunshihao <sunshihao@huawei.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
We already have support for spdk_nvme_zns_zone_append(),
add support for spdk_nvme_zns_zone_appendv() (zone append with
NVME_PAYLOAD_TYPE_SGL).
_nvme_ns_cmd_rw() currently performs verification of the SGL,
if the parameter check_sgl is set. This parameter is set for all
calls with payload of type NVME_PAYLOAD_TYPE_SGL.
In order to be able to perform the same check_sgl verfication on
zone append vectors, we need to refactor _nvme_ns_cmd_rw() a bit.
Setting check_sgl ensures that _nvme_ns_cmd_split_request_sgl() or
_nvme_ns_cmd_split_request_prp() gets called.
These functions will split an oversized I/O into several different
requests. However, they also iterate the SGE entries, verifies that
the total payload size, total SGE entries is not too many, and that
buffers are properly aligned. A proper request will not get split.
For zone append, splitting a request into several is not allowed,
however, we still want the verification part to be done, such that
(e.g.) a non first/last SGE which is not page aligned, will cause
the whole request to be rejected.
(In the case of spdk_nvme_ns_cmd_write(), a non first/last SGE which
is not page aligned will instead cause the request to be split.)
An alternative would be to try to rip out the verification part from
_nvme_ns_cmd_split_request_sgl() and _nvme_ns_cmd_split_request_prp().
However, that is non-trivial, and would most likely end up with a lot
of duplicated code, which would easily get out of sync.
Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Change-Id: I2728acdcadeb70b1f0ed628704df19e75d14dcca
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6248
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Use the new function spdk_nvme_zns_ns_get_zone_size_sectors() where
it is appropriate (in comparison to the existing
spdk_nvme_zns_ns_get_zone_size() variant).
Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Change-Id: Ic929ffbc5a1f4a16ba6719a985c05ae625caed46
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6417
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Sometimes it is more optimal to get the zone size in number
of sectors, instead of in number of bytes.
Therefore, add a new spdk_nvme_zns_ns_get_zone_size_sectors()
function to get zone size in number of sectors.
Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Change-Id: I0fe67e00a3d74dd27acfc895ae97448d995b89a3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6416
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Identify application prints the PMR details if it is supported
Signed-off-by: Krishna Kanth Reddy <krish.reddy@samsung.com>
Change-Id: Iaba4c15e18e1402035b11a34b2defe8078855751
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6209
Community-CI: Broadcom CI
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
There is a special case when using 8-byte metadata + PI + PRACT
where no metadata is transferred to/from controller.
Since _nvme_ns_cmd_rw() already calculates the proper sector size
using _nvme_get_host_buffer_sector_size(), which takes PRACT into
account, change the sectors_per_max_io calculation to also take
PRACT into account.
This will avoid certain requests that don't need splitting getting
split.
Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Change-Id: I8d450d37c2458453701189f0e0eca4b8fe71173b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6247
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Zone append commands cannot be split.
_nvme_ns_cmd_rw() should never cause a NVME_PAYLOAD_TYPE_CONTIG
zone append request to be split.
This is currently true, but add an assert to make sure that
any refactoring to _nvme_ns_cmd_rw() does not break this promise.
Also add error handling, since release builds are built with
asserts disabled.
Follow-up patches will refactor _nvme_ns_cmd_rw().
Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Change-Id: I5fd2440c4c9d6bd8d56f30354b208a9047b64729
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6246
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
This function allows applications to specify whether
they wish to allow probing a newly attached NVMe
PCIe SSD.
The env layer will only even probe devices that have
been allowed. By default, this is all devices, but
if the user has specified some list of
allowed PCI addresses (via spdk_env_opts pci_allowed)
then newly attached PCIe devices are implicitly not
allowed. This API allows applications to add
device addresses to the allowed list after the
application has started.
This API will be useful for use cases where multiple
SPDK processes are running on one server, and assignment
of PCIe SSDs to those processes are based on some function
of the SSD's PCIe address.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I919bc267f2ad9130ab5c875ff760a301028b047e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6184
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Fixes#1777.
When a qpair cannot be allocated because the transport connection fails,
the qpair was freed without unlinking it from the other structures.
This was leading to a segfault when attempting to create and free other
qpairs.
Also added a unit test to cover this case.
Change-Id: I74b78d1847f90117248b07203b43a11ff5cfa5d6
Signed-off-by: Vasuki Manikarnike <vasuki.manikarnike@hpe.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6272
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Reviewed-by: Michael Haeuptle <michaelhaeuptle@gmail.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
If rdma_qp_disconnect is not correctly sent out, we will not wait
for the event.
Change-Id: I99701e421dc93909d481ccf35e9bfd8004e60da8
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6163
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Add the real support in nvme tcp transport.
Change-Id: I2aa9b0284d6fe009925e67f602a055e787f77987
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5734
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
This patch is used to add spdk_nvme_poll_group_get_optimal
public API.
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: Iee34c89e0e1ff1f81167b18e198c144ca28f71de
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3311
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
The data path for PCIe and vfio-user transports are almost
same too, so move the code from nvme_pcie.c to nvme_pcie_common.c,
so that these APIs can be reused by vfio_user.
No logic change for this patch.
Change-Id: I82f480bba3bae0ce35e2a98f29839081095f7d50
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6040
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Intel P55XX SSDs can support SGL feature but can't use Dataset Management
command with SGL format, so add a quirk here for now, if the limitation was
fixed in future, we can remvoe this. Also SPDK doesn't privoide scatter buffer
API for DSM, so using PRP with DSM is totally fine.
Change-Id: Ibe92f4deb5b8bc2077115f5b7244bc17be4f3b23
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5858
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
No need these returns at the end of void functions.
So remove them.
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Change-Id: I8889745f3ef82af513d03259a77a33c1f4f536cb
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6015
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
The Zone Append command is an optional command in the Zoned Namespace
Command Set.
Zone Append differs from a regular write, in that the command is not
given an exact LBA of where to write the data.
Instead the user has to set the zslba field to the start of a zone,
and the data will be appended to that zone.
The actual LBA where the data was stored is returned in the
spdk_nvme_cpl, where Dword0 contains 31:00 of the ALBA field,
and Dword1 contains bits 63:32 of the ALBA field.
Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Change-Id: Iabae1b3456bfbb62c07b63d79afe9a14e460fe83
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6013
Community-CI: Broadcom CI
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Create a _nvme_get_host_buffer_sector_size helper function,
to avoid the same code being duplicated in several functions.
Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Change-Id: I8c14683c683a44e03c97eefa186833831f754bcc
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6035
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
It does not make sense to send in check_sgl == true,
when we are calling _nvme_ns_cmd_rw() with a payload
of type NVME_PAYLOAD_CONTIG.
_nvme_ns_cmd_rw() simply cannot "check SGL" if the payload
is not a SGL. Doing so regardless just makes the code harder
to read.
We still send in check_sgl == true, when we are calling
_nvme_ns_cmd_rw() with a payload of type NVME_PAYLOAD_SGL.
Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Change-Id: I56d49a2abf7819d20cf5974c9e0df8f04f1ccd10
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6009
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
In the next patch this member will be used to track
both read and write offsets
Change-Id: I852125ff35257f9821ddf4a641d96afb29ebf0a0
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5924
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
When NVMf target linked with vfio-user library, we can use
vfio-user client library to connect to the target.
Here is the three examples that can work with target:
identify -r 'trtype:VFIOUSER traddr:/var/run/muser/domain/muser0/8' -g
perf -r 'trtype:VFIOUSER traddr:/var/run/muser/domain/muser0/8' -g -q 1 -o 4096 -w read -t 10
reconnect -r 'trtype:VFIOUSER traddr:/var/run/muser/domain/muser0/8' -g -q 32 -o 4096 -w randrw \
-M 50 -t 10 -c 0xE
You can run the following test script test/nvmf/target/nvmf_vfio_user.sh to have a quick test,
currently enabled with NVMe Identify,Perf,Reconnect tools.
Change-Id: Ieb9842b2f372184fffbf7f23e4aad26feb47c350
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3839
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Libvfio-user assumes the memory translation is IOVA=VA mode,
since SPDK CI is running inside a VM, the memory mode is
IOVA=PA mode, so when testing NVMe vfio-user transport inside
a VM spdk_vtophys doesn't work with libvfio-user, so here
we add a function to return memory address based on TRTYPE.
Change-Id: I11d1c87197f7bbfc243b6bf368795c9a74bd1303
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5958
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
There are some common data structures and APIs in pcie transport
which can be used both for pcie and vfio-user transport, so move
the common code into a new header and source file.
No actual logic change just the code movement except remove the
static function declarations.
Change-Id: Ie9021e703a5780fdd6840f0e3cfea76a0017a811
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5923
Community-CI: Broadcom CI
Reviewed-by: sunshihao <sunshihao@huawei.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
The new custom transport can enable NVMe driver running with
NVMe over vfio-user target.
Change-Id: I5f90e8516eaca08fc3eab658b29b760a03326ff7
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5996
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Add two async API for Directive Send and Directive Receive.
spdk_nvme_ctrlr_cmd_directive_send;
spdk_nvme_ctrlr_cmd_directive_receive;
Signed-off-by: sunshihao <sunshihao@huawei.com>
Change-Id: Icb6974f74902df1512a5ffa9835188132634291b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5803
Community-CI: Broadcom CI
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
According to kernel, use an inline function spdk_nvme_bytes_to_numd
to transfer paload_size form bytes to numer of dwords.
Signed-off-by: sunshihao <sunshihao@huawei.com>
Change-Id: I8b9ded122bbf4a3c8e46988993ea52404783c0b0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5926
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Allocate memory with zero number or size, maybe return a unique
pointer rather than NULL. Add a check before common allocation APIs.
Change-Id: I83e07cab5145035e705bc32364652be90f238633
Signed-off-by: Mao Jiang <maox.jiang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5809
Reviewed-by: GangCao <gang.cao@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Current version provides unclear output
Change-Id: Ib044b00b5f91b1e363911f1b79c51c73c8a6920c
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5743
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Here "return rc == -EINPROGRESS ? 0 : rc;"
They are the same meaning in these two functions.
Keep the comments here. This makes more clear to readers.
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Change-Id: I8590de3f0fe27337163ee8b02ea63e166f1bbe7c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5689
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
As log shows
00:06:32.300 [2020-12-18 21:13:35.511014] nvme_ctrlr.c:1414:spdk_nvme_ctrlr_reset: *ERROR*: Controller reinitialization failed.
00:06:32.300 [2020-12-18 21:13:35.511104] nvme_ctrlr.c: 925:nvme_ctrlr_fail: *ERROR*: ctrlr 192.168.100.8 in failed state.
00:06:32.300 [2020-12-18 21:13:35.511132] bdev_nvme.c: 392:_bdev_nvme_reset_complete: *ERROR*: Resetting controller failed.
00:06:32.300 [2020-12-18 21:13:35.511240] nvme_ctrlr.c: 925:nvme_ctrlr_fail: *ERROR*: ctrlr 192.168.100.8 in failed state.
00:06:32.300 [2020-12-18 21:13:35.511511] bdev_nvme.c: 556:bdev_nvme_failover: *NOTICE*: Unable to perform reset, already in progress.
if spdk_nvme_ctrlr_reset() failed, nvme_ctrlr_fail() is called, and
then if spdk_nvme_ctrlr_process_admin_completions() failed,
nvme_ctrlr_fail() is called.
We don't know which one comes first but nvme_ctrlr_fail() should do
nothing if the ctrlr is already failed.
Hence we should avoid setting ctrlr->is_failed and calling
nvme_transport_ctrlr_disconnect_qpair() twice.
However we should set ctrlr->is_removed if the parameter hot_remove is true.
We do these changes in this patch.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Iac37c892e054fb59d78e69346ca7f0575d596235
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5694
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
In some extreme use cases, an SPDK process could get
swapped out for a long period of time just after
we checked the state but before we called spdk_get_ticks().
So now we will only timeout if the timer expired before
we checked the state *and* the state did not advance.
It's possible we could just move the timeout check
to before the ctrlr->state switch, but I was
hesitant to change the flow for this case.
Fixes issue #1720.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I95b1db3365b5d2d8a65e528f53c302a724d44460
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5596
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
When a device is removed, we should use the remove_cb
that was specified when the device was originally probed
and attached, if one was set.
Also add a new spdk_nvme_ctrlr_set_remove_cb API. This
can be used for cases where a different remove_ctx is
desired than was specified for the probe call. This
also enables setting a remove_cb when using connect APIs
which do not have a way currently to provide a remove_cb.
This also requires fixing the bdev nvme module, which
was depending on the previously errant behavior.
Fixes issue #1715.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Id848b39040099ff7a21fe57ea6b194a8c25ae015
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5510
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
spdk_vtophys() already checks that, so we don't need
to check it in the NVMe driver again.
Change-Id: I74288ae8cab80e1be34583475fa02a3ae13e090c
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5166
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
