it is possible that some specific transport doesn't support
NVMF_MAX_ASYNC_EVENTS (although it is a recommended value by spec)
with that change it is possible to reduce aerl on transport specific
layer so it can be advertised correctly during identify controller
Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: Ife6465b5324fb39f9b343c6f42b860e9dd1164b2
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10422
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Not every transport requires accept poller - transport specific
layer can have its own policy and way of handling new connection.
APIs to notify generic layer are already in place
- spdk_nvmf_poll_group_add
- spdk_nvmf_tgt_new_qpair
Having accept poller removed should simplify interrupt mode impl
in transport specific layer.
Fixes issue #1876
Change-Id: Ia6cac0c2da67a298e88956734c50fb6e6b7521f1
Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7268
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
On aarch64 platforms, doorbells update from guest VM may not be seen
on SPDK target side. This is because there is memory type mismatch
situation here. That is on guest VM side, the doorbells are treated as
device memory while on SPDK target side, it is treated as normal
memory. And this situation cause problem on ARM platform.
Refer to "https://developer.arm.com/documentation/102376/0100/
Memory-aliasing-and-mismatched-memory-types". Only using spdk_mb()
cannot fix this. Use "dc civac" to invalidate cache may solve this.
Profiling data did not show big performance degradataion.
Signed-off-by: Rui Chang <rui.chang@arm.com>
Change-Id: I9a18718f8c4307b3007b18c32ab02e6796548958
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10222
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
If a subsystem has no listeners, then there is no need
to update the discovery log when adding a host, or setting
a subsystem to allow all hosts.
This eliminates some unnecessary discovery log update
notifications, especially when setting 'allow any hosts'
on a subsystem immediately after it is created (and before
it has any listeners).
Update unit test to check the adding a host to a
subsystem without listeners does not rev the genctr.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I63dab5df564269e574bb925890088f52063aa378
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10546
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
The discovery log isn't updated when a subsystem is created
or deleted, it's only updated when a listener for a
subsystem is added or removed.
So remove the nvmf_update_discovery_log() in the subsystem
create and delete paths. They just generate extra AER
completions that potentially cause the host to do unneeded
work.
Note that if a subsystem is deleted with active listeners,
the subsystem delete path will remove each of the listeners
before deleting the subsystem itself. So the discovery log
will still get updated when those listeners are removed.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Id01bbfa3b24d3e1279a614a2fd60be41387a03b1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10545
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
The nvmf library will use INTEL VID/SSVID/IEEE values by default,
each transport can overwrite them if needed.
Change-Id: I9dad521c4d080b6f0cc1aaeb4b5d5f6863c6846d
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10095
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
When NVMe passthru command (IO or admin) fails on submission (e.g. it
is not supported), set DNR bit in completion status field. There is no
sense in retrying the command in this case.
Signed-off-by: Evgeniy Kochetov <evgeniik@nvidia.com>
Change-Id: I55960c128bd9fc31f6defef0b9832259a71684b1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8578
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
If NVMe admin passthru command is not supported by underlying bdev,
set status code in NVMe completion to INVALID_OPCODE.
Signed-off-by: Evgeniy Kochetov <evgeniik@nvidia.com>
Change-Id: I29c4e1f8263b76b27c199cfd2d9b2474432ec70b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10517
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
in case of failure groups shall be destroyed
Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: I8933f4128a7a3361bbb55d6a9c08a540521e5bda
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10435
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Doorbell offset starts from 0x1000 is defined by the NVMe
specification, so rename it to remove `VFIO_USER` prefix.
Change-Id: Ie34b12b3d2618f9b0ad0cf7ccbb103ad2c900f47
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10364
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Calculate supported maximum number of queue pairs based on
BAR0 size, this value isn't allowed to change at runtime, also
define BAR4/5 based on number of MSIX vectors.
Since the maximum number of queues is a large value(512), so we
still define a default value when starting, users still can
overwrite this value with a number no greater than 512.
Change-Id: I1b4b6bdf2ff9d129c8bdd493ffdf0a51f8772d51
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10334
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
libvfio-user will save a copy inside the library.
Change-Id: If7bb052b03fb92e46abe50fa945b812d149ef01d
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10363
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
adding transport to tgt should be the last step
also there is an issue before change i.e. if calloc failed then
transport remains on the list
Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: Iaf29cfc7b0f535d40160c6fdf9ef6a7e6bfb127c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10429
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
When no optimal poll group exists for a qpair,
assignment for round robin happens in spdk_nvmf_tgt_new_qpair().
RDMA transport implments the logic for this assignment in
nvmf_rdma_get_optimal_poll_group().
TCP relied on the spdk_nvmf_tgt_new_qpair() instead.
This resulted in race condition when looking up and assigning
optimal poll groups - see #2113.
To remedy that, TCP now follows the same pattern as RDMA.
Next patch will improve the sock map lookup to fix the #2113.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I672d22ac15d06309edf87ece5d30f8e8d1095fbb
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10270
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I58794b7b946eeb8ff82512905af0a296e3b534aa
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9817
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Add a parameter which determines the owner of the
map - target or initiator. It allows to set different
access flags when creating Memory Regions
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Change-Id: I0016847fe116e193d0954db1c8e65066b4ff82bf
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10283
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
The pointer to struct spdk_nvmf_ctrlr is used to save mandatory
controller registers to the migration region.
Also rename some ctrlr/qpiar to vu_ctrlr/vu_qpiar.
Change-Id: Ifcb862bf4543a9df3c62d3b0a57b7f93228ccaba
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7629
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Thanos Makatos <thanos.makatos@nutanix.com>
1. use the transport lock to protect transport endpoints list.
2. don't use mixed errno and -1 as the return value, use -1 for all error cases.
Change-Id: I657fa06a6d82ee8dbeefaa3397df2285ba574b80
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9579
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
When destroying controller, we will disconnect each connected qpair,
and in the spdk_nvmf_qpair_disconnect() call, qpair_fini() will also
try to hold the same lock, so existing vfio-user implementation assume
that qpair_fini() will not be called in the same context. Patch
https://review.spdk.io/gerrit/c/spdk/spdk/+/8963 remind me that
vfio-user has this issue. While here, we add one more thread poll
to avoid such issue.
Change-Id: I83b82ddcce3eb54c724291223e794dcb53a08059
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9998
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
For live migration support in vfio-user transport, we need to pause
the subsystem when starting migration in source VM, then after
migration, the subsystem is in paused state, when exiting the
application, we will call spdk_nvmf_subsystem_stop() at last,
and existing code will assert this case.
Change-Id: If5214c45973b27f6092c4a6d71ede336e54d89e8
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9407
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
There is no need to sum SPDK_CEIL_DIV(length, io_unit_size) and
req->iovcnt as the later is always zero (assignment in spdk_nvmf_request_get_buffers).
Checking SPDK_CEIL_DIV(length, io_unit_size) is enough.
Signed-off-by: Maciej Szulik <maciej.szulik@intel.com>
Change-Id: I0fea1d86706d83e4dd9083e6f0ce09e4e4b033a8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10089
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
The specification says "host specifies an offset (i.e., LPOL and LPOU)
that is greater than the size of the log page requested, then the
controller shall abort the command with a status of Invalid Field
in Command."
Offset is used (if needed) to retrieve specific records of
Discovery Log Page, so we don't check it for Discovery Log Page.
Change-Id: I76ce929600b9f2ca9b69397d25f339d55729e6d3
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10093
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This can be used for multipath validation.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iba62c5e90b22a9a85078fbc783d3a7273c029cde
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10137
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
CAP.CQR (Contiguous Queues Required) is always 1, so we should
return invalid field when PC bit is 1 and return Invalid Interrupt
Vector if Interrupt Vector is too big.
Also fix the issue that just creat/delete a CQ.
Change-Id: I7cc64a5946a1ed3161448fca8b433d08e5fee715
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10080
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
As per the nvme specs,
If OPTPERF is set to ‘1’ indicates that the fields
NPWG, NPWA, NPDG, NPDA, and NOWS are defined for this namespace and
should be used by the host for I/O optimization
Setting NPWA, NPDG, NPDA same as NPWG and NOWS same as MDTS
Fixes#2197
Signed-off-by: Swapnil Ingle <swapnil.ingle@nutanix.com>
Change-Id: Ic769a21b6821fa731eeae83e7d30c380e8092e37
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10007
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
User may configure opts.max_qpairs_per_ctrlr, so use
opts.max_qpairs_per_ctrlr instead of using fixed default
NVMF_VFIO_USER_DEFAULT_MAX_QPAIRS_PER_CTRLR.
Also do not allow users to configure max_qpairs_per_ctrl >
NVMF_VFIO_USER_DEFAULT_MAX_QPAIRS_PER_CTRLR.
Signed-off-by: Swapnil Ingle <swapnil.ingle@nutanix.com>
Change-Id: Id9f1f796559f3bb8b2d3f5031606050470681b99
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9994
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: John Levon <levon@movementarian.org>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
We generally shouldn't do ERRLOGs based on bad
inputs from the host, so change some of these to
DEBUGLOGs instead.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Id42a8e51964e0238cd2841be8ac1d4ccaa1d150d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10037
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
When doing controller reset and shutdown, we may change the
CSTS.RDY and CSTS.SHN even there are pending IOs in the IO
queues, so here we add a timer in the reset and shutdown
callback, it will change the status when there are no
connected IO queues.
Fix#2199.
Change-Id: I3a54d30b257973661b269ad5e37637490f9390f4
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9908
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
SPDK nvmf target reports all listeners on all subsystems
in discovery pages, kernel target reports only subsystems
listening on a port where discovery command is received.
NVMEoF specification allows to specify any addresses/
transport types. Ch 5: The set of Discovery Log entries should
include all applicable addresses on the same fabric as the
Discovery Service and may include addresses on other fabrics.
To align SPDK and kernel targets behaviour, add filtering
rules to allow flexible configuration of what should be
listed in discovery log page entries.
Fixes#2082
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Change-Id: Ie981edebb29206793d3310940034dcbb22c52441
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9185
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
This reverts commit 3b1f13ef29.
It seems like this particular commit is causing failures on the
CI side related to the following issue:
https://github.com/spdk/spdk/issues/2214
Reverting for now to make the CI stable.
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I7f32c612b8549477de0b9079d114cec4ec9bda59
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9986
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
There is a race condition between controller destruction and
subsystem state change, e.g. admin qpair may already be freed
when a namespace is added or removed. As result in function
poll_group_update_subsystem we may get heap-use-after-free error
Another problem is that some qpair's live time may exceed controller's
life time. To avoid it, start controller destruction process when the last
qpair finished the disconnect process (previously controller started
the descruction process before the last qpair starts to disconnect
and it could lead to raise conditions)
Fixes#2055
Change-Id: I11612979eb914a5fdcc7b9e3c812bf1e450b6120
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8963
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Matt Dumm <matt.dumm@hpe.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Some of our tpoints have a name exceeding 24 characters.
Althought this is not problem for SPDK, it might cause
confusion, because error messages are printed.
Tpoints regstered inside fc.c had their _REQ_ part removed,
since it was used in all of them.
Fixes#2208
Change-Id: I598eb9c1d252d8ca6c83f82e564a6b53037936f4
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9963
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
This patch makes use of the changes from previous patch and to
show connections between bdev events and tcp events.
Change-Id: If28c256d74a9a5d581ee4d8292a08fc061fee968
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9622
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@gmail.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
nvmf_ctrlr_cc_shn_done() and nvmf_ctrlr_cc_reset_done() are
almost same, so consolidate them together by adding a flag.
Change-Id: Ib7714d31a40f9d8d344ec2630c083c5d76dac8a1
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9907
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
It's too strict to fail the controller when there are no free requests.
Change-Id: I0a66ff2d294a2fd9326506ea50af4213aaaf8e92
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9848
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: GangCao <gang.cao@intel.com>
We already support Set Features with Host Behavior, so here also
add the support in Get Features.
Change-Id: I27d973e81fd6be89cc67ad559439334fc1087c9e
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9818
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: GangCao <gang.cao@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: John Levon <levon@movementarian.org>
nvmf_subsystem_get_qpairs RPC handler may cause the program
could not exit normally (e.g. ctrl c). Reason is that,
spdk_get_io_channel() will be called during the getting
qpairs stream, which will add refcount value for each
existing channel. When end target, channel cannot be
destroyed since refcount be added additionally and
its value could not be subtracted to 0. As a result,
the program will hang in the process of exiting.
So here we don't need to allocate a new channel, just use
the exist one.
Signed-off-by: zhaoshushu.zss <zhaoshushu.zss@alibaba-inc.com>
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Change-Id: I08b4678edaa9404b8e8af125ebae572b31edf77e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9881
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This is to help with binding trace objects together and
for the convenience (all trace definitions are in one place
instad of being scattered accross multiple files).
Change-Id: Ib15bc9c2eeee9c4d0816bcee509ab69f3f558e19
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9574
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@gmail.com>
Enable tracing of tcp qpairs in lib/nvmf/tcp.c.
Change-Id: I692e74a972dcddd0bff193f1703470926e28b4db
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9288
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@gmail.com>
This change aims to help with tracepoint handling in the future.
Instead of printing each trace directly with given values, we
will use this function to pass status change value.
Change-Id: Icc7f2863703899f818f0a2d5f49b69aa4e26a62c
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9690
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@gmail.com>
For GET/SET FEATURES command, some feature IDs will have data buffer
while some don't have, so here we will return the buffer length base
on the feature ID. The length is defined by the specification.
Change-Id: Ie9798585fc74544b77998aeebc2a20614c1c25f0
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9689
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
It is a bit confusing for the variable names to not match
the parameter names for spdk_bdev_io_get_nvme_fused_status().
These changes should make it a bit simpler to understand.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I833bf2a75cfec7b86da2d8a234f800de3510c2b7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9867
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
seems it is not required, string is used instead enum
Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: Ic67af6cfa6f8e22a6593f904cd5b5c1b51cae6ec
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9623
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
The spec treats the sizes (MQES or qsize from create/delete
IO queue command) as a 0-based value of uint16_t, but vfio-user
treats them as 1-based value, so we need to use uint32_t to
make sure the value can't overflow. The same for NLB(number of
logical blocks).
Change-Id: I7654b7e12234525c0fce78a713dd50097e9b3d58
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9632
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
The function is used to check the IO SQ or CQ exists or not,
so return bool type is better and also rename it.
Change-Id: I1af2a25255b012dcf1e1952b828617ededf9b897
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9680
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Andreas Economides <andreas.economides@nutanix.com>
Previously vfio-user will hit an error when creating a deleted IO SQ.
For lookup_io_q() function, actually we should not use the queue's
address to check the queue is exist or not, as when there is a memory
removal, we will also set the queue's address to NULL and reset it again,
so here we use the queue state to indicate the queue is exist or not.
Fix issue #2174.
Change-Id: I56bdd63947c2b30a598c427e64ff2dafe1cc1d2e
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9605
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Previously we only use an assertion to address this sceanrio.
Fix issue #2173.
Change-Id: I7a6e715977218d2a3a08c48a9935880f3fe4ec63
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9604
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Andreas Economides <andreas.economides@nutanix.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
According to the specification, we should also post an AER
error event for this error case.
Fix#2171.
Change-Id: Ifb2343453ea5e36ce244938a939537ee6ed1c4e1
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9584
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
We still don't support get log page with error
information LID.
Change-Id: I92db361dc956ea3ed4f6e7bdfdca763d0fea6886
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9583
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Similar is already done for json-rpc bdev_get_bdevs, it might be
useful for the upper layer which has no interest in all but only
in one specified.
Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: Ie1af1cb4778edd265914bbfdc2777f66c6c76572
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9362
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Signed-off-by: John Levon <john.levon@nutanix.com>
Change-Id: Ifa8e80e0a25af7757181f480ab0405ec902a61ff
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9596
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
IO commands with invalid OPCs are not freeing the
associated request object after handling the response.
This would eventually result in requests on the qpair
becoming exhausted which ends up failing the controller.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I7c1c46265a38b31181cd5d9a98c528816ab482d3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9601
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This commit fixes a race condition when calling free_ctrlr(),
nvmf_vfio_user_close_qpair->free_qp will set controller `ctrlr->qp[qid] = NULL`
finally, when calling free_ctrlr() we also need to check `ctrlr->qp[qid]`
is NULL or not, when there are multiple IO queues, we need a lock to protect
`ctrlr->qp[qid]`. However, the call to free_qp() in free_ctrlr() is valid
only when killing SPDK target, for all other cases, e.g: VM disconnected,
the queue pairs are already freed, so here we can process these different
cases separately, and avoid extra lock.
Change-Id: I7ab71f08bf4d737843b2af42e27b1571be0b45e9
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9351
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Thanos Makatos <thanos.makatos@nutanix.com>
Ideally, SPDK should make sure no pending I/Os in this queue
pair are using the removed memory region. Currently we just
stop the submission path and leave a TODO comment here until
we have an asynchronous way to do this.
Also use the `<=` for the boundary check.
Change-Id: I63a2189022978811dc21f92f2599f28a5191ecd7
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9352
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
lookup_io_q() should return NULL when qid == 0 (admin queue). This
ensures that handle_del_io_q() won't delete the admin queue (which is
prohibited by the spec) and fixes#2172.
Also fixes a few related off-by-one errors.
Signed-off-by: Andreas Economides <andreas.economides@nutanix.com>
Change-Id: I7ab063f25bba45b755d84c9ddde82072cf01f5e8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9593
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
We can avoid extra iteration to null queue pairs by using a list.
Change-Id: Idebd5a396959372004c33f8b21c555032af67aae
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9350
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Existing vfio-user assumes that the IO SQ/CQ are paired, so we
return an error when creating SQ with shared CQID.
Leave a TODO comment, we may support this in future.
Change-Id: I297ce53eb9a119469145e2453177d03252b629c8
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9173
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Thanos Makatos <thanos.makatos@nutanix.com>
Some spaces needed to separate some words from each
other.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I8a3f5b0a848ca9cc8771c11545652c275c194e89
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9547
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
The host driver should do a wmb() before it updates the SQ tail doorbell
to ensure that any writes to the SQ are guaranteed to be visible if a
doorbell update is visible (a store-release) - and indeed, this is what
the Linux NVMe driver does.
Therefore, we require a rmb() after we read the tail doorbell in order
to synchronise properly with the host driver (we need a load-acquire),
and guarantee that the updates to the SQ are visisble to us.
Signed-off-by: Andreas Economides <andreas.economides@nutanix.com>
Change-Id: I57eb1e1f0dfb1091a8f10f40f8ee0e2604d9268c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9514
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
libvfio-user will assert if the command didn't have data
buffers.
Fix issue #2124.
Change-Id: I43e9562c614bc65e54dbf650447369317c10f7b5
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9264
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Thanos Makatos <thanos.makatos@nutanix.com>
Reviewed-by: John Levon <levon@movementarian.org>
When removing a listener, there's a small window when the listener is
already freed, while the controllers that were created on that listener
are still active. It happens, because the listener is removed before
disconnecting its qpairs and in turn destroying the controllers.
The ctrlr->listener pointer is now cleared when a listener is freed and
its value is checked for NULL before each use.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Iace65a46eebc5972cc4ef0f1c1822ffc5d3e559e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9237
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
An error might occur after succesful transport creation
when the new transport is added to nvmf poll groups, e.g.
in nvmf_transport_poll_group_create. In that case
transport is not detroyed and poll groups are not fully
functional. To correct this behaviour, destroy transport if
spdk_nvmf_tgt_add_transport fails. Also update nvmf_tgt
initialization step to check that all poll groups were
created.
Change-Id: I116e6944729d846c1755c2844c77825f65db8c12
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9255
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
When a subsystem is being deleted, we disconnect all qpairs
and when the last qpair for some controller is disconnected,
we start controller desctruction process. This process requires
to send a message to subsystem's thread to remove the controller
from the list in the subsystem and after that send a message to
controller's thread to release resources.
The problem is that the subsystem also destroys all attached
controllers. This order is unpredictable and we may get
heap-use-after-free or double free.
To fix this problem we can rely on the fact that the subsystem
can only be destroyed in incative state, that means that all
qpairs linked to the subsystem are already disconnected and
all controllers are already destroyed or in the process of
destruction.
spdk_nvmf_subsystem_destroy API is now can be asyncrhonous,
it accepts a callback with cb argument.
Change-Id: Ic72d69200bc8302dae2f8cd8ca44bc640c6a8116
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6660
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Similar is already done for json-rpc bdev_get_bdevs, it might be
useful for the upper layer which has no interest in all but only
in one specified.
Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: I25f7d20ff953e612d7982207d57607f1c3bbaa90
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9361
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Rename endpoint->fd to ->devmem_fd to better reflect its purpose.
Signed-off-by: John Levon <john.levon@nutanix.com>
Change-Id: Ic73cb6684228d4ecb3d58c1865c80a3d7d07c223
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9402
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
transport specific options are already introduced however dump was missed
Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: I2fe533898d82534cc6cb8a3e5de4e2b2c72ae00a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9355
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Refine ANA state from per subsystem listener to per subsystem listener
per ANA group.
Add an array of ANA state per ANA group to subsystem listener. The array is
indexed by ANA group ID - 1.
Then in I/O paths, we get ANA state by
ctrlr->listener->ana_state[ns->anagrpid - 1].
The NVMe specification indicates the existence of NVM subsystem specific
ANA state when FFFFFFFFh is specified as NSID for the Get Features
and the Set Features commands. For these, we return the optimized state.
Update the nvmf_subsystem_get_listeners RPC to return all ANA states
of the underlying ANA groups. The nvmf_subsystem_get_listeners RPC is
not matured and not used in the test code yet. Hence compatibility is
not high priority.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ia2d4d5361ac01236f595c22765fd35e4c5fdee0e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9064
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This is the first patch in the patch series to control ANA states not only
as a unit of subsystem listener but also as a unit of ANA group and create
user preferred mapping between namespaces and ANA groups within a single
subsystem.
This patch adds anagrpid to both spdk_nvmf_ns and spdk_nvmf_ns_opts, and adds
ana_group array to spdk_nvmf_subsystem to count number of namespaces per ANA
group within a single subsystem. The size of the ana_group array is equal
with the size of the namespaces.
For each subsystem, allocate ana_group array regardless of the value of
ana_reporting of the subsystem.
For each namespace, at its creation, initialize anagrpid explicitly to be equal
with nsid by default and increments the corresponding entry of the ana_group
array of the subsystem regardless of teh value of the ana_reporting of thee
subsystem.
Hence the contents of the created ANA log page is not changed even if the
algorithm to crete ANA log page is changed.
Additionally this patch adds a unit test case that one ANA group
has multiple namespaces.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I78539db4e7248c2953c6927ff8128cb5a7e34b96
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9102
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
The next patch will add anagrpid to spdk_nvmf_ns_opts. This patch
is for the upcoming change and futhre potential changesto ensure the
ABI compatibility.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ibe1024a27ea4ab823d40d775470cde78b9eb8d2c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9112
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Previously we used assert for this check.
Fix#2141.
Change-Id: Ieaf8e34721a5b65e6d7790d56587239e2e6d75f4
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9383
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
If a response is returned prior to _nvmf_request_complete being called then the cid in the response is
not set correctly and the PDU state is not reset which causes a hang and the PDU state machine is
expecting more data but none will be sent. There are two cases where this can occur:
1) If the request is bi-directional
2) If nvmf_tcp_req_parse_sgl returns an error (e.g max_io_size exceeded)
Signed-off-by: matthewb <matthew.burbridge@hpe.com>
Change-Id: Icc3ed02a4499a12d8920e6433a746b72022a72fe
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9327
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reservation commands aren't supported for vfio-user transport,
we should return error in case VM send such commands intentionally.
Also don't use one err variable for multiple functions return values.
Change-Id: Ie0d51ce3c85a1b8ef80c678a3ec9fb6523bab462
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9125
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Only one active socket connection is supported in libvfio-user,
RESERVATION should not be supported in this case.
Change-Id: I36a746f479da767d857425e84dfed5f2bef30b17
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9124
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
We could not restore the setting of ana_reporting because it was not
included in the JSON config dump.
Add the parameter ana_reporting into JSON config dump by adding and
using a new helper function nvmf_subsystem_get_ana_reporting().
Besides, previously the JSON RPC nvmf_subsystem_get_listeners had
ana_state regardless of the value of ana_reporting. We make it
conditional in this patch. The JSON RPC nvmf_subsystem_get_listeners
had not been used in the test code in the repository. Hence this
change will be acceptable.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ia4e04600c969c254e0a816d3eb34983ee951091e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9111
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
PRP1 value 0 is a valid Guest physical address and it may use two vectors
for one page, so we need to set `iovcnt` as the IO commands. Also
fix the calculation for Get Log Page command.
Change-Id: Ib036bcf1ca9f6644dfd8a7406fea37269cea5c73
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9122
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This function is called in the IO processing context, also
rename vfio-user controller data structure with "vu_" prefix.
Change-Id: Icb8fd18772f64c1e633f4a8b8eac26d0bafeed16
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9121
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This function is called in the IO processing context, also rename
internal queue pair variable with "vu_" prefix.
Change-Id: Iac6be1fddf9ee28e0485ff0c3a707de4e98be96a
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9120
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
The submission queue doorbell will be cleared to 0 at the end
of this function.
Change-Id: I0146f3dc208b8ca1d7fec4b2dae1d7395f304444
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8992
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Thanos Makatos <thanos.makatos@nutanix.com>
There is only one place to call it now, so just remove it.
Change-Id: Ib16d640c80dbe79df46c965c5ee8bcd092e57b0d
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9013
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
We should maintain phase bit in vfio-user target, it's not safe
to use Guest completion queue's phase bit.
Change-Id: I94856853784802f7695a0484b783d8de4042f8a6
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8988
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Thanos Makatos <thanos.makatos@nutanix.com>
It's wrong to call enable_admin_queue() in the memory hotplug context,
as the function will change doorbells and clear queue pair memory.
We can remap the ADMIN queue pair's memory just same as IO queue pairs.
Change-Id: I090ea40ce0f0613c15319bfb18771a154ca788c7
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8987
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Thanos Makatos <thanos.makatos@nutanix.com>
The map_q function is used to map Guest's physical memory address
to Host virtual address for SQ/CQ. This function can be used both
for initializing SQ/CQ and remap SQ/CQ, when used for remap SQ/CQ,
we don't need to unmap the related memory region.
Change-Id: Ia42e01a68482e5678dbaf8d0d4e8c04c1a94789d
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9012
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Actually we should not re-enable the ADMIN queue during
the memory region callback, similar with the IO queues,
we should just do the remap, but to avoid changing other
values such as doorbells.
This is a preparation patch just to rename it, and will fix
the issue in coming patchs.
Change-Id: Ic19654e944e4c58ce0fac9582c6bb187110fe571
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8986
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Thanos Makatos <thanos.makatos@nutanix.com>
This field is required for Multiple SQs share one Completion queue.
Windows requires SQID field even not for the shared Completion
queue case, so we need to fill it.
Fix issue #2009
Change-Id: Ifb2f20196baeaf9b982154f0480f360bc97a8c2d
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8971
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Thanos Makatos <thanos.makatos@nutanix.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Currently, queue creation is delegated to nvmf_vfio_user_accept() for both I/O
and admin queues, by adding it to a ->new_qps list. But there's no good reason
for this indirection to exist, and it would make interrupt support for the
accept poller harder to implement. Instead, directly call
spdk_nvmf_tgt_new_qpair().
Signed-off-by: John Levon <john.levon@nutanix.com>
Change-Id: I182bfac9c621eda2f746201db7022ab6fce1bf30
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9053
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
The comment no longer reflected the code.
Signed-off-by: John Levon <john.levon@nutanix.com>
Change-Id: I5903e9483f01dd9b7a61c8f725eba4c38e1b8459
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9008
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Only the CID is required when posting a completion response to the
completion queue.
Change-Id: I6a009386994d19a2fa05a0f5ed8c3237263644f3
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8970
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Thanos Makatos <thanos.makatos@nutanix.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Multiple IO Submission Queue can share one Completion Queue, and
we use field 'cqid' to save it in Submission Queue, so when posting
completion response, we need to get the Submission Queue's CQID first,
then post the completion queue based on CQID.
Also rename vfio-user internal variables with 'vu_' prefix in this
function.
Change-Id: Ib73b91a86740d105b5fb5c73127484ebfb6b55ef
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8969
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Thanos Makatos <thanos.makatos@nutanix.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
No actual logic change for this patch.
Change-Id: I486e889499ee2e4bdf05d1f633ded1712333fa88
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8968
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Thanos Makatos <thanos.makatos@nutanix.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Currently, the poller that calls vfu_run_ctx() always returns SPDK_POLLER_BUSY.
Update libvfio-user and adjust the API usage so that it can accurately
report SPDK_POLLER_IDLE when needed.
Additionally, renaming the poller to better reflect its meaning: it's not just
for mmio handlers, but libvfio-user handling in general.
Signed-off-by: John Levon <john.levon@nutanix.com>
Change-Id: I5e598241ac0a692f03ee36242ff977c4e6f14987
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8568
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Return the number of events handled as expected by the poller.
Signed-off-by: John Levon <john.levon@nutanix.com>
Change-Id: I70c4d32bf091b2c1a293eaa41f00869a3a7303f2
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8563
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Thanos Makatos <thanos.makatos@nutanix.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
The transport poller is supposed to return the number of events handled to the
generic nvmf code; correct the vfio-user implementation so it does that.
Signed-off-by: John Levon <john.levon@nutanix.com>
Change-Id: I804e1e8c75701c0b22ea6afd350b455c39908511
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8562
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Windows will always sends a Set Feature Interrupt Coalescing even
SPDK reports we can't support it in Get Feature command. Here
we return Feature Not Changeable instead of Invalid Field which
is more meaningful.
Change-Id: Ie08086c3eba1e2d790a7ae4976653b6f9085028c
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8923
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
The data buffer isn't available at the beginning.
Change-Id: Ieeb1a297ff52dfdc6cd999d04862a0cd96483650
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8932
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
There were a few references to "SPDK thread context", which are no longer
relevant in the current codebase. Additionally clean up another XXX to be
clearer as to the context, and fix two minor typos.
Signed-off-by: John Levon <john.levon@nutanix.com>
Change-Id: I2aeda8aaa0edc973fdb67feccc32d792c5f7292e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8548
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Thanos Makatos <thanos.makatos@nutanix.com>
Implemented nvmf code to allow transports to use ZCOPY. Note ZCOPY
has to be enabled within the individual transport layer
Signed-off-by: matthewb <matthew.burbridge@hpe.com>
Change-Id: I273b3d4ab44d882c916ac39e821505e1f4211ded
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6817
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
When the QP is set to INACTIVE state, we always unmap the QP's
address to NULL, we can just check the address is valid or
not before posting completion response, so there is no need
to do the special process for the aborted AERs.
Change-Id: I1dc9893e04810e9a15e1eeb4d9405b775eab38d7
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8803
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
We should use the diff bits to decide the action to CC.
Change-Id: I08cf6f65711f86328825dcd9e204ed79766c59fe
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8802
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Similar with Create IO SQ command, we should also defer the completion
of Delete IO CQ command until the IO QP is disconnected finally. However,
since the NVMf library will disconnect/free the queue pair finally, we
can't use the queue pair data structure to save the context, so define
a delete_cq context for Delete IO CQ command.
Change-Id: I005ad86c2af59540323205e9e928a2d573d5c448
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8796
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
The NVMf library doesn't process Create IO SQ command, so for
this command we will use a fabric connection command instead,
however, the fabric connect command is called asynchronously,
so we need to defer the completion for Create IO SQ command after
fabric connect command is completed.
Fix issue #2043.
Change-Id: Id8c71ca69fb750a3ffb0fb26b54fa8b6d87e8ff4
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8786
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
We can set endpoint's controller pointer to NULL before free_ctrlr, as
controller is a session in vfio-user, while endpoint is related with
Unix Domain socket.
Change-Id: If6d26c7522804029f8c2425bc478df9e2d53d32a
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7966
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
The VM may already delete all queue pairs and just leave the
socket when killing VM, so we can check number of connected
queue pairs here, if no connected queue pairs, free the
controller immediately.
It's an optimization so that we don't need to loop all
queue pairs below.
Change-Id: Ia1be868ce2c74ce6953b1f44a81b51d605e642f0
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7625
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
The specification says:
"A host may replace its reservation key without regard to its registration
status or current reservation key value by setting the Ignore Existing Key
(IEKEY) bit to '1' in the Reservation Register command."
So for this case we treat it as a new registrant, also add UT to cover
the added cases.
Change-Id: I5990f15da36706063a35565d110ed4c6eb30a3f3
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8024
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Since we are using NVMf fabric library to emulate a PCIe based SSD via
vfio-user target, so there maybe some commands that are related with
PCIe SSD only, such as set/get features with interrupt coalescing
and Interrupt Mask Set/Interrupt Mask Clear registers. Even the
NVMf library doesn't support that, it is not a fatal error to Host
NVMe driver, so here we use the info log instead of error log for
this case so that to avoid noise logs.
Fix#2036.
Change-Id: I8283bcde5779080835d6ab827dbd852b3816176f
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8766
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
The NVMf library will not implement interrupt coalescing and ignore them, but we can
report this via get_features.
Some OS may check the result from get_features so that it will not send set_features
for interrupt coalescing.
Change-Id: I7466bcbc0ea5b3b067751cdf1979b2e0681c0043
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8765
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
There is no need to map the PRP/SGL list RW since this memory is never written
to. In fact, SeaBIOS might submit a request where the PRP list resides on
read-only memory, so attempting to map it RW can break things.
Change-Id: I7e4e90b1fa7e33e81b8d5cd8dcb9568c038938ec
Signed-off-by: Thanos Makatos <thanos.makatos@nutanix.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7288
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Nvmf/vfio-user uses this API to map NVMe command sent from
VM from Guest Physical Address to Host Virtual Address, so
now we moved this API from the nvme library to nvmf/vfio-user
as an internal API.
UT code will be added back in coming patch.
Change-Id: I54817fc9811ccd9ddd97b3aa6762a2fce4bbdda6
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8574
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: GangCao <gang.cao@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Bit 1 in the CMIC of the Identify Controller Data Structure specifies
if the NVM subsystem may have multiple controllers or not.
However, multi_host indicated a particular use case such that the NVM
subsystem is used by multiple hosts.
multi_ctrlr will be more appropriate.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I0246096a5cc44721aeff3ff6f96473a2abe11964
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8719
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
If NGUID is not specified with nvmf_subsystem_add_ns json-rpc request
then it is possible to expose the same NGUID as bdev nvme module
attached.
Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: Ie0ed7189e55a5abd6bc0904fc356d26f62b50549
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8628
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
The vfio-user target emulated NVMe device is treated as
PCIe NVMe SSD in the Guest VM, so when doing controller
reset or shutdown, we should abort the AERs which in the
NVMf library.
Users may switch kernel NVMe driver to SPDK NVMe driver
in the VM, without this fix, we will got "AERL exceeded"
response very frequently, because the AERs submitted by
previous driver will never be aborted in runtime.
Change-Id: I0222ed509629ccb0e98217414dd9043857105686
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8558
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
When users remove kernel NVMe driver in the VM, after 120 seconds,
SPDK NVMf target will disconnect ADMIN queue pair due to association
timer timeout, and for vfio-user transport, the ADMIN queue pair
connection is associated with the socket connection, so when probing
the NVMe controller again, because there is no active ADMIN connection
for fabric register R/W commands, it will cause segment fault.
Here we set the association timeout value to 0 for vfio-user transport,
so that the ADMIN connection will not be disconnected when shutdown the
controller, the ADMIN queue pair will be disconnected when the socket
connection breaks.
Change-Id: I3613169229bae384405889653e50f581d30d7c07
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8557
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
The NVMf library will set cdw0 based on specific command,
so we use it directly in vfio-user, otherwise, some NVMe
commands such as AER can't work.
Fix issue #2016.
Change-Id: Ie1a80a92c0856b61822ee51ce5d8faaaf1d463de
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8556
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Also fix one incorrect print log.
Change-Id: I3254baf4bbff4acfc0ef43f628d025931e8589ea
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8555
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: John Levon <levon@movementarian.org>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
These macros are only valid for Fabric transports.
Change-Id: Ia456eebdcdab28e81226c1b3a7211fcb41b5e481
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8554
Community-CI: Mellanox Build Bot
Reviewed-by: John Levon <levon@movementarian.org>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
It is better to not fail connect commands when a subsystem
is not ready. The host will not be expecting that and will
typically treat it as a catastrophic failure (i.e. it won't
retry the connect).
So instead when this situation occurs, start a poller for
the connect request. We will continue to retry processing
it until the subsystem is ready to handle it.
Fixes issue #1985.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Id8835df8f0edf1e889fdd7e754e261c2a880cbb6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8571
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
It is possible for a controller to get added to the
subsystem before its admin_qpair has been assigned.
We need to account for that when traversing the subsystem's
ctrlr list when determining ns and ana_changes that need
to be reported for the ctrlr.
Found while doing stress testing with connects and
subsystem ns add/remove.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ie54dc6ac202faeaeace054e6599f2dea2f30211e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8570
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Pollers are supposed to return SPDK_POLLER_{BUSY,IDLE}.
Signed-off-by: John Levon <john.levon@nutanix.com>
Change-Id: I92bd184aaba9e3efb730b68a6024ebc9757ffd8b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8559
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
The function comment was referring to a non-existent caller; instead, expand
with a little more detail on the path taken for new QPs.
Signed-off-by: John Levon <john.levon@nutanix.com>
Change-Id: I42478194f3cfc18a6ff6c434964630ac42866f1d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8534
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
When the property is 8 bytes but the host only requested
4, we need to mask and only return the bytes requested
by the host. Wait to do the DEBUGLOG until after
that has happened.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I8f476a47e9fd07bf652fd64f3b1c17d650374167
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8506
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
When a qpair is destroyed and the qpair is the last,
_nvmf_ctrlr_free_from_qpair() (in lib/nvmf/nvmf.c) sends two messages,
one is for _nvmf_ctrlr_destruct() and another is for
_nvmf_transport_qpair_fini().
We do not know which of two completes earlier.
_nvmf_ctrlr_destruct() frees the qpair->ctrlr in the end.
On the other hand, _nvmf_ctrlr_free_from_qpair() calls
spdk_nvmf_poll_group_remove() in the end, and spdk_nvmf_poll_group_remove()
accesses the qpair->ctrlr to free queued requests to the qpair.
Before one recent change, spdk_nvmf_poll_group_remove() had been called
before _nvmf_ctrlr_free_from_qpair() was called.
Hence extrace the operation to free queued requests from
spdk_nvmf_poll_group_remove() and inline it into _nvmf_qpair_destroy().
Fixes one showstopper error to investigate the issue reported in #1819.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I29c43ff7b289fc77a5de9c33e0266301c412e208
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8438
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Now that the trace library can handle multiple arguments, there's no
point in passing 0 for tracepoints that don't have any arguments. This
patch removes all such instances. It allows us to to verify that
`spdk_trace_record()` was issued with the exact number of arguments as
specified in the definition of the tracepoint.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Idbdb6f5111bd6175e145a12c1f0c095b62d744a9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8125
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
If we're not a DEBUG build, vfu_setup_log() was effectively forcing a
libvfio-user logging level of LOG_ERR. Instead, let the log handler decide what
to report, so we can respect the SPDK levels.
Signed-off-by: John Levon <john.levon@nutanix.com>
Change-Id: Ib3ad62589f495a377885f7deabaf02b428e83d30
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8452
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Thanos Makatos <thanos.makatos@nutanix.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Commit b7cc4dd added support multiple AERs, but didn't
remove the code that hardcodes aerl=0 for non-discovery
controllers. So even though the target now supports
multiple AERs, we never indicate that for non-discovery
controllers.
The spec also recommends that implementations support
a minimum of 4 AERs - so the current behavior is not
recommended.
It seems that at least on Windows (when testing with
vfio-user transport) we see the limit get exceeded
which results in ERRLOGs. Let's keep the ERRLOG there
for now, assuming that once we report we support 4
AERs that Windows won't try to send more than that.
Fixes issue #2000.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I6a07a6f37aaa6e531ae2cf1e1c46da036b00785b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8488
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
We cannot control what the host may send to the target.
For example, we have empirical evidence that Windows
will send vendor-specific IDs for features and log pages
(when testing with the vfio-user target transport).
So let's change the ERRLOGs in these cases to DEBUGLOGs.
Fixes issues #2004, #2007, #2008.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I8d5b92fc5e33d698af246f2f1c34f7cf51e6488a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8487
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
1. Update with latest vfio-user specification changes.
2. The new libvfio-user will not expose dma_sg_t data structure
any more, SPDK should use pointer and allocate memory for it.
Change-Id: I619b0c0828cbe3b050c628bff4c4ce7ee840510f
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8377
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Thanos Makatos <thanos.makatos@nutanix.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Community-CI: Mellanox Build Bot
When creating queue pairs, the original code uses a stack
queue variable and copy it to queue pair in insert_queue
function, the coming changes in libvfio-user doesn't expose
dma_sg_t data structure any more, we need to change it to
a pointer and allocate memory for it, so here we eliminate
insert_queue function as a preparation.
Change-Id: Iee94029d24bc8882ec169665e229e6cbc11564c0
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8376
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: GangCao <gang.cao@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: <dongx.yi@intel.com>
in-capsule data over 4KiB when using the Linux initiator.
This is fixed in the latest kernel. See
https://lists.infradead.org/pipermail/linux-nvme/2021-May/025641.htmlFixes#1823
Change-Id: Ie383ea774ee31ef8fe255119095b21603483c33f
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8424
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
nvmf_ctrlr_get_log_page used req->data to store the log page result.
While the req->data only contains the first iov, if req->iovcnt is
larger than 1, the req->data may not hold the complete log page; and
even worse, the log page result may be written to invalid address and
cause memory corruption.
Change-Id: Ie6415a6bd2327419fe4b32f21ac814fd827c9e95
Signed-off-by: Jiewei Ke <jiewei@smartx.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7970
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
In nvmf_vfio_user_listen(), fd should be closed before
set it to endpoint->fd, otherwise, the fd leakage probem
occurs.
Signed-off-by: Zhiqiang Liu <liuzhiqiang26@huawei.com>
Change-Id: I3fabc65d2764926e5873475962e4362e46eb37e4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8309
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: sunshihao <sunshihao@huawei.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
POSIX defines %z for printing size_t values in a portable way.
Replace a reference to %ld to remove the assumption about
the type of size_t.
Signed-off-by: Nick Connolly <nick.connolly@mayadata.io>
Change-Id: I2186aa5e7072f565ea75de935e22c2c23acf1a1a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8341
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Because we use spdk_dma_malloc, then it does not init
the the contents in the memory.
Fixes#1996
Change-Id: Ieef411f6ae5114de9f732df6096e0bb123efb7e0
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8374
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: GangCao <gang.cao@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
In spdk_nvmf_subsystem_add_ns_ext(), ns->ptpl_file is set to strdup(),
which may return NULL. We should deal with it.
Signed-off-by: Zhiqiang Liu <liuzhiqiang26@huawei.com>
Change-Id: If95102fe9d6d789b8ba9e846c4d7f4e22e48a93c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8305
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
The arguments of a tracepoint are formatted when they're printed now, so
there's no need to append ":" or pad it with spaces.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I74f5568f1982dacc079e3b80bd19a9cd740b48ce
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7955
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Both values should provide similar information, while the qpair can also
be matched to the traces from lib/nvmf allowing the user to track the
qpairs across these modules.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Iba9abdd3f41b93100c0403b1c90fc4549d39189e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7159
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
The traces are tracking the lifecycle of a poll group: creating it,
adding and disconnecting qpairs, and finally destroying the group.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I075b7f24d14b8fbb42bb18ddd70a668a8bace118
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7158
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Also add scripts/bpf/nvmf.bt to enable and log these
probes.
This patch also adds a script that can generate
a bpftrace script snippet with string maps for
needed enumerations (currently nvmf_tgt_state and
spdk_nvmf_subsystem_state). This allows us to
dynamically generate this from the source code, and
can be extended for other enums we may want to
add in the future.
Thanks to Michal Berger for converting my original
gen_enums.py script into gen_enums.sh!
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Iff34a6218aef40055ac14932eea5fc00e1c8bcf5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7194
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
The original function will disconnect queue pairs first and then free
controller memory finally, so rename it to vfio_user_destroy_ctrlr().
Change-Id: Idc235e4186bd4164be712fc9d4cda4991efc6248
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7624
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: <dongx.yi@intel.com>
The coming destroy_ctrlr() function will disconnect
queue pairs and free controller at last, so here
rename the original destroy_ctrlr() to free_ctrlr().
Change-Id: I527b2742142d60b0383be5a12391c77dd50d47a7
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7623
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: <dongx.yi@intel.com>
The original destroy_qp() only release the queue pair
related memory, and free_qp() will be called inside
destroy_ctrlr() function, so also remove one duplicated
line here.
Change-Id: I2a06a6704b514361685068acda4e65ed5d502f0d
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7622
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: <dongx.yi@intel.com>
The controller concept in NVMf is like a session, for any
new connection in nvmf_vfio_user_accept() with the endpoint,
we treat it as a new controller, we don't need the `ready`
field in controller to indicate the connection state, we
need the connection state in endpoint, so here just use
endpoint->ctrlr point to indicate the socket connection
is valid or not.
Change-Id: I588dbba7973cb61a1d79d81324a43e052f7dafb0
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7621
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
1. struct pxdcap.per is changed to struct pxdcap.rer
Which matches the name in the nvme spec.
2. use new API return value.
3. update specification changes.
Signed-off-by: Swapnil Ingle <swapnil.ingle@nutanix.com>
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Change-Id: Ida421c4cffd1c65d550e83011ab123b321ea9dff
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8088
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
vfio-user transport `req_complete` callback will zero the internal
NVMe command and response fields, the common NVMf library should
not use them after the callback, so here we use stack variables
to save them before the `req_complete` callback.
Fix issue #1965.
Change-Id: Iff2342b6095d9496cdf112d657a0a99ce1fb5d12
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8129
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
According to NVMe-oF 1.1 spec, it is not a fatal error.
So according to Figure 126 in NVMe Base specification,
we should return "Transient Transport Error".
Change-Id: I601304ae2bb24508882fb1ec8c7e53ec587ab515
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7795
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
In struct spdk_nvmf_poll_group_stat, there are statistics of cumulative IO and
admin queue pair counts. But current qpair counts are not reflected. Use
this patch to add current admin and io qpair counts for a poll group.
Signed-off-by: Rui Chang <rui.chang@arm.com>
Change-Id: I7d40aed8b3fb09f9d34e5b5232380d162b97882b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7969
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Eugene Kochetov <evgeniik@nvidia.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: GangCao <gang.cao@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Now, vfu_setup_region() must specify the region fd offset (which is always zero
in our case).
Signed-off-by: John Levon <john.levon@nutanix.com>
Change-Id: I10795d848a4c73ee9e1e78ea63776074401c4b17
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8022
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
nvmf_get_ana_log_page used req->data to store the log page result.
While the req->data only contains the first iov, if req->iovcnt is
larger than 1, the req->data may not hold the complete log page; and
even worse, the log page result may be written to invalid address and
cause memory corruption.
The following patch will fix the same issue for other commands in
nvmf_ctrlr_get_log_page.
Fix#1946
Signed-off-by: Jiewei Ke <jiewei@smartx.com>
Change-Id: I495f3be05c82be5cd53609772c655c8924b9179f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7923
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Community-CI: Mellanox Build Bot
Get all of the hot stuff to the first cache line.
* Shrink the xfer enum to one byte (it only has 3 values).
* Pull out the dif enabled flag form the dif structure so it
can be access separately
* Rearrange the members
Change-Id: Id4a2fe90a49c055a4672642faac0028671ebfae9
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7827
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Support ACRE (Advanced Command Retry Enable) feature. Currently set
all crdt to 0 and only select crdt[0] (crd=1) when the IO has any
error.
Signed-off-by: Peng Yu <yupeng0921@gmail.com>
Change-Id: If7bc30f91f5b2d0839002dead17188a4b3a52d5d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7885
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
That prevents nvmf target from starting to destroy poll
groups prematurely
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Change-Id: I833f6198ef0e3083fdadf70dd3b62844c905aceb
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7881
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Currently we allocate buffers perf each SGL descriptor.
That can lead to a problem when we use NVME bdev with
PRP controller and length of the 1st SGL descriptor is
not multiple of block size, i.e. the initiator may send
PRP1 (which is SGL[0]) which end address is page aligned
while start address is not aligned. This is allowed by
the spec. But when we read such a data to a local buffer,
start of the buffer is page aligned when its end is not.
That violates PRP requirements and we can't handle such
request. However if we use contig buffer to write both
PRP1 and PRP2 (SGL[0] and SGL[1]) then we won't meet
this problem.
Some existing unit tests were updated, 1 new was added.
Fixes github issue #1853
Change-Id: Ib2d56112b7b25e235d17bbc6df8dce4dc556e12d
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7259
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
We cannot solely rely on the qpair_ctx->count reaching
0, because qpairs that are in process of being
disconnected will immediately invoke the qpair
disconnect cb.
Instead, we need to wait until the poll group
no longer has any qpairs remaining on the subsystem.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I977747d367d14a4bf60f66a1147b3d75679e5179
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7870
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: <dongx.yi@intel.com>
This patch reduces admount of changes in the next patch,
no functional changes added.
The next patch will add usage of contig IO buffers for
multi SGL payload. To support it we need to pass an
offset to fill_wr_sgl function. Also in the current
version we assume that for 1 iteration we fill 1 IO
buffer, the next patch will change it and we'll need
to swtich to the next IO buffer in special case. That
can't be done easily if we use fill_wr_sge function
Change-Id: Iee8209634637697f700f8fa9fe61ead156b6d622
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7258
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
The header is small enough that it likely won't ever make sense
to offload the digest computation.
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: Ib6baa201a76d769d978f498f5c65985d5ab06ffd
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7766
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
We should use `ns->registrants` to count the number of registered
controllers(REGCTL) for Reservation Report command, as `subsystem->ctrlrs`
only list current active sessions(controllers).
Also use the output data buffer directly so that we don't need to calloc/free
during the process of Reservation Report command.
Fix#1928
Change-Id: I650224b751a08416208b8a504b82debff31e92fd
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7822
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: wanghailiang <hailiangx.e.wang@intel.com>
Add {min,max}_cntlid to spdk_nvmf_subsystem, defaulting to 1 and
0xFFEF, respectively, and add nvmf_subsystem_set_cntlid_range() to
allow the controller range to be configured in the range [min_cntlid,
max_cntlid].
Also add {min,max}_cntlid to the nvmf_create_subsystem RPC to allow
the controller ID range to be specified when creating an nvmf
subsystem.
Signed-off-by: Jonathan Teh <jonathan.teh@mayadata.io>
Change-Id: I936db3bb0c9a38569063a6fd3c11df262dfad776
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7322
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
ctrlr_discovery.c doesn't need this #include.
Including it causes bdev_module.h types to be
emitted to the debug symbols at least with some
compilers, which can result in unwanted abidiff
errors.
The unit tests do need it, so just include it
there instead.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iad28f9778ce08b11b52325658583ae9032295f3a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7813
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
This patch fixes missing free of qpair_mask when a listener error
occurs in ctrlr_create.
Signed-off-by: Jonas Pfefferle <pepperjo@japf.ch>
Change-Id: I09162b86d8ac73bf9fc2006a08dcc0a955f222b3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7818
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
The purpose is to prepared for implement the async crc32 caculation
in the future patch.
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I49f84ea1966f0acdd6f5aeb7192896f91fd16dee
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7793
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
This is used to prepare using the accel framework to calculate
the crc32 because some fields in this structure needs to be allocated
in DMA addressable memory.
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: Ib8def5596e60f4702709da647145c4e2b6d6848f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7767
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Note: a lot of the TCP and FC trace registers were
specifying '1' which means the arg type is a pointer,
but in reality it is always passing 0 for arg1. So
this patch just changes them to SPDK_TRACE_ARG_TYPE_INT.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I18d3cedd21e516f16cb2cd0a7f8c16670b1895d7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7763
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
It can be useful for testing or development purposes to force clients to write
to doorbells using vfio-user messages instead of directly into shared memory;
add a transport-specific option to disable the shared mapping.
Signed-off-by: John Levon <john.levon@nutanix.com>
Change-Id: I7ed062fbe211ba27c85d00b12d81a0f84a8322ed
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7554
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
(Note: this patch was previously applied as b32cfc46 and then reverted
as 63642bef.)
Today the in-guest nvme device shows physical_block_size=512 even though
the backend iSCSI bdev supports physical_block_size=4K
iSCSI targets exposes physical block size using
logical_block_per_physical_block_exponent in READ_CAPACITY_16
NPWG is one of the way to let Linux nvme driver set
physical_block_size of the nvme block device.
This patch adds spdk_bdev.phys_blocklen which is updated if the iSCSI
backend exposes physical_block_size.
Later phys_blocklen is used in nvmf to set NPWG and NAWUPF to report
back during NS identity.
Linux driver uses min(nawupf, npwg) to set physical_block_size.
Similarly in scsi_bdev fill lbppbe in READ_CAP16 response
based on spdk_bdev.phys_blocklen.
Fixes#1884
Signed-off-by: Swapnil Ingle <swapnil.ingle@nutanix.com>
Change-Id: I0b6c81f1937e346d448f49c927eda8c79d2d75c0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7739
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
1) use spdk_bdev_get_name() accessor
2) use __SPDK_BDEV_MODULE_ONLY #define
The latter allows nvmf to just get the spdk_bdev_module
definitions and APIs that it needs for claiming bdevs
for purposes of avoiding the same namespace used in
different subsystems.
This also ensures that future changes to structures
like spdk_bdev and spdk_bdev_io will not cause
lib/nvmf so version changes.
Note: we include bdev_module.h explicitly in the
nvmf/subsystem unit tests now, before including
subsystem.c, because the unit tests do depend on
knowing the internal structure of spdk_bdev.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I2f499a741d19f4749eadb402641f28137245fd23
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7738
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Reviewed-by: Anil Veerabhadrappa <anil.veerabhadrappa@broadcom.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
libvfio-user DMA APIs report all regions notified by the client, including those
that don't have a corresponding shared mapping. There are several of these for
a typical VM, so just ignore this case.
Signed-off-by: John Levon <john.levon@nutanix.com>
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Change-Id: I37b06f4bc6d1818a03c8742616ed142f575d3f0e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7532
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
It is already set by nvmf_tcp_req_pdu_init
when we get the pdu. So we do not set it again.
Change-Id: I034bbc46e600afd802457c0b152e303f16bafba3
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7714
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
This reverts commit b32cfc467b.
This commit fails the ABI checks and only got through because the checks
were disabled until 21.04 hit.
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: Id26b8f8ba551193d99b1ccbd31b35378b4095a20
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7731
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Today the in-guest nvme device shows physical_block_size=512 even though
the backend iSCSI bdev supports physical_block_size=4K
iSCSI targets exposes physical block size using
logical_block_per_physical_block_exponent in READ_CAPACITY_16
NPWG is one of the way to let Linux nvme driver set
physical_block_size of the nvme block device.
This patch adds spdk_bdev.phys_blocklen which is updated if the iSCSI
backend exposes physical_block_size.
Later phys_blocklen is used in nvmf to set NPWG and NAWUPF to report
back during NS identity.
Linux driver uses min(nawupf, npwg) to set physical_block_size.
Similarly in scsi_bdev fill lbppbe in READ_CAP16 response
based on spdk_bdev.phys_blocklen.
Fixes#1884
Signed-off-by: Swapnil Ingle <swapnil.ingle@nutanix.com>
Change-Id: I0b6c81f1937e346d448f49c927eda8c79d2d75cf
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7310
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
tcp transport doesn't send a response capsule when
c2h_success is set even if cdw0 or cdw1 are non-0.
Signed-off-by: Ed rodriguez <edwinr@netapp.com>
Signed-off-by: John Meneghini johnm@netapp.com
Change-Id: Ieba81fcc50342a2009f7931526e6f8392e26b6a5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6808
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Also use debug log when the memory region isn't 2MiB aligned,
The QEMU may only use one page for a memory region, we are sure
these memory regions will not be used as NVMe data buffers.
Previously libvfio-user will help us to round up these memory
regions to 2MiB alignment, and it doesn't do it anymore, this
isn't an error case so change it to debug log.
Change-Id: I6c397f50407d4f2a14f78d9f99fffc2e4054ff51
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7545
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
private tag added to the experimental or internal rpcs
which might be removed in future or not documented.
Signed-off-by: Monica Kenguva <monica.kenguva@intel.com>
Change-Id: I3e967252412f2491860eea5fa69750a7562b994a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7510
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
The API `spdk_nvme_map_prps` is used in nvmf/vfio-user to
remap VM's NVMe command data buffer to local virtual address,
and for command using PRP, there maybe multiple pages, when
parsing the PRP list to local IOVs, we need a parameter to check
that the maximum number of vectors can't exceed the IOVs, this API
can't meet the requirement, while here, we add a new API `spdk_nvme_map_cmd`
and with a new parameter `max_iovcnt` to fix this case, and it can
also cover the command using SGL in the coming patches.
Change-Id: I71063524bed16ee3434103867a556d3741e55326
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7278
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This is required by libvfio-user APIs.
Change-Id: I675a3be0a9650d146c8d37e42debf1191656903b
Signed-off-by: John Levon <john.levon@nutanix.com>
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7472
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I068e74f5b7d078ad37572eff47e772ad6967b827
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7436
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Unlike tcp/rdma transport, the vfio-user transport doesn't need to
wait for the data buffers, so here we add two request states for
now.
The request state will help us for coming request abort API.
Change-Id: Ibbb193fbbd358333f81aa29341493c19ab7bd108
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7435
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Previously we reset them when getting a new request, but it's more
reasonable in the completion path.
Change-Id: I3dab35ce471d2a5bbd37576540d30a09dcf93410
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7434
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Also rename transport request and controller variables
with "vu_" prefix.
The consolidated function will be used in coming patch.
Change-Id: I5219c13d7089dfdaea4a54e0b15cc5e6ecf2eb16
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7433
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: I2e47b148209ce4c232dbdc5f20c90548be995e1a
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7334
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This will ensure that we can't exceed the iovcnt when parse
NVMe PRP list to req->iov.
Also comment that the iovcnt in vfio-user transport is used to track
each gpa_to_vva map, for NVMe PRP list command, the PRP2 itself also
will use one entry, so we need add one more entry for this case.
Fix issue #1864.
Change-Id: I06c7137e2c4637c9501f82a9eb1c8e4395d819cd
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7264
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
For NVMe backend device, we should use vtophys to calculate
physical address when doing DMA from/to VM to drives.
Fix#1822.
Change-Id: Ib8fbc371e19e77a20202d408340e7d65644b1eeb
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7261
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Previously we poll the MMIO callbacks in the context of ADMIN queue's
poll group, here we do some improvement to start a poller to do MMIO
poll, then the group poll will only process NVMe commands while the
MMIO poller will process MMIO access.
This is useful when doing live migration, because the migration region
defined by VFIO is a BAR region, we should stop polling queue pairs
but ack the MMIO accesses during the live migration.
Change-Id: I63bac44889cbe0c31d47599810aab8335dfd4ff5
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7251
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Just code movement for the coming patch.
Change-Id: I7e844bc27a037e086796f9659351f20cdbb517fb
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7333
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
nvmf_poll_group_remove_subsystem_msg() disconnects all
qpairs associated with controllers in the specified
subsystem. If it finds any controllers that need to
be disconnected, it sends a message to the running
thread to execute the same function again later.
But when it runs again later, the qpair may no longer
be in the poll group, but there could still be
outstanding messages being sent between threads. For
example, _nvmf_qpair_destroy() needs to send a message
to the ctrlr->thread to clear the qpair mask bit.
All of this could result in the nvmf target starting
to destroy poll groups prematurely. Destroy poll
groups results in the nvmf spdk_threads exiting. If
there are still messages being processed from
the STOP_SUBSYSTEMS target state, we can get
use-after-free errors since processing of those
messages could access freed memory associated with
the exited thread.
Fixes issue #1850.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I1e63b9addb2956495a69b5108a41e029f6f9a85d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7275
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Custom transport may not provide the `qpair_abort_request`
callback function, so here for transport API we will just
call it when it's not empty. We will add the callback
support with vfio-user in another patch.
Fix#1883.
Change-Id: Icd82a26bde4ed90068bc85ee04cce9642cb6135d
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7291
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
We used the controller ready field to indicate ADMIN queue connection,
but the accept poller and ADMIN poll group may run in different
threads, this may lead vfu_attach_ctx() be called several times, so
change the 'ready' to true when a new socket connection is created.
Fix issue #1854.
Change-Id: Iab6ffd6dffb3fff5cf893e79774bc28fe0b2830c
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7073
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This will make the code easier to understand.
Change-Id: I7112d3fd5f0d6dce9b66d44375b68ce7d1e8951d
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7072
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
unmap_q is only be called in unmap_qp, so remove this function to make the
code more clear to read.
Change-Id: I627c7a1efdcb85476cb618fced8b0bfc2d8f1f62
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6886
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
When killing QEMU or remote client is terminated normally,
we can release current controller related data structure,
users may restart QEMU to connect the same socket file
again, for the new connection, vfio-user will create
a new controller data structure for it.
Here we add a lock in the endpoint data structure to protect
number of connected queue pairs variable, because controller
data structure is like a session, while endpoint is related
with the socket file, so it's safe here. Moreover, we can
use this lock to protect live migration related data
structures in future.
Change-Id: Ie7060041a253604e7a2242813ec284eae46fe4e8
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6862
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Using nvmf_tcp_destroy() would destroy ttransport->lock, which hasn't
been initialized by that point yet.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ie9ced97ef520236dddaa70453b6807e8382ce534
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7235
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
The max_io_size transport option should be a power of 2 and be >= 8KB.
Max data tranfer size is defined in NVMe-oF spec as 2^(mdts cmd field) * 4KB.
Mdts cmd field is calculated as spdk_u32log2(transport->opts.max_io_size / 4096),
so max_io_size < 8KB results in mdts=0, which means no size limit (according to spec).
User can set max_io_size = 0 explicitly to allow no size limit.
Signed-off-by: Maciej Szulik <maciej.szulik@intel.com>
Change-Id: Id88a77efce5f217e1fc7750f61c0bd330aaa3791
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6384
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
We can send a message to repeat subsystem pause
and free a context that will be used later
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Change-Id: Ia5e8b0ff43f5e38bd8e659a8a64d42926e1d3c6e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6661
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: <dongx.yi@intel.com>
With this change, each polling group will use one
accel_engine channel. This change will be more suitable
to utlize the underlying accelerated device.
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: Ibab183a1f65baff7e58529ee05e96b1b04731285
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7055
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Actually we already do this when removeing a memory region, but
the check for it is too strict, we should unmap queue pairs when
the queue pair is in the memory region.
Change-Id: Ia646a0255e32ecdd0a70537a8011ce622eb59195
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6861
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
The error response should be processed at the beginning of this
function.
Change-Id: Id583951c82981cf58984ab68b23ad6f7ea80cd3f
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6859
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
When starting VM, there are error logs such as:
vfio_user.c: 510:acq_map: *ERROR*: Map ACQ failed, ACQ 3ffde000, errno -1
vfio_user.c:1043:map_admin_queue: *ERROR*: /var/run/muser/domain/muser1/1: failed to map CQ0: -1
vfio_user.c:1103:memory_region_add_cb: *NOTICE*: Failed to map SQID 1 0x3ffd8000-0x3ffdc000, will try again in next poll
This isn't the error case, because when the Guest memory hot add/remove from QEMU, vfio-user
target will stop and unmap all queue pairs and remap them again, so let's use a more friendly
log instead.
Also use a notice log when adding listener.
Change-Id: Iaa4dc29e02523b5e85ec716d200ec355f8a575ed
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6650
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
nvmf_subsystem_remove_listener RPC handler may fail to remove
the listener (e.g. it doesn't exist) but in eror case we
spdk_nvmf_transport_stop_listen_async and send an error
response. In a completion callback passed to
spdk_nvmf_transport_stop_listen_async we try to send a
response again but the response handler had already been
released and we dereference a NULL pointer.
The fix is to skip spdk_nvmf_transport_stop_listen_async
in error case and continue with the subsystem resuming.
Fixes github issue #1821
Change-Id: I8d96b943cca25d9f95d19e8ea600242f019e6b21
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6699
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
When we iterate qpairs that belong to a subsystem
and try to disconnect them, there is a chance that
some qpair can be disconnected on transport level,
e.g. the initiator may receive a disconnect for
the first qpair and disconnect others. That may lead
to a dead loop when we call spdk_nvmf_qpair_disconnect
with a callback, the callback is called immediatelly
and tries to disconnect the qpair again.
To solve this problem, move part of nvmf_poll_group_remove_subsystem
function to another function nvmf_poll_group_remove_subsystem_msg
which disconnects all qpair at once without any callback
and calls itself via thread_send_msg untill all qpairs are
disconnected.
Fixes github issue #1780
Change-Id: I1000cda73e6164917fc13f7f374366af90571b99
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6597
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
This change refactors the way nvmf_get_stats RPC works.
The RPC layer passes JSON write context to custom dump function defined within transport ops.
The RPC layer no longer needs to know the structure of transport poll group statictics.
Functions and structures used in the previous flow have been deprecated and will be removed.
JSON returned for RDMA transport should be the same as before this change.
Signed-off-by: Maciej Szulik <maciej.szulik@intel.com>
Change-Id: I03308c45be120793d316bf79814a1295afd9fb95
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6681
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
spdk_nvmf_tgt_listen() is deprecated, so moved
the remaining instance to spdk_nvmf_tgt_listen_ext().
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I32b54e99f83fa10f1074f80aad82bb0608c9ae11
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6630
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Anil Veerabhadrappa <anil.veerabhadrappa@broadcom.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
This statistic is incremented when we don't reap
anything from the CQ. Together with the total number
of polls it can be useful to estimate idle percentage.
Change-Id: I61b51d049b0bc506fb8a896e225187e46e75a564
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6295
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Previously we only process Read/Write/Flush IO commands, we should
not block the DSM command in vfio-user layer if the backend block
device can support it.
Change-Id: Ia6b90397adcc36015f331f011a5bdf3e3d6562d8
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6525
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
nvmf_create_transport rpc parameter to configure the CQ size helps
if the user is aware of CQ size needed as iWARP doesn't support CQ resize.
Fixes issue #1747
Signed-off-by: Monica Kenguva <monica.kenguva@intel.com>
Change-Id: Ia9ba2b5f612993be27ebfa3455fb4fefd80ae738
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6495
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
This patch is used to leverage accelerated engine to compute
the data digest in the following case:
1 DIF is not used.
2 The data to compute is aligned with size 4, i.e, %4 = 0.
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I51fb6e3ab04391062b244cba6e249c8e20d3180f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6014
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This patch provides two new accelerated crc32c function interface.
And the next patch will be used to add the real support of chained crc32c feature.
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I3f8dd55c3da636e29e5fb02fc229b51f05653cd6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6456
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
