spdk_reactor_set_interrupt_mode() writes/reads from fds created during
reactor_interrupt_init(). Since spdk_fd_group_create() depends
on eventfd, this will not work for systems that do not have it.
reactor_interrupt_init() handled lack of support for eventfd correctly,
while spdk_reactor_set_interrupt_mode() did not check for it.
Reported-by: Nick Connolly <nick.connolly@mayadata.io>
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I5181d436636c55cca3a06b1947e944502a9204ce
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9131
Reviewed-by: Nick Connolly <nick.connolly@mayadata.io>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
PRP1 value 0 is a valid Guest physical address and it may use two vectors
for one page, so we need to set `iovcnt` as the IO commands. Also
fix the calculation for Get Log Page command.
Change-Id: Ib036bcf1ca9f6644dfd8a7406fea37269cea5c73
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9122
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This function is called in the IO processing context, also
rename vfio-user controller data structure with "vu_" prefix.
Change-Id: Icb8fd18772f64c1e633f4a8b8eac26d0bafeed16
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9121
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This function is called in the IO processing context, also rename
internal queue pair variable with "vu_" prefix.
Change-Id: Iac6be1fddf9ee28e0485ff0c3a707de4e98be96a
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9120
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
The submission queue doorbell will be cleared to 0 at the end
of this function.
Change-Id: I0146f3dc208b8ca1d7fec4b2dae1d7395f304444
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8992
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Thanos Makatos <thanos.makatos@nutanix.com>
There is only one place to call it now, so just remove it.
Change-Id: Ib16d640c80dbe79df46c965c5ee8bcd092e57b0d
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9013
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
We should maintain phase bit in vfio-user target, it's not safe
to use Guest completion queue's phase bit.
Change-Id: I94856853784802f7695a0484b783d8de4042f8a6
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8988
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Thanos Makatos <thanos.makatos@nutanix.com>
It's wrong to call enable_admin_queue() in the memory hotplug context,
as the function will change doorbells and clear queue pair memory.
We can remap the ADMIN queue pair's memory just same as IO queue pairs.
Change-Id: I090ea40ce0f0613c15319bfb18771a154ca788c7
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8987
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Thanos Makatos <thanos.makatos@nutanix.com>
The map_q function is used to map Guest's physical memory address
to Host virtual address for SQ/CQ. This function can be used both
for initializing SQ/CQ and remap SQ/CQ, when used for remap SQ/CQ,
we don't need to unmap the related memory region.
Change-Id: Ia42e01a68482e5678dbaf8d0d4e8c04c1a94789d
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9012
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Was set at a pretty high number during early development.
Instead of a #define, lets use the same math we use to
determine the size of the operation and descriptor pools as
that's the max number of batches that will be needed.
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Ide5311b4b6c931010413d0f82baed0e05d5fcde7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9099
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
When address translation failed we were not putting the op back
into the pool. In one place also changed the return error from
translation from hardcoded value to the rc returned by the call
to be consistent with the rest of the code.
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I6d0c9c0f469a213721c6a5ce37c4e9dbdcef8183
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9097
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
These are fixed, no reason to tranlate them with every IO. This
patch is just for non-batched IO. Batched IO will come later.
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Ifb7bdf147658a23714d94bb7f88d7fedd1749153
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9073
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
DPDK requires each command line parameter to be
passed as a separate string in the args array. So
we need to tokenize opts.env_context to handle the
case where a user passes multiple arguments in the
env_context string.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ifc72a7c25b31f1296d3aba3a3f7664ad8edf128f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9132
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
When run spdk_top against vhost, the poller name is displayed as
"bvdev->bdev ? vdev_worker : no_bdev_vdev_worker" which should be "vdev_worker".
Signed-off-by: Rui Chang <rui.chang@arm.com>
Change-Id: Ie97c69bd8cdd3fdafc5fe4dd80e1fde85a8e3238
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9130
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
The current num_md_clusters doesn't include the the part before
md_start. So the bs_recover will get more num_free_clusters than it
should be. This patch can fix it.
Signed-off-by: Peng Yu <yupeng0921@gmail.com>
Change-Id: I911926beb69aca677da508ba71f292496c917e7f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9034
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
The generic transport layer still does a busy wait, but at least
the logic in the PCIe transport now creates the queue pair
asynchronously.
Signed-off-by: Monica Kenguva <monica.kenguva@intel.com>
Change-Id: I9669ccb81a90ee0a36d3f5512bc49c503923b293
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8910
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
In prep for upcoming patch
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Iebc5bf5f6715c85cc09b404128338cd6f08c421e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9072
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
We can instead use a combination of the op code and the batch
element in the op structure to determine if the op that is
completing is part of a batch or not so we know whether to
return it to the free list or not.
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I8e27c7b991f5a20e82394a25a52ece560cab5543
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9071
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
No real material impact but easier to read and cleans up error
handling when submitting a batch. Before we put the operation on
the list of operations to poll when it was prepared, a few LOC
before it was submitted to HW. Now we do it in the hw submission
function. For batch elements, they are added just before the
batch operation itself (as was before).
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Icfeaeeb650d5b63f81e13a8ffd27e349c7118089
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9069
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
The bit arrays were used for dynamic flow control in a previous
implementation. They are no longer needed as flow control is
now static and managed solely in the idxd plug in module. Use
simple lists of descriptors and completion records instead.
This is a simpler implementation and will allow for some future
clean up of structures as well.
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I8c4cce12e88ac5416e3fe29a416487759214ec9f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8922
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Actually we should not re-enable the ADMIN queue during
the memory region callback, similar with the IO queues,
we should just do the remap, but to avoid changing other
values such as doorbells.
This is a preparation patch just to rename it, and will fix
the issue in coming patchs.
Change-Id: Ic19654e944e4c58ce0fac9582c6bb187110fe571
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8986
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Thanos Makatos <thanos.makatos@nutanix.com>
This field is required for Multiple SQs share one Completion queue.
Windows requires SQID field even not for the shared Completion
queue case, so we need to fill it.
Fix issue #2009
Change-Id: Ifb2f20196baeaf9b982154f0480f360bc97a8c2d
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8971
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Thanos Makatos <thanos.makatos@nutanix.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Currently, queue creation is delegated to nvmf_vfio_user_accept() for both I/O
and admin queues, by adding it to a ->new_qps list. But there's no good reason
for this indirection to exist, and it would make interrupt support for the
accept poller harder to implement. Instead, directly call
spdk_nvmf_tgt_new_qpair().
Signed-off-by: John Levon <john.levon@nutanix.com>
Change-Id: I182bfac9c621eda2f746201db7022ab6fce1bf30
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9053
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
The comment no longer reflected the code.
Signed-off-by: John Levon <john.levon@nutanix.com>
Change-Id: I5903e9483f01dd9b7a61c8f725eba4c38e1b8459
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9008
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Only the CID is required when posting a completion response to the
completion queue.
Change-Id: I6a009386994d19a2fa05a0f5ed8c3237263644f3
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8970
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Thanos Makatos <thanos.makatos@nutanix.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Multiple IO Submission Queue can share one Completion Queue, and
we use field 'cqid' to save it in Submission Queue, so when posting
completion response, we need to get the Submission Queue's CQID first,
then post the completion queue based on CQID.
Also rename vfio-user internal variables with 'vu_' prefix in this
function.
Change-Id: Ib73b91a86740d105b5fb5c73127484ebfb6b55ef
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8969
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Thanos Makatos <thanos.makatos@nutanix.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
No actual logic change for this patch.
Change-Id: I486e889499ee2e4bdf05d1f633ded1712333fa88
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8968
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Thanos Makatos <thanos.makatos@nutanix.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This issue is introduced by the refactoring, i.e.,
changed from dst to crc_dst. And this code
part is missed.
This patch can fix this issue.
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: Iad03d08707167c957193f6101eb9166a30e5cdd3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8935
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Currently, the poller that calls vfu_run_ctx() always returns SPDK_POLLER_BUSY.
Update libvfio-user and adjust the API usage so that it can accurately
report SPDK_POLLER_IDLE when needed.
Additionally, renaming the poller to better reflect its meaning: it's not just
for mmio handlers, but libvfio-user handling in general.
Signed-off-by: John Levon <john.levon@nutanix.com>
Change-Id: I5e598241ac0a692f03ee36242ff977c4e6f14987
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8568
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Return the number of events handled as expected by the poller.
Signed-off-by: John Levon <john.levon@nutanix.com>
Change-Id: I70c4d32bf091b2c1a293eaa41f00869a3a7303f2
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8563
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Thanos Makatos <thanos.makatos@nutanix.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
The transport poller is supposed to return the number of events handled to the
generic nvmf code; correct the vfio-user implementation so it does that.
Signed-off-by: John Levon <john.levon@nutanix.com>
Change-Id: I804e1e8c75701c0b22ea6afd350b455c39908511
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8562
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Windows will always sends a Set Feature Interrupt Coalescing even
SPDK reports we can't support it in Get Feature command. Here
we return Feature Not Changeable instead of Invalid Field which
is more meaningful.
Change-Id: Ie08086c3eba1e2d790a7ae4976653b6f9085028c
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8923
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
.ctrlr_connect_qpair
Previously this was assumed to be a synchronous process so the generic
layer transport code updated the state after .ctrlr_connect_qpair
returned. In preparation for making this support asynchronous mode,
shift that responsibility down into the individual transports.
While none of the transports actually do this asynchronously, insert a
busy wait in nvme_transport_ctrlr_connect_qpair to wait for the qpair to
exit from the CONNECTING state. None of the upper layer code can
actually correct handle a transport doing this asynchronously, so the
busy wait will cover that.
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: I3c1a5c115264ffcb87e549765d891d796e0c81fe
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8909
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
If there is hardware issues, we do not need to assign
the result. Because we will report the error status to the uplayer.
Change-Id: I647ddd609a1d5d0d52cc4fee59699b9992da4fa4
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8864
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Those functions are exported publicly, so better to
add some assert functions to detect some null pointer
errors.
We do not use if/else check, because it is too heavy.
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I16814efb84a5a41876657f0caf5f0a6d0c2db8f3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8863
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
This assert is used to make sure that there is no
active batch (spdk_accel_batch) task is used.
If there are active batches found, it means that
we did not handle this case in a good manner.
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I7ea6247d2d5a40cf4f3f31cd8b1240fed4648d62
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8857
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
After compiling SPDK with `--enable-ubsan` option, ocf tests fail with the
following error:
src/ocf/mngt/ocf_mngt_common.c:170:2: runtime error: member access within
misaligned address 0x200003800188 for type 'struct ocf_cache', which requires
64 byte alignment
The mentioned line of code is `list_for_each_entry()` macro used for iterating
lists. Forcing `struct list` alignment removes the issue.
Signed-off-by: Michal Mielewczyk <michal.mielewczyk@intel.com>
Change-Id: I803dd962ff873679f42568e6f42fec7fed278f67
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8516
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Rafal Stefanowski <rafal.stefanowski@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
The assertion should verify that a clone has been found. Without the
dereference, it makes no sense, as that pointer is dereferenced earlier.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I67fa17b33df6d507822a17ffc221a6d360985646
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8919
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Using `bs_allocate_and_copy_cluster()` instead of a zero-length write
makes it possible to inflate/decouple snapshots, as the writes would
fail with -EPERM, because the snapshots are marked as read-only.
Additionally, zero-length non-vector requests are now completed
immediately. It makes it consistent with the vector path (which already
does that) and allows us to use the zero-length reads as a context for
cluster copy.
Fixes#2028.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ib7fdee352972ecf808833aa179820d85cfab7eed
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8918
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
The data buffer isn't available at the beginning.
Change-Id: Ieeb1a297ff52dfdc6cd999d04862a0cd96483650
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8932
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
There were a few references to "SPDK thread context", which are no longer
relevant in the current codebase. Additionally clean up another XXX to be
clearer as to the context, and fix two minor typos.
Signed-off-by: John Levon <john.levon@nutanix.com>
Change-Id: I2aeda8aaa0edc973fdb67feccc32d792c5f7292e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8548
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Thanos Makatos <thanos.makatos@nutanix.com>
Split the NVMe controller reset into pre-init and reinit stages so
that the latter begins with a call to nvme_ctrlr_process_init(),
returning -EAGAIN if the controller is not yet ready so that a poller
can call it again later.
Signed-off-by: Jonathan Teh <jonathan.teh@mayadata.io>
Change-Id: Ia182b04e438241b139109be93f3ed858fac7f3d4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8486
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
BUG FIX: call nbd_bdev_hot_remove will stuck if
it is called when nbd has in-flight IOs.
nbd_bdev_hot_remove is asynchronous. It will
guarantee the stop of this nbd.
nbd hot remove test will be added later
Signed-off-by: MengjinWu <mengjin.wu@intel.com>
Change-Id: I0a0dfab31fafd3d61212ade53c74ad05dbbff268
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8039
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: GangCao <gang.cao@intel.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This parameter is still part of API spdk_sock_impl_opts
structure but it is not used. Keep it to support ABI
compatibility since it is located in the middle of the
structure and removing it may break socket opts initialization
or parsing.
Change-Id: Ib641ad7d965d68bc9ebb65dba531408d88cf6fa1
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8914
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Implemented nvmf code to allow transports to use ZCOPY. Note ZCOPY
has to be enabled within the individual transport layer
Signed-off-by: matthewb <matthew.burbridge@hpe.com>
Change-Id: I273b3d4ab44d882c916ac39e821505e1f4211ded
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6817
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
This batch_op field is not necessary because we can
use the comp_ctx->desc->opcode to judge whether it is related
a batched task or not.
Change-Id: Id329221ccf272c4c3bb8c1b5ec08433029a9a1f8
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8865
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Implement an async variant of spdk_nvme_ctrlr_reset(). This initial
implementation only allocates a context and returns it to the caller,
relying on the caller to poll the context to execute the existing
spdk_nvme_ctrlr_reset() implementation.
Wire up spdk_nvme_ctrlr_reset() to use this async variant to verify
that NVMe controller reset still works.
Signed-off-by: Jonathan Teh <jonathan.teh@mayadata.io>
Change-Id: I75d4b75dbf5897db452ee65286aef5a4eb839fca
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8330
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
When the QP is set to INACTIVE state, we always unmap the QP's
address to NULL, we can just check the address is valid or
not before posting completion response, so there is no need
to do the special process for the aborted AERs.
Change-Id: I1dc9893e04810e9a15e1eeb4d9405b775eab38d7
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8803
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
We should use the diff bits to decide the action to CC.
Change-Id: I08cf6f65711f86328825dcd9e204ed79766c59fe
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8802
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Similar with Create IO SQ command, we should also defer the completion
of Delete IO CQ command until the IO QP is disconnected finally. However,
since the NVMf library will disconnect/free the queue pair finally, we
can't use the queue pair data structure to save the context, so define
a delete_cq context for Delete IO CQ command.
Change-Id: I005ad86c2af59540323205e9e928a2d573d5c448
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8796
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
The NVMf library doesn't process Create IO SQ command, so for
this command we will use a fabric connection command instead,
however, the fabric connect command is called asynchronously,
so we need to defer the completion for Create IO SQ command after
fabric connect command is completed.
Fix issue #2043.
Change-Id: Id8c71ca69fb750a3ffb0fb26b54fa8b6d87e8ff4
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8786
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
We can set endpoint's controller pointer to NULL before free_ctrlr, as
controller is a session in vfio-user, while endpoint is related with
Unix Domain socket.
Change-Id: If6d26c7522804029f8c2425bc478df9e2d53d32a
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7966
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
The VM may already delete all queue pairs and just leave the
socket when killing VM, so we can check number of connected
queue pairs here, if no connected queue pairs, free the
controller immediately.
It's an optimization so that we don't need to loop all
queue pairs below.
Change-Id: Ia1be868ce2c74ce6953b1f44a81b51d605e642f0
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7625
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
In rte_power all that enabling/disabling turbo does is allows
for additional entry in frequency array for particular core.
Instead of exposing this API through spdk governor,
just make sure that dpdk_governor enables turbo by default.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I994b326a57c01889bccea26635753c56637259d2
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8789
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Those calls went unused, in favor or much more useful
up/down/min/max variants.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I432896196a1a6edfc6799c8658df49567f73d457
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8788
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
The spdk_governor_capabilities added lots of capabilities
which went unused, suposedly to mark which callbacks
a governor had implemented.
This made little sense, since capabilities are per core and
not implmenting this APIs made little sense.
With this patch spdk_governor_capabilities is brought in
line with rte_power_core_capabilities.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I85296fce2999cb41957162b63ee13d86a0be919f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8787
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Remove _spdk_scheduler_disable() to avoid confusion as there is
no spdk_scheduler_enable function. Since spdk_scheduler_disable
sets scheduler period to 0, use spdk_scheduler_period_set(0) instead.
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I4f1390a635f80e8b92775aa4be2e37f5b95467f8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7448
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
The size of the core_info->threads will always be equal
to reactor thread_count, there is no need to count it
separately.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Icfa84606bd29d7766738eb2053362a20d78c23af
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8733
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Replaced multiple functions calls to _reactors_scheduler_update_core_mode(),
with a for loop.
Since changing reactor to interrupt mode is rare operation, most of the
time we ended up with unnecessarily long callstack.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I1c7858653be9e2256943c1da5a27001be41682b6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8714
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
There is only one g_scheduling_reactor (main core), the is_scheduling
flag for it is used to block starting new gather_metrics before
previous one is finished.
Meanwhile is_scheduling flag on other reactors was used to block
destroying lw_threads while scheduling happens. It was only needed
because scheduler interacted with the same lw_thread pointers as
each reactor. Previous patch removed this dependency, instead
spdk_thread ids is used. If an spdk_thread is destroyed,
while scheduling _threads_reschedule_thread() handles it.
It is no longer required to block destruction of lw_threads
based on this flag.
Instead of using the main core reactor flag, a g_scheduling_in_progress
is introduced.
Removed _spdk_get_scheduling_reactor() and instead shared the value
of g_scheduling_in_progress between reactor.c and app.c.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ica57326a552477add522174cc3e96b3bab918350
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8732
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Removing dependency on schedulers to directly modify
lw_thread field structures will help making schedulers
truly plugable.
Instead of using lw_thread, new structure is created
that holds copy of stats and refer to the thread by
spdk_thread id.
As an added benefit of not changing lw_thread directly,
we won't run into issue of balancing function changing it
while other reactor removes and frees it.
In the future an API will be added for scheduler to call
in order to move the thread directly. Rather than for
event framework to rely on modified core_info/thread_info
structure.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I8f85bb8dc080fd13b78b07ee9ef8e8be7051659b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8411
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Limit of 50% to mark thread as active or idle
didn't allow for multiple active threads to be placed
on single core.
Lowering the limit to 20% will allow that and force
more threads to be actively balanced.
Removing the limit was considered, but that would
cause too much thread moves when a thread with load
in single digits increased briefly. Either
due to actually doing any operation or placement
of other threads on the same core.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I5f8c9ff15461feb71a2d82853cfe048e412ba921
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8289
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Fixes#1933
When decoupling parent the updated parent_id was
not persisted to the blob if it was a snapshot.
Due to having md_ro set to true, blob_set_xattr()
failed.
Later on the incorrect parent_id could cause troubles
like in the github issue, when deleting that snapshot.
This patch adds return code check for blob_set_xattr
and forces md_ro to false during blob md sync.
Since some of code paths are shared between decouple,
inflate and clone operations, the final callback for them
is doing revert of the original md_ro.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: If017455f72e4d809fe533d9f986e5ae6bb8e2035
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8420
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
After this patch, nbd will no longer receive any requests if
NBD_CMD_DISC is received. But it will handle the requests
already received.
Previously we called spdk_bdev_abort() for NBD_CMD_DISC and
it will reply to the rest requests in the channel of this bdev.
But there should be no reply to NBD_CMD_DISC. Hence we silently
discards requests after NBD_CMD_DISC.
Signed-off-by: MengjinWu <mengjin.wu@intel.com>
Change-Id: I551dea1887cb2d108ed5e0d621309f62cfaabbb9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8038
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
The specification says:
"A host may replace its reservation key without regard to its registration
status or current reservation key value by setting the Ignore Existing Key
(IEKEY) bit to '1' in the Reservation Register command."
So for this case we treat it as a new registrant, also add UT to cover
the added cases.
Change-Id: I5990f15da36706063a35565d110ed4c6eb30a3f3
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8024
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Since we are using NVMf fabric library to emulate a PCIe based SSD via
vfio-user target, so there maybe some commands that are related with
PCIe SSD only, such as set/get features with interrupt coalescing
and Interrupt Mask Set/Interrupt Mask Clear registers. Even the
NVMf library doesn't support that, it is not a fatal error to Host
NVMe driver, so here we use the info log instead of error log for
this case so that to avoid noise logs.
Fix#2036.
Change-Id: I8283bcde5779080835d6ab827dbd852b3816176f
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8766
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
The NVMf library will not implement interrupt coalescing and ignore them, but we can
report this via get_features.
Some OS may check the result from get_features so that it will not send set_features
for interrupt coalescing.
Change-Id: I7466bcbc0ea5b3b067751cdf1979b2e0681c0043
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8765
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Use the macros for red black tree provided by Free BSD to speed up
io_channel lookup.
Signed-off-by: Jiewei Ke <jiewei@smartx.com>
Change-Id: Icfd87a8a2f60c082a17b8c501a03faba83edb762
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7895
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: GangCao <gang.cao@intel.com>
In current implementation, io_channel list will be accessed by
spdk_for_each_channel() and spdk_get_io_channel(). We will try to
accelerate spdk_get_io_channel() in the following change "thread: speed
up io_channel lookup by using rbtree" by changing io_channel from list
into RB tree.
To make it cleaner, we prefer to use ch->dev as the key for the
io_channel RB tree instead of ch->dev->io_device. This patch makes
spdk_for_each_channel() use the i->dev to find the expected io_channel.
And the io_device in structure spdk_io_channel_iter is not needed in
spdk_for_each_channel_continue() but we keep it for the compatibility of
spdk_io_channel_iter_get_io_device().
After this patch, spdk_for_each_channel() has to access both io_device
list and io_channel list, and spdk_for_each_channel_continue() still has
to access only io_channel list.
Both io_device list and io_channel list will become RB tree. Hence
performance degradation will be negligible. spdk_for_each_channel() is
not so performance critical than spdk_get_io_channel().
Signed-off-by: Jiewei Ke <jiewei@smartx.com>
Change-Id: Idd486b0aa1b63b57ede90527dcd1631cbb008a1a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8749
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Use the macros for red black tree provided by Free BSD to speed up
io_device lookup.
This change was reverted once but is re-submitted because the critical
issue was fixed by the preceding patches.
In addition to the fix, add unit tests to verify the fix explicitly.
Signed-off-by: Jiewei Ke <jiewei@smartx.com>
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I97ed77f6e5ceacdf2593c9751b55a7d0b92c0b35
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8525
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Previously we used a counter of our own to make sure all batch
elements plus the batch itself were done before we freed the batch.
This was due to some observations early on that the batch desc
could complete before the individual elements and a lack of clarity
as to whether this was due to the simulator or the fact that
we poll on completions and could therefore "see" completions in
a different order at that time (we were using bit arrays to poll).
Now we use an ordered (in time) list to poll locations so if we
instead put the elements on the list first and then the batch desc
itself we are assured to always "see" them in order provided the
underlying device meets spec which there's no reason to assume it
does not.
This simplifies things a bit at the same time and still assures
that we call list calbacks in order and then the batch callback
without "special" handling.
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I4d9e3997786f2116ce6515682b8117799c645f51
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8397
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Left over from when the field was a void *
Signed-off-by: paul Luse <paul.e.luse@intel.com>
Change-Id: If9dfe2878f6afd6137d6d8efec90e310baf417f1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8280
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
For clarity, this element was added when crc+copy API was
added so might as well have all the CRC related functions use
it instead of `dst` to avoid confusion.
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Ic43adbd0df51c1a349847701ef318f452306d0b3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8229
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
We've always used `dst` as the destination for CRC result, with
the recent addition of a copy_CRC API `dst` was needed for the
copy destination and `crc_dst` was used for the CRC. This
patch just makes all the CRC functions use `crc_dst` to avoid
confusion. The accel_task struct also has a `crc_dst1 field,
that will be used consistently in the next patch.
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Ia84c4a9e7940c6ebd31410c12272bd22b0c6dd29
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8228
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Support in accel_perf is coming up in a later patch.
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I63a1d3b9b1a3254fdca78e27c473b9b3468c93c8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8202
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Allows for better performance by not hitting the same portal
address with every submission.
Signed-off-by: paul luse <paul.e.luse@intel.com>
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I1ec8eae6f3acec9e98161029cd5406ec08603aa6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8190
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
We find a few files to get the size of a member of a struct. How to
do it is a little complex. So add a macro to do it will be helpful
to read the current code and develop new features.
lib/dif had used member_size() internally but Linux use sizeof_member()
as the macro. Besides, SPDK have used upper case letters for similar
macros, SPDK_CONTAINEROF() and SPDK_COUNTOF(). Hence spdk_member_size()
may be good but propose SPDK_SIZEOF_MEMBER() as the macro.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I2179c845a3b75fb71aa039075cc4dfd30617b898
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8738
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
In this case, user could specify the core number like:
-m [0,1,10] besides the core mask like -m 0xF
Change-Id: I48621c5a84e5436deae07101591d0ef85b1e129e
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8746
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Now nbd stop will not be processed if this nbd is not fully started.
However, it will remember the stop command and do it asychronously
until nbd is fully started.
Signed-off-by: MengjinWu <mengjin.wu@intel.com>
Change-Id: Iea5ba143332c7d3fd85f816726788f05e7ae3c8d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8037
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
If there are completed asynchronous events that have not been notified
to the user, free them during controller shutdown to avoid memory leaks.
It can happen if an event completes before user has a chance to execute
`spdk_nvme_ctrlr_process_admin_completions()`.
Fixes#2032.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ie608bf9100342f8dfd709e070326f67335d27fed
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8740
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: GangCao <gang.cao@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
NVMe bdev module manages ANA log page itself now. So NVMe driver
should disable managing ANA log page.
Add a new option disable_read_ana_log_page to struct spdk_nvme_ctrlr_opts.
Then NVMe bdev module enables it when calling spdk_nvme_connect_async().
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Id5249efe90a4d50763c3a7eaa1eb9572f60fbc8c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8313
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
