If the queue was on another poll group, we need to send a message back
to the admin CQ's thread to post the completion from the correct
context.
Signed-off-by: John Levon <john.levon@nutanix.com>
Change-Id: I997987d5d6b822a1a5124f54fc29ce5d7f03190d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14057
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Swapnil Ingle <swapnil.ingle@nutanix.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
spdk_nvme_qpair_process_completions() had called
always _nvme_qpair_complete_abort_queued_reqs() at its end.
However, the call was accidentally removed by a commit
59c8bb527b
to fix an issue.
By this removal, aborting request was not completed for some error
cases.
Fix the degradation by restoring the call.
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I0099eb7a008f823e1282576504423cdc248911d7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14045
Reviewed-by: Michael Haeuptle <michaelhaeuptle@gmail.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
This option determines if underlying nvme device must be zoned.
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I65cb91db81d4c5ce694010db2e79e8af389687ab
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14008
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
glibc 2.36 added arc4random(), which breaks
the SPDK iSCSI build since it always implements its
own arc4random() implementation on non-FreeBSD OS
(meaning always on Linux).
So instead add a CONFIG_HAVE_ARC4RANDOM and remove
the explicit FreeBSD dependency - this will work on
FreeBSD as well as Linux with >= glibc 2.36.
Also fix check_format.sh, so that it does not
enforce spdk/stdinc.h checks on code snippets in
the configure file.
Fixes issue #2637.
Reported-by: Karl Bonde Torp <k.torp@samsung.com>
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iab9da8ae30d62a56869530846372ffddf7138eed
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14028
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Avoid putting a new req on the outstanding_reqs
TAILQ until we know it can be initialized
successfully. This avoids adding to the TAILQ
only to remove it just after.
This allow simplifies the outstanding_reqs TAILQ
handling, since reqs are now only inserted and
removed in one place each.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I5ccc41c14abd541ffcf2a602246e0671386840c7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13991
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
We were using "TR" for "tracker" previously, but
we are tracing the nvme_requests, not nvme_trackers,
so use the right names for the trace object to avoid
confusion.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ia3886d74b162138c2cdbe0017224d9494f74966c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13990
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
This allows us to pack more arguments into the same
amount of shared memory, for cases where those arguments
don't need a full 8 bytes.
1- and 2-byte sizes not supported for now, variadic args
do automatic promotion of types smaller than int, so
support for those may need more work.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iec56cfa851b408a77d7995126d2111b0bf3d7f95
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13999
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
The public interface of lib/accel is now include/spdk/accel.h
Change-Id: Id94f623a494eb1b524b060f4413f633073ea7466
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13916
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
They no longer need to see the definition of this structure.
Change-Id: I3e3bb5942a50da22e0bf34aa8c10b9d812f42d2f
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13915
Community-CI: Mellanox Build Bot
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
These hold a pointer to the channel which eliminates the need to look it
back up in the completion path.
Change-Id: Ie4fc98d92d6434262e64b9483ef8b3b0591d764a
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13914
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
This expects the full size of the task for each module. This only worked
because the software module returned the right size.
Change-Id: I481cfad8b4bb9c3748301bdacd90e7f44fd2d878
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13913
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
This will help keep the mixing of this code with the framework code to a
minimum.
Change-Id: I5937ebd84f32068456cdf2b9e03d3e194c760a87
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13912
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
spdk_internal/accel_engine.h will become the API for accel modules. Move
anything in there that a module doesn't need to see into
lib/accel/accel_internal.h
Some of the software fallback definitions didn't even need to be in a
header and were moved to accel_engine.c
Change-Id: Idb8b12b1c0c1de3d462b906e3df3ba9ee8f830b8
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13911
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
These are 1:1 - they do not need to be separate objects.
Change-Id: I74ab52863f911d9be59ce98e1525302b5bd40846
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13910
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Fix broken link to Vhost-user protocol documentation.
Update terminology to match protocol documentation.
Change-Id: Id1aeb0fc8bc99990577efb767ab3dde2411bd664
Signed-off-by: John Kariuki <John.K.Kariuki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13849
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
This change fixes the lower performance issue of the
xNVMe BDEV for libaio and io_uring backends. There is no
drop in the performance for the io_uring passthrough backend.
Signed-off-by: Krishna Kanth Reddy <krish.reddy@samsung.com>
Change-Id: I21de1b49a534cfc642d1873ef623271063da6af8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14015
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
The multiple reactors support for vfio-user was enabled recently,
and enable this tests in CI now.
Change-Id: I8e145f62c995f968298040b1c550c421a25a7c9f
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13547
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: John Levon <levon@movementarian.org>
There is a race condition if we call this function in the
polling mode when running with multi-cores, same as other
places where the function is called, we only kick controller
in interrupt mode, also in `vfio_user_ctrlr_intr`,
`ctrlr->sqs[0]` may be set to NULL after the controller
poll call, so return earlier for this case.
Change-Id: I03a7b74a39c966a2b8be610bca0e492d902f6b08
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13696
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Note, this change only sets defaults for the ID/KEY,
more specific use cases like NVMe/TCP may set the ID and KEY on a per connection basis.
Also simplify PSK identity string, that isn't NVMe focused.
NVMe libraries using this will need to construct more complicated
identity strings and pass them to the sock layer.
Example:
rpc.py sock_impl_set_options -i ssl --psk-key 4321DEADBEEF1234
rpc.py sock_impl_set_options -i ssl --psk-identity psk.spdk.io
./build/examples/perf --psk-key 4321DEADBEEF1234 --psk-identity psk.spdk.io
./build/examples/hello_sock --psk-key 4321DEADBEEF1234 --psk-identity psk.spdk.io
Change-Id: I1cb5b0b706bdeafbccbc71f8320bc8e2961cbb55
Signed-off-by: Boris Glimcher <Boris.Glimcher@emc.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13759
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
This avoids having to update this comment should
we increase the number of arguments supported.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I67591ce96433f38a2463ec462151b3ce0ae44e53
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13987
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
There were a few error cases that weren't caught
as errors, meaning the "invalid tpoint mask" string
wouldn't be printed.
But also change it so that when an invalid tpoint mask
is specified, it fails spdk_app_start and causes the
application to exit, rather than just silently
stopping processing of the tpoint group mask string.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I567a4eee740559914e089dca7d7c3865ed9ce35b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13986
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
If a nvme_ctrlr is unregistered while I/O path caches are clearing, the
unregistration would fail. This race was not considered for a case that
a nvme_ctrlr is unregsitered when adminq poller started I/O path cache
clearing.
As a bug fix, control ANA log page updating and I/O path cache clearing
separately by adding a new flag io_path_cache_clearing.
Fixes issue #2617
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: Id58684caf9b2a10fa68bdb717288ff2bd799c3f7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13924
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
For the case `nvmf_subsystem_remove_listener` RPC call when VM is connected,
we should not relisten the accept poller, because the endpoint will be
destroyed for this case.
Change-Id: Icf8299f26a3bbf7bbe44fd01edb4ede344692d25
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13548
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: John Levon <levon@movementarian.org>
These sleeps should not be required. The steps before them properly
wait.
Change-Id: Ifc278c582dbf8fd837ca7f1f02873789a4e616d6
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13835
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
We can use the nvmf_get_stats RPC to directly count the number of
connections on each poll group.
Also, we cannot rely on knowing the exact algorithm that the driver will
use to assign connections to NAPI values. We can only know that the
number of unique NAPI values is AT MOST equal to the number of traffic
classes.
Fixes#2577
Change-Id: I6758fdcda752ff5372b1bc59984257b804ad0ea7
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13833
Reviewed-by: wanghailiang <hailiangx.e.wang@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This allows our setup to be more flexible - extra Vagrantfiles can
provide additional configuration which may override|extend existing
configuration without having a support for specific set of options
on our end.
CI will attempt to use this functionality to mitigate the following
issue:
https://github.com/spdk/spdk/issues/2628
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I80cbe67b2fe3618f93e9233e0586417b208f987b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13929
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Changed vfio-user source branch for installing and testing.
Replaced vfio-user-dbfix branch with vfio-user-irqmask2 due
to the recent issue #2600.
Signed-off-by: Kamil Godzwon <kamilx.godzwon@intel.com>
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: Ide525cdd3496c2a3dbecea4e44e6edff63438859
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13867
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This is needed to make sure tests use proper QEMU binary.
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I868bf60b5480e6a258fbe067ec77d9af26f3a9c9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13964
Reviewed-by: <sebastian.brzezinka@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
ioccsz is specific for fabrics. spdk_nvme_ctrlr_is_fabrics() returns
true for custom fabrics transport. Hence we can use
spdk_nvme_ctrlr_is_fabrics() safely in nvme_ctrlr_update_nvmf_ioccsz().
Before this change, in the unit tests, ctrlr->trid.trtype was set to
zero at initialization. After this change, for most cases,
spdk_nvme_ctrlr_is_fabrics() should return false for most cases.
SPDK_NVME_TRANSPORT_PCIE did not work. Hence, initialize
ctrlr->trid.trtype by SPDK_NVME_TRANSPORT_CUSTOM_FABRICS instead.
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I4bedcab4a9f2876c1c9463ff10ad0966754f1713
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13948
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
There was SPDK_NVME_TRANSPORT_CUSTOM but it was not usable for fabric
custom transport because spdk_nvme_trtype_is_fabrics() returned false.
Use 4097 for SPDK_NVME_TRANSPORT_CUSTOM_FABRICS and update
spdk_nvme_trtype_is_fabrics() to include SPDK_NVME_TRANSPORT_CUSTOM_FABRICS
because it is not defined in the specification.
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I457845e74d4e8150c9376f9bc253a8ee1ad8c1d3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13947
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
