Commit Graph

18835 Commits

Author SHA1 Message Date
Jim Harris
97661e86b7 nvme/pcie: add cpl status to PCIE_COMPLETE trace event
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I51e87f0f23b84956f96ab2efc62ad99a8d74cd4e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13989
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-08-16 10:23:10 +00:00
Jim Harris
7b05b29d48 nvme/pcie: use 4-byte trace arguments where possible
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I24c3fd545cadc403ac1f3589c6242a08a7a2f517
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14000
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-08-16 10:23:10 +00:00
Jim Harris
cdb0726b95 trace, trace_parser: support 4-byte INT/PTR arguments
This allows us to pack more arguments into the same
amount of shared memory, for cases where those arguments
don't need a full 8 bytes.

1- and 2-byte sizes not supported for now, variadic args
do automatic promotion of types smaller than int, so
support for those may need more work.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iec56cfa851b408a77d7995126d2111b0bf3d7f95
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13999
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2022-08-16 10:23:10 +00:00
Ben Walker
081f080a49 accel: Rename public header to accel.h
The public interface of lib/accel is now include/spdk/accel.h

Change-Id: Id94f623a494eb1b524b060f4413f633073ea7466
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13916
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2022-08-16 10:22:55 +00:00
Ben Walker
10ab81b83e accel: Hide the definition of accel_io_channel from modules
They no longer need to see the definition of this structure.

Change-Id: I3e3bb5942a50da22e0bf34aa8c10b9d812f42d2f
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13915
Community-CI: Mellanox Build Bot
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-08-16 10:22:55 +00:00
Ben Walker
225ae35a2f accel: Add per-module task structures
These hold a pointer to the channel which eliminates the need to look it
back up in the completion path.

Change-Id: Ie4fc98d92d6434262e64b9483ef8b3b0591d764a
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13914
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2022-08-16 10:22:55 +00:00
Ben Walker
df892eed67 accel: Return correct values for .get_ctx_size()
This expects the full size of the task for each module. This only worked
because the software module returned the right size.

Change-Id: I481cfad8b4bb9c3748301bdacd90e7f44fd2d878
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13913
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2022-08-16 10:22:55 +00:00
Ben Walker
678025c914 accel: Move the software module to its own file
This will help keep the mixing of this code with the framework code to a
minimum.

Change-Id: I5937ebd84f32068456cdf2b9e03d3e194c760a87
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13912
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-08-16 10:22:55 +00:00
Ben Walker
6074b3a3f9 accel: Move definitions not needed by modules to accel_internal.h
spdk_internal/accel_engine.h will become the API for accel modules. Move
anything in there that a module doesn't need to see into
lib/accel/accel_internal.h

Some of the software fallback definitions didn't even need to be in a
header and were moved to accel_engine.c

Change-Id: Idb8b12b1c0c1de3d462b906e3df3ba9ee8f830b8
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13911
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2022-08-16 10:22:55 +00:00
Ben Walker
aa156d53be accel: Combine spdk_accel_engine and spdk_accel_module_if
These are 1:1 - they do not need to be separate objects.

Change-Id: I74ab52863f911d9be59ce98e1525302b5bd40846
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13910
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2022-08-16 10:22:55 +00:00
John Kariuki
ba7331a712 doc/vhost: update Vhost-user link and terminology
Fix broken link to Vhost-user protocol documentation.
Update terminology to match protocol documentation.

Change-Id: Id1aeb0fc8bc99990577efb767ab3dde2411bd664
Signed-off-by: John Kariuki <John.K.Kariuki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13849
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2022-08-16 07:21:00 +00:00
Krishna Kanth Reddy
17948bcaa9 bdev : xNVMe BDEV module performance fix
This change fixes the lower performance issue of the
xNVMe BDEV for libaio and io_uring backends. There is no
drop in the performance for the io_uring passthrough backend.

Signed-off-by: Krishna Kanth Reddy <krish.reddy@samsung.com>
Change-Id: I21de1b49a534cfc642d1873ef623271063da6af8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14015
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-08-15 19:11:14 +00:00
Changpeng Liu
5a21d4c5b0 test/vfio-user: enable multiple reactors tests
The multiple reactors support for vfio-user was enabled recently,
and enable this tests in CI now.

Change-Id: I8e145f62c995f968298040b1c550c421a25a7c9f
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13547
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: John Levon <levon@movementarian.org>
2022-08-15 19:10:54 +00:00
Changpeng Liu
91eb10b4be nvmf/vfio-user: only kick controller when in interrupt mode
There is a race condition if we call this function in the
polling mode when running with multi-cores, same as other
places where the function is called, we only kick controller
in interrupt mode, also in `vfio_user_ctrlr_intr`,
`ctrlr->sqs[0]` may be set to NULL after the controller
poll call, so return earlier for this case.

Change-Id: I03a7b74a39c966a2b8be610bca0e492d902f6b08
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13696
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
2022-08-15 19:10:54 +00:00
Boris Glimcher
6212597bda sock/ssl: Add psk_key and psk_identity options to spdk_sock_impl_opts
Note, this change only sets defaults for the ID/KEY,
more specific use cases like NVMe/TCP may set the ID and KEY on a per connection basis.

Also simplify PSK identity string, that isn't NVMe focused.
NVMe libraries using this will need to construct more complicated
identity strings and pass them to the sock layer.

Example:
  rpc.py sock_impl_set_options -i ssl --psk-key 4321DEADBEEF1234
  rpc.py sock_impl_set_options -i ssl --psk-identity psk.spdk.io

  ./build/examples/perf --psk-key 4321DEADBEEF1234 --psk-identity psk.spdk.io

  ./build/examples/hello_sock --psk-key 4321DEADBEEF1234 --psk-identity psk.spdk.io

Change-Id: I1cb5b0b706bdeafbccbc71f8320bc8e2961cbb55
Signed-off-by: Boris Glimcher <Boris.Glimcher@emc.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13759
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2022-08-15 16:52:28 +00:00
Jim Harris
21d9b32fa3 trace: remove number of args from comment
This avoids having to update this comment should
we increase the number of arguments supported.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I67591ce96433f38a2463ec462151b3ce0ae44e53
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13987
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2022-08-12 14:18:05 +00:00
Jim Harris
e1eee2ebac event: always fail if invalid tpoint mask is specified
There were a few error cases that weren't caught
as errors, meaning the "invalid tpoint mask" string
wouldn't be printed.

But also change it so that when an invalid tpoint mask
is specified, it fails spdk_app_start and causes the
application to exit, rather than just silently
stopping processing of the tpoint group mask string.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I567a4eee740559914e089dca7d7c3865ed9ce35b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13986
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
2022-08-12 14:18:05 +00:00
Kozlowski Mateusz
a68a12a478 FTL: Initial nv cache structure
Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Change-Id: Ie40cc25ed9bf28976a5ae6d6a67491f438152fca
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13317
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-08-12 09:01:40 +00:00
Kozlowski Mateusz
ae6a35256b FTL: L2P tests
Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Change-Id: I13ce65785a4367ab42f87bf398470d17dc7e9747
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13316
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-08-12 09:01:40 +00:00
Artur Paszkiewicz
b16bdc6d49 FTL: Add L2P API and flat L2P implementation
Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Change-Id: Ifadc8c6986164584235ee6a67799025fa7703b5d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13315
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-08-12 09:01:40 +00:00
Artur Paszkiewicz
b6eecb21e5 FTL: Add address store/load utils
Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Change-Id: Ibac2fe36ba0f3038915075d7105e2d6119b8ed20
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13314
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-08-12 09:01:40 +00:00
Shuhei Matsumoto
a4ebc5714f bdev/nvme: Fix race between for_each_channel() and io_device_unregister()
If a nvme_ctrlr is unregistered while I/O path caches are clearing, the
unregistration would fail. This race was not considered for a case that
a nvme_ctrlr is unregsitered when adminq poller started I/O path cache
clearing.

As a bug fix, control ANA log page updating and I/O path cache clearing
separately by adding a new flag io_path_cache_clearing.

Fixes issue #2617

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: Id58684caf9b2a10fa68bdb717288ff2bd799c3f7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13924
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
2022-08-12 09:00:59 +00:00
Changpeng Liu
d0cf194bc4 nvmf/vfio-user: only relisten accept poller when connection is disconnected
For the case `nvmf_subsystem_remove_listener` RPC call when VM is connected,
we should not relisten the accept poller, because the endpoint will be
destroyed for this case.

Change-Id: Icf8299f26a3bbf7bbe44fd01edb4ede344692d25
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13548
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: John Levon <levon@movementarian.org>
2022-08-12 09:00:50 +00:00
Ben Walker
bc7f73996e test/adq: Remove unnecessary sleeps
These sleeps should not be required. The steps before them properly
wait.

Change-Id: Ifc278c582dbf8fd837ca7f1f02873789a4e616d6
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13835
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2022-08-12 09:00:26 +00:00
Ben Walker
9a9c9d3303 test/adq: Move nvmfappstart outside of adq_start_nvmf_target
Also rename adq_start_nvmf_target to adq_configure_nvmf_target.

Change-Id: I92bf4b7743061543a4d5fcbeaa59d0ff387e1395
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13834
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2022-08-12 09:00:26 +00:00
Ben Walker
8ebb8e65e7 test/adq: Simplify connection counting strategy
We can use the nvmf_get_stats RPC to directly count the number of
connections on each poll group.

Also, we cannot rely on knowing the exact algorithm that the driver will
use to assign connections to NAPI values. We can only know that the
number of unique NAPI values is AT MOST equal to the number of traffic
classes.

Fixes #2577

Change-Id: I6758fdcda752ff5372b1bc59984257b804ad0ea7
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13833
Reviewed-by: wanghailiang <hailiangx.e.wang@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-08-12 09:00:26 +00:00
Michal Berger
7f1b530ac0 scripts/vagrant: Remove some outdated comments
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I5c8f0b47afd38aab2ac0b45ea7f4f8e9f4c8fc54
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13930
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
2022-08-12 09:00:15 +00:00
Michal Berger
8aee39f906 scripts/vagrant: Add support for loading extra Vagrantfiles
This allows our setup to be more flexible - extra Vagrantfiles can
provide additional configuration which may override|extend existing
configuration without having a support for specific set of options
on our end.

CI will attempt to use this functionality to mitigate the following
issue:

  https://github.com/spdk/spdk/issues/2628

Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I80cbe67b2fe3618f93e9233e0586417b208f987b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13929
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-08-12 09:00:15 +00:00
Kamil Godzwon
66f47a03fb pkgdep/vfio-user: change vfio-user source branch
Changed vfio-user source branch for installing and testing.

Replaced vfio-user-dbfix branch with vfio-user-irqmask2 due
to the recent issue #2600.

Signed-off-by: Kamil Godzwon <kamilx.godzwon@intel.com>
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: Ide525cdd3496c2a3dbecea4e44e6edff63438859
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13867
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-08-12 09:00:04 +00:00
Michal Berger
8add2e5acb test/sma: Source common.sh from vfio_user suite
This is needed to make sure tests use proper QEMU binary.

Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I868bf60b5480e6a258fbe067ec77d9af26f3a9c9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13964
Reviewed-by: <sebastian.brzezinka@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-08-12 09:00:04 +00:00
Jaroslaw Chachulski
cfbab2dcd7 test/sma: Add new SPDK_TEST_SMA test flag
Signed-off-by: Jaroslaw Chachulski <jaroslawx.chachulski@intel.com>
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: Ib3ee2c147f07345c9880f63d75e3a87144584653
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13901
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: <sebastian.brzezinka@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-08-12 09:00:04 +00:00
Shuhei Matsumoto
227d83e2fa nvme: Use spdk_nvme_ctrlr_is_fabrics() to update ioccsz
ioccsz is specific for fabrics. spdk_nvme_ctrlr_is_fabrics() returns
true for custom fabrics transport. Hence we can use
spdk_nvme_ctrlr_is_fabrics() safely in nvme_ctrlr_update_nvmf_ioccsz().

Before this change, in the unit tests, ctrlr->trid.trtype was set to
zero at initialization. After this change, for most cases,
spdk_nvme_ctrlr_is_fabrics() should return false for most cases.

SPDK_NVME_TRANSPORT_PCIE did not work. Hence, initialize
ctrlr->trid.trtype by SPDK_NVME_TRANSPORT_CUSTOM_FABRICS instead.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I4bedcab4a9f2876c1c9463ff10ad0966754f1713
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13948
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-08-12 08:59:52 +00:00
Shuhei Matsumoto
f1dabc228f nvme: Add SPDK_NVME_TRANSPORT_CUSTOM_FABRICS for custom fabric transport
There was SPDK_NVME_TRANSPORT_CUSTOM but it was not usable for fabric
custom transport because spdk_nvme_trtype_is_fabrics() returned false.

Use 4097 for SPDK_NVME_TRANSPORT_CUSTOM_FABRICS and update
spdk_nvme_trtype_is_fabrics() to include SPDK_NVME_TRANSPORT_CUSTOM_FABRICS
because it is not defined in the specification.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I457845e74d4e8150c9376f9bc253a8ee1ad8c1d3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13947
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2022-08-12 08:59:52 +00:00
Shuhei Matsumoto
cd65512d08 nvme_rdma: Fix assertion for rqpair->current_num_sends/recvs
assert() in nvme_rdma_queue_recv_wr() was wrong and
assert() in nvme_rdma_cq_process_completions() was missing.

This patch fixes both.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Signed-off-by: Denis Nagorny <denisn@nvidia.com>
Signed-off-by: Evgeniy Kochetov <evgeniik@nvidia.com>
Change-Id: Ied057d75dbfd9e54ce3c3671355b9ec3acad7ff5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13597
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-08-12 08:59:43 +00:00
Shuhei Matsumoto
41bb31a36d nvme_rdma: Replace rdma_dereg_mr() by ibv_dereg_mr()
rdma_reg_msgs() was replaced by ibv_reg_mr() recently to support
persistent PD per RDMA device. The difference between rdma_dereg_mr()
and ibv_dereg_mr() is only return value and errno. For consistency,
replace rdma_dereg_mr() by ibv_dereg_mr().

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I55e0743690e74f9510863bfa122a75d0632dce4e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13949
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-08-12 08:59:43 +00:00
Shuhei Matsumoto
d75daea532 nvme_rdma: Use persistent protection domain for qpair
Get a PD for the device from the PD pool managed by the RDMA provider
when creating a QP, and put the PD when destroying the PD.

By this change, PD is managed completely by the RDMA provider or the hooks.
nvme_rdma_ctrlr::pd was added long time ago but is not referenced
anywhere. Remove nvme_rdma_ctrlr::pd for cleanup and clarification.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: If8dc8ad011eed70149012128bd1b33f1a8b7b90b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13770
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2022-08-12 08:59:43 +00:00
Shuhei Matsumoto
b5f360c425 rdma: Maintain per device PD which is persistent across reconnect
SPDK NVMe RDMA initiator used the default PD per RDMA device. Default PD
may be changed when all QPs for the RDMA device are destroyed and created
again.

For multipath, the RDMA zero copy feature require the PD per RDMA device
to be persistent when all QPs for the RDMA device are destroyed and
created again.

Maintain such persistent PDs in this patch.

Add two APIs, spdk_rdma_get_pd() and spdk_rdma_put_pd().

In each call of two APIs, synchronize RDMA device list with
rdma_get_devices().

Context may be deleted anytime by rdma-core. To avoid such deletion,
hold the returned array by rdma_get_devices().

RDMA device has PD, context, ref. count, and removed flag. If context
is missing in rdma_get_devices(), set the removed flag to true. Then,
if the ref count becomes zero, free the PD and the RDMA device.

The ref. count of a RDMA device is incremented when spdk_rdma_get_pd()
is called and decremented when spdk_rdma_put_pd() is called.

To simplify synchronization, sort the returned array by
rdma_get_devices().

To avoid resource leakage, add destructor function and free all PDs
and related data at termination.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I093cb4ec2c7d8432642edfbffa270797ccf3e715
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13769
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2022-08-12 08:59:43 +00:00
Jim Harris
3403be99bb test: disable reconnect test
This is causing too many intermittent failures, let's
disable for now while we continue to debug it.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I8f1f1824b1aa57fe242a77f5d96c9301a2921824
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14001
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: GangCao <gang.cao@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-08-12 02:55:56 +00:00
Jim Harris
b7f04e2be0 bdev/nvme: add newline in aer_cb WARNLOG
Noticed while looking at mhae's log in issue #2632.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ie7f17a010368696eb59585a2d31191e309c4576c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13888
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
2022-08-11 21:04:38 +00:00
Shuhei Matsumoto
a26d74173e nvme: Increase major SO version
An earlier commit added ctrlr_ready into struct
spdk_nvme_transport_ops. However, the major SO
version was not increased.

Fixes: 3dd0bc9e (nvme: Add transport controller ready step)
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: Id903634f9aaf5bdaa62fd30e92a4fb39a985b86f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13981
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-08-11 19:16:32 +00:00
Ben Walker
32ee475a5e accel: SPDK_ACCEL_MODULE_REGISTER is now passed the module
Instead of passing each parameter to create a module, just have the user
make one and pass it in. This makes it easier to change the module
definition later.

Change-Id: I3a29f59432a6f0773129d7b210fbc011175b2252
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13909
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-08-10 11:00:17 +00:00
Changpeng Liu
c029483193 libvfio-user: update submodule
Fix issue #2449.

Change-Id: I863521f455b746a8bcc62a180b5e7a4d4b5ea23a
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13925
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-08-10 07:30:05 +00:00
Shuhei Matsumoto
6b8de2ea66 bdev/nvme: Move up _nvme_ctrlr_read_ana_log_page_done() in a file
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: Ida517be9e16dd6f7cf39a0c6ef2044fd500f19a3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13923
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: GangCao <gang.cao@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-08-10 07:26:10 +00:00
paul luse
8416273819 examples/accel/perf: remove support for compress
In prep for the next few patches that will it back using a
different implmentation.  For review purposes it will be much
easier to remove it first. The new implmentation will be nothing
like what was here before.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I3d14e524751442bfd5f0ade87333d879137b5d05
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13435
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2022-08-10 07:25:29 +00:00
paul luse
6799d46a44 examples/accel_perf: only increment outstanding IO in one place
Simplify things a bit for an upcoming patch. This also fixes an
annoyance when there's a submission error, previously the
outstanding IO assert would have hit in the completion routine
because of the previous location of the increment..

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Ibfb914f10590f2370be58cabe06df466bbb3dc90
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12902
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-08-10 07:25:29 +00:00
paul luse
b21221e174 examples/accel_perf: add case for bogus workload selection
Could have sworn this was in there but when we check for workload
selections we weren't printing usage and exiting when there was
an unknown selection.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Iadc02457d9a15602bb4138c611a982333a8a2b66
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12932
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-08-10 07:25:29 +00:00
paul luse
9260fa0cd4 examples/accel_perf: print engine currently in use
Took a little bit of rework to both get the engine name and print
it before the results (as before).

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Iac38c30941eeae1501d58b0a28d241f7c2fad2d7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12935
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2022-08-10 07:25:29 +00:00
paul luse
3d5fd5a59f lib/idxd: fix bugs with IAA decompression descriptor construction
Masked by how accel_perf was doing decomp verificiation which is
changed in the next few patches and verifies these fixes.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Icb03fc169bf8d2f05396addaf1db56d6de1827d1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13038
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-08-10 07:25:29 +00:00
paul luse
8718eba606 unit test/accel: Add 2 new unit tests for recent additions
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I2cd589b36ea6923fd64c320f04c2caf1a5140218
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12877
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
2022-08-10 07:25:29 +00:00
paul luse
d49c9274bf doc/accel: add section on opcode and engine overrides
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I70d5d544f213f30018cce6d5ce316aafa5ed3925
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12873
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2022-08-10 07:25:29 +00:00