Commit Graph

9501 Commits

Author SHA1 Message Date
Changpeng Liu
91eb10b4be nvmf/vfio-user: only kick controller when in interrupt mode
There is a race condition if we call this function in the
polling mode when running with multi-cores, same as other
places where the function is called, we only kick controller
in interrupt mode, also in `vfio_user_ctrlr_intr`,
`ctrlr->sqs[0]` may be set to NULL after the controller
poll call, so return earlier for this case.

Change-Id: I03a7b74a39c966a2b8be610bca0e492d902f6b08
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13696
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
2022-08-15 19:10:54 +00:00
Boris Glimcher
6212597bda sock/ssl: Add psk_key and psk_identity options to spdk_sock_impl_opts
Note, this change only sets defaults for the ID/KEY,
more specific use cases like NVMe/TCP may set the ID and KEY on a per connection basis.

Also simplify PSK identity string, that isn't NVMe focused.
NVMe libraries using this will need to construct more complicated
identity strings and pass them to the sock layer.

Example:
  rpc.py sock_impl_set_options -i ssl --psk-key 4321DEADBEEF1234
  rpc.py sock_impl_set_options -i ssl --psk-identity psk.spdk.io

  ./build/examples/perf --psk-key 4321DEADBEEF1234 --psk-identity psk.spdk.io

  ./build/examples/hello_sock --psk-key 4321DEADBEEF1234 --psk-identity psk.spdk.io

Change-Id: I1cb5b0b706bdeafbccbc71f8320bc8e2961cbb55
Signed-off-by: Boris Glimcher <Boris.Glimcher@emc.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13759
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2022-08-15 16:52:28 +00:00
Jim Harris
e1eee2ebac event: always fail if invalid tpoint mask is specified
There were a few error cases that weren't caught
as errors, meaning the "invalid tpoint mask" string
wouldn't be printed.

But also change it so that when an invalid tpoint mask
is specified, it fails spdk_app_start and causes the
application to exit, rather than just silently
stopping processing of the tpoint group mask string.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I567a4eee740559914e089dca7d7c3865ed9ce35b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13986
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
2022-08-12 14:18:05 +00:00
Kozlowski Mateusz
a68a12a478 FTL: Initial nv cache structure
Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Change-Id: Ie40cc25ed9bf28976a5ae6d6a67491f438152fca
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13317
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-08-12 09:01:40 +00:00
Artur Paszkiewicz
b16bdc6d49 FTL: Add L2P API and flat L2P implementation
Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Change-Id: Ifadc8c6986164584235ee6a67799025fa7703b5d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13315
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-08-12 09:01:40 +00:00
Artur Paszkiewicz
b6eecb21e5 FTL: Add address store/load utils
Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Change-Id: Ibac2fe36ba0f3038915075d7105e2d6119b8ed20
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13314
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-08-12 09:01:40 +00:00
Changpeng Liu
d0cf194bc4 nvmf/vfio-user: only relisten accept poller when connection is disconnected
For the case `nvmf_subsystem_remove_listener` RPC call when VM is connected,
we should not relisten the accept poller, because the endpoint will be
destroyed for this case.

Change-Id: Icf8299f26a3bbf7bbe44fd01edb4ede344692d25
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13548
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: John Levon <levon@movementarian.org>
2022-08-12 09:00:50 +00:00
Shuhei Matsumoto
227d83e2fa nvme: Use spdk_nvme_ctrlr_is_fabrics() to update ioccsz
ioccsz is specific for fabrics. spdk_nvme_ctrlr_is_fabrics() returns
true for custom fabrics transport. Hence we can use
spdk_nvme_ctrlr_is_fabrics() safely in nvme_ctrlr_update_nvmf_ioccsz().

Before this change, in the unit tests, ctrlr->trid.trtype was set to
zero at initialization. After this change, for most cases,
spdk_nvme_ctrlr_is_fabrics() should return false for most cases.

SPDK_NVME_TRANSPORT_PCIE did not work. Hence, initialize
ctrlr->trid.trtype by SPDK_NVME_TRANSPORT_CUSTOM_FABRICS instead.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I4bedcab4a9f2876c1c9463ff10ad0966754f1713
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13948
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-08-12 08:59:52 +00:00
Shuhei Matsumoto
cd65512d08 nvme_rdma: Fix assertion for rqpair->current_num_sends/recvs
assert() in nvme_rdma_queue_recv_wr() was wrong and
assert() in nvme_rdma_cq_process_completions() was missing.

This patch fixes both.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Signed-off-by: Denis Nagorny <denisn@nvidia.com>
Signed-off-by: Evgeniy Kochetov <evgeniik@nvidia.com>
Change-Id: Ied057d75dbfd9e54ce3c3671355b9ec3acad7ff5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13597
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-08-12 08:59:43 +00:00
Shuhei Matsumoto
41bb31a36d nvme_rdma: Replace rdma_dereg_mr() by ibv_dereg_mr()
rdma_reg_msgs() was replaced by ibv_reg_mr() recently to support
persistent PD per RDMA device. The difference between rdma_dereg_mr()
and ibv_dereg_mr() is only return value and errno. For consistency,
replace rdma_dereg_mr() by ibv_dereg_mr().

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I55e0743690e74f9510863bfa122a75d0632dce4e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13949
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-08-12 08:59:43 +00:00
Shuhei Matsumoto
d75daea532 nvme_rdma: Use persistent protection domain for qpair
Get a PD for the device from the PD pool managed by the RDMA provider
when creating a QP, and put the PD when destroying the PD.

By this change, PD is managed completely by the RDMA provider or the hooks.
nvme_rdma_ctrlr::pd was added long time ago but is not referenced
anywhere. Remove nvme_rdma_ctrlr::pd for cleanup and clarification.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: If8dc8ad011eed70149012128bd1b33f1a8b7b90b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13770
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2022-08-12 08:59:43 +00:00
Shuhei Matsumoto
b5f360c425 rdma: Maintain per device PD which is persistent across reconnect
SPDK NVMe RDMA initiator used the default PD per RDMA device. Default PD
may be changed when all QPs for the RDMA device are destroyed and created
again.

For multipath, the RDMA zero copy feature require the PD per RDMA device
to be persistent when all QPs for the RDMA device are destroyed and
created again.

Maintain such persistent PDs in this patch.

Add two APIs, spdk_rdma_get_pd() and spdk_rdma_put_pd().

In each call of two APIs, synchronize RDMA device list with
rdma_get_devices().

Context may be deleted anytime by rdma-core. To avoid such deletion,
hold the returned array by rdma_get_devices().

RDMA device has PD, context, ref. count, and removed flag. If context
is missing in rdma_get_devices(), set the removed flag to true. Then,
if the ref count becomes zero, free the PD and the RDMA device.

The ref. count of a RDMA device is incremented when spdk_rdma_get_pd()
is called and decremented when spdk_rdma_put_pd() is called.

To simplify synchronization, sort the returned array by
rdma_get_devices().

To avoid resource leakage, add destructor function and free all PDs
and related data at termination.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I093cb4ec2c7d8432642edfbffa270797ccf3e715
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13769
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2022-08-12 08:59:43 +00:00
Shuhei Matsumoto
a26d74173e nvme: Increase major SO version
An earlier commit added ctrlr_ready into struct
spdk_nvme_transport_ops. However, the major SO
version was not increased.

Fixes: 3dd0bc9e (nvme: Add transport controller ready step)
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: Id903634f9aaf5bdaa62fd30e92a4fb39a985b86f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13981
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-08-11 19:16:32 +00:00
Ben Walker
32ee475a5e accel: SPDK_ACCEL_MODULE_REGISTER is now passed the module
Instead of passing each parameter to create a module, just have the user
make one and pass it in. This makes it easier to change the module
definition later.

Change-Id: I3a29f59432a6f0773129d7b210fbc011175b2252
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13909
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-08-10 11:00:17 +00:00
paul luse
3d5fd5a59f lib/idxd: fix bugs with IAA decompression descriptor construction
Masked by how accel_perf was doing decomp verificiation which is
changed in the next few patches and verifies these fixes.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Icb03fc169bf8d2f05396addaf1db56d6de1827d1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13038
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-08-10 07:25:29 +00:00
paul luse
efa33b8590 lib/accel: add RPC to enable override of opcode to engine
Docs explaining how to use the RPC are in the next patch in the
series.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I7dab8fdbeb90cdfde8b3e916ed6d19930ad36e66
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12848
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-08-10 07:25:29 +00:00
王亚飞/Yafei WANG
6fcd7a79e9 lib/vhost: Add submit_inflight_desc() to cpu usage statistic
submit_inflight_desc() actually do some meaningful work, so when it really process tasks, the poller should return BUSY status.

Signed-off-by: YafeiWangAlice <yafei.wang@samsung.com>
Change-Id: I2103cea6d28e8b355dad4ddd603d917f10e44c08
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13486
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: GangCao <gang.cao@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-08-09 11:48:51 +00:00
Jim Harris
d33497d3f4 thread: defer unregistration when for_each ops exist
There may be for_each operations outstanding on an
io_device when it is unregistered.  Currently we just
return when this happens, not unregistering the
device but also not notifying the caller that this
happened (since it returns void, and the callback
function doesn't have a status parameter either).

We could just push this responsibility to the caller,
to never unregister an io_device if it knows it has
outstanding for_each calls waiting to complete.  But
I think we can simplify this a lot by just handling
this inside of the thread library.  Mark that the
device is pending registration, and unregister it
(on the original requesting thread!) when the
for_each count gets back to zero.  Also don't
allow any new for_each operations either.

Note this requires a bit of refactoring on the
thread unit tests, since it is now possible to
unregister a device with outstanding for_each
operations.

Fixes issue #2631.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I978f2d99a25e65d2b7d71ce9b1926a79a6c94263
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13890
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-08-09 08:27:15 +00:00
Jim Harris
821e673c1d thread: set non-zero status when spdk_for_each_channel fails
If spdk_for_each_channel is called on a device that doesn't
exist, we need to set a non-zero status (-ENODEV in this
case) to the completion function.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I898ad5ea499fb6087338b621b2befcadd6a05414
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13889
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-08-09 08:27:15 +00:00
GangCao
0c980660b6 FTL: move assert earlier before accessing the field
Fix Klockwork issue.

Change-Id: Iae9557c152a745549c8963f4f0510ae829f871a4
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13860
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-08-08 13:48:28 +00:00
Jim Harris
5d651b31c9 event: require opts->name is set
This has been implicitly required before, and all
in-tree apps (except accel_perf) set it, so let's
explicitly require it.  This name gets used for
things like the shm name for spdk trace event file.

While here, add the name for accel_perf.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I47a22466550d4b31bacafee58d30339b4f22f4b4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13876
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michal Berger <michal.berger@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-08-05 10:48:42 +00:00
vagrant
fa09c9ac9b lib/blob: Fix deleting a snapshot after decoupling it from its parent
When decoupling a snapshot from its parent, we need to clear its parent.
So we should remove the xattr BLOB_SNAPSHOT. Modifying the xattrs of a blob
only works if its metadata are not in read-only mode.
By default, a snapshot is in read-only mode so this operation fails. When we
later want to delete the snapshot, we will see that it has a parent, so we will
try to remove the snapshot from its parent's clones list. This will cause a
crash.
The fix is to remove the BLOB_SNAPSHOT xattr only after setting the snapshot's
metadata in rw mode.

Signed-off-by: Alex Michon <amichon@kalrayinc.com>
Change-Id: I80efa6dd3dcb38b4c738ce2e97aa2ffc281cefa5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13723
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2022-08-05 08:30:30 +00:00
yidong0635
5de98ef86c reactor: Check error return for spdk_thread_lib_init_ext.
DPDK may use this NULL pointer to access its member,
And then got segmentation fault. But we only need it
exit or report normal error.

To minimize the impact, and to prevent these going on,
we add check the error return for creating NULL mempool
in spdk_thread_lib_init_ext in spdk_reactors_init.

when error returning from spdk_thread_lib_init_ext in spdk_reactors_init.
It contains thread_lib_init which reports error for failed mempool.

Thus, codes will return and will not cause segmentation fault.

Fixes issue #2620.

Signed-off-by: yidong0635 <dongx.yi@intel.com>
Change-Id: I63369fdaeb231196e8f8daa826eb5b057ed829b8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13842
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Michal Berger <michal.berger@intel.com>
2022-08-05 08:29:53 +00:00
yidong0635
c9eb502a4a thread: Return -ENOMEM for no mempool.
Here should return -ENOMEM, and other places are
changed.

Signed-off-by: yidong0635 <dongx.yi@intel.com>
Change-Id: Id81cd7485733e66d996b1501061a45f774f2b51a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13863
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Michal Berger <michal.berger@intel.com>
2022-08-05 08:29:53 +00:00
Changpeng Liu
a02483e67c module/bdev_virtio_scsi: use the correct num_queues value
Parameter `num_queues` for virtio_scsi PCI device means
maximum number of queues, it SHOULD include the `eventq`
and `controlq`, while for `vhost_user` RPC call, it means
the number of IO queues, so here we use it as `max_queues`
in lib/virtio and add the fixed number queues for `vhost_user`
SCSI device.

Also fix `vhost_fuzz` to get `num_queues` earlier than
negotiate the feature bits.

Change-Id: I41b3da5e4b4dc37127befd414226ea6eafcd9ad0
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13791
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-08-04 11:24:40 +00:00
Changpeng Liu
84ac072e2c lib/virtio: eliminate virtio_user_backend_ops
The `vhost_user` socket transport APIs are already in the
same source file, so just call the function directly.

No code logic changes in this commit.

Change-Id: If471b9b0166d43591fb8614e95a17473c964e87c
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13789
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-08-04 11:24:40 +00:00
Changpeng Liu
4e6e7eafef lib/virtio: merge vhost_user.c and virtio_user.c into one source file
Similar with NVMe device driver, here `virtio` is a specification
abstraction library, `pci` and `vhost_user` are transports layer,
here we merge vhost_user.c and virtio_user.c into one new source
file `virtio_vhost_user.c` so that to make code more clear.

No logic change, just code movement in this commit.

Change-Id: I8e3e5c477e7c45e6eeebad240b8cc3c9476b86d1
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13788
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-08-04 11:24:40 +00:00
Michal Berger
be1883d978 lib/ftl: Fix -Wunused-function under clang
Builds under clang fail with the following:

  utils/ftl_mempool.c:45:1: error: unused function 'is_element_valid'
    [-Werror,-Wunused-function]
  is_element_valid(struct ftl_mempool *mpool, void *element)
  ^
  1 error generated.

Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: Ic776f3f226e9ea6ed9d0bbd0a3d8e2a0661e0d11
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13844
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Kamil Godzwon <kamilx.godzwon@intel.com>
Community-CI: Mellanox Build Bot
2022-08-04 07:30:59 +00:00
Changpeng Liu
c60cb1a8be lib/nvmf: don't raise assertion in nvmf_tgt_destroy_cb
While running into this function, even the subsystem can't be
destroyed due to error subsystem state, it's better to continue
the execution.

Continue to fix #2590, QEMU is stuck for the failure case, and
nvmf target should process such error because it may support other
normal subsystems at the same time.

Change-Id: Ib05e24996378b52070d2b760519f476f9b2d7e76
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13839
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-08-04 07:29:27 +00:00
Evgeniy Kochetov
3dd0bc9e09 nvme: Add transport controller ready step
This step allows custom transports to perform extra actions or checks
at controller initialization and fail initialization if required.

Signed-off-by: Evgeniy Kochetov <evgeniik@nvidia.com>
Change-Id: Ic7cadae5398a35903917ceace3828f4371be63a3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12631
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-08-04 07:29:03 +00:00
paul luse
44cbea402e lib/accel: Add new RPC to get valid engine info.
The RPC provides a list of initialized engine names along with
that engine's supported operations.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I59f9e5cb7aa51a6193f0bd2ec31e543a56c12f17
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13745
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2022-08-03 07:53:21 +00:00
paul luse
c6ecddcc1c lib/accel: add RPC to get list of OP codes per module
In prep for upcoming patch that will provide an RPC to override
and automatic assignment of an op code to an engine.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I17d4b962fb376a77f97ce051a513679d0fba698e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12829
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-08-03 07:53:21 +00:00
Bin Yang
1cddc829ff lib/scsi: use bkdr hash to avoid naa identifier collision
fix: If the first six characters of two scsi lun's name are the same,
such as aaaaaa0 and aaaaaa1, so do theirs naa identifier

Signed-off-by: Bin Yang <bin.yang@jaguarmicro.com>
Change-Id: I4e0541b372a0e20e95e0a24d62dd3d85b7abe230
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13824
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-08-03 07:43:10 +00:00
yidong0635
5daedcc22e ftl: Fix compile warning.
Issue reports:

spdk/lib/ftl/ftl_io.c:121:9: warning: variable ‘result’
set but not used [-Wunused-but-set-variable]
  121 |  size_t result;
      |         ^~~~~~

Signed-off-by: yidong0635 <dongx.yi@intel.com>
Change-Id: I9ed7daea97f311ca33c4116299be32f275e33fbb
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13838
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: GangCao <gang.cao@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2022-08-03 07:42:38 +00:00
Artur Paszkiewicz
c6880a3974 ftl: superblock
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Change-Id: Ic8ca0cd3bf3621ad5604e83ed24c0fa59a83f124
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13313
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-08-02 19:00:42 +00:00
Artur Paszkiewicz
f725ca81cf ftl: vss emulation
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Change-Id: If22933834d640606526dec9185e849df367ac789
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13311
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-08-02 19:00:42 +00:00
Artur Paszkiewicz
884980d0aa ftl: vss null buffer workaround
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Change-Id: I94ea399ed30fae29f92b4216eaa9209c02b3478b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13310
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-08-02 19:00:42 +00:00
Artur Paszkiewicz
d67952540f ftl: wrappers for nv cache bdev io
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Change-Id: I33d99ae35e2bd853a16a6d20336632a955679197
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13309
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-08-02 19:00:42 +00:00
Kozlowski Mateusz
950cce2c9e FTL: Add ftl_io unit tests
Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Change-Id: I1052fbfe7516b12e50e4bc4b3b7a4f452f56349f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13308
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-08-02 19:00:42 +00:00
Artur Paszkiewicz
d9a631ad4c FTL: Add io channel logic
Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Change-Id: Ibf6bfbabc03c43e7938531c4fe08fde01ce02a3f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13307
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-08-02 19:00:42 +00:00
Kozlowski Mateusz
e7a03e68e1 FTL: Add ftl_rq
Used for internal metadata update requests

Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Change-Id: I742ef2030070e7e159d4354159fb596b98742631
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13306
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-08-02 19:00:42 +00:00
Artur Paszkiewicz
06790f25f1 FTL: Add ftl_io helper structure
Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Change-Id: I608b500c6fb14efe289932955f508484f2ecf1b6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13305
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-08-02 19:00:42 +00:00
Kozlowski Mateusz
b431640409 FTL: Add ftl mempools
Optimized for single thread utilization

Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Change-Id: I56602a3d85e0cd47256c8f3e5d7a3f0ed4e38743
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13303
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-08-02 19:00:42 +00:00
Shuhei Matsumoto
4f2f1aa9c5 nvme_rdma: Use pd of rdma_qp instead of default pd of cm_id
This is another preparation to create and use ibv_context and pd.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Signed-off-by: Denis Nagorny <denisn@nvidia.com>
Signed-off-by: Evgeniy Kochetov <evgeniik@nvidia.com>
Change-Id: Id594fa1ccb2daf535b1aaaef0a397bda2ec98578
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13710
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-08-02 07:39:41 +00:00
Shuhei Matsumoto
a3a51453b8 nvme_rdma: Pass pd instead of cm_id to nvme_rdma_reg_mr()
The following patches will create and use ibv_context and pd
explicitly instead of using default ibv_context and pd created
by rdmacm.

As a preparation, pass pd instead of cm_id to nvme_rdma_reg_mr().

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Signed-off-by: Denis Nagorny <denisn@nvidia.com>
Signed-off-by: Evgeniy Kochetov <evgeniik@nvidia.com>
Change-Id: Ifdcd18ed363b8ba4a23a920bf3559237e38821c6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13599
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-08-02 07:39:41 +00:00
Apokleos
89c1e5bfc0 SPDK Interrupt Mode: Improve processing of reactor interrupt mode.
spdk in interrupt, reactor dosen't correctly handle exited threads,
causing vhost threads still in reactor's lw_threads list. The fix
will do cleanup thread when it's state becomes EXITED. Though it's
exposed in v22.05.x, but the master branch also has the problem.

We will do this as below:

(1) When thread's state becomes SPDK_THREAD_STATE_EXITED, reactor
    process thread exits first.
(2) Then reactor do remove lw_thread and destroy it.

Fix issue: #2574

Signed-off-by: Apokleos <oliverliyn@gmail.com>
Change-Id: I3ac2681d70480563db3a0aee4aff61c2f272b140
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13706
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-08-02 07:38:15 +00:00
Konrad Sztyber
a818564374 nvme: check CSTS.CFS when initializing ctrlrs
If Controller Fatal Status (CFS) bit is set, there's no point in waiting
for CSTS.RDY and the only way to move forward with the initialization is
to perform a controller reset.

This fixes issues with test/nvme/sw_hotplug.sh when running under qemu.
It seems that during that test, qemu marks the emulated NVMe drives as
fatal, so if we didn't check CSTS.CFS, the initialization would time
out.

Fixes #2201.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I97712debc80c3dd6199545d393c0f340f29d33b2
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13820
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Michal Berger <michal.berger@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2022-08-02 07:37:04 +00:00
Changpeng Liu
78ca4b27c5 nvmf: don't raise assertion when destroying an non-inactive subsystem
Sometimes VM may get a kernel panic when starting, and SPDK CI will kill
`nvmf_tgt` after 60 seconds, and for this exception, SPDK will raise an
assertion when destroying the subsystem, while here, we remove this
assertion and print the error information.

CI will still mark this case as a failed case, then we can use this error
information to understand error subsystem state in vfio-user.

Fix issue #2590.

Change-Id: I20b16f9e96a566730eca2dd9ea165645bd9160bd
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13773
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-08-02 01:26:10 +00:00
Jacek Kalwas
8c35e1bd79 nvmf/rdma: remove lock on few transport ops
it simplifies the code and improves readability
sync is done on generic layer

Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: If324039ef2b26fa8ba026b80ec49788a7b2dcaa3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13667
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-07-29 16:34:41 +00:00
Jacek Kalwas
c7ac84d1f2 nvmf/tcp: remove lock on few transport ops
it simplifies the code and improves readability
sync is done on generic layer

Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: I75753511842dff237bb27561e406c43ea68269fe
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13666
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-07-29 16:34:41 +00:00