Commit Graph

9841 Commits

Author SHA1 Message Date
Jim Harris
bb926e803d nvmf: make poll groups count unassociated qpairs
We make decisions on how to pick a poll group for a new
qpair by looking at each poll group's current_io_qpairs
count.  But this count isn't always accurate since it
doesn't get updated until after the CONNECT has
been received.

This means that if we accept a bunch of connections
all at once, they may all get assigned the same poll
group, because the target poll groups counter doesn't
get immediately incremented.

So add a new counter, current_unassociated_qpairs,
to account for these qpairs.  We protect this counter
with a lock, since the accept thread will increment
the counter, and the poll group thread will
decrement it when the qpair receives the CONNECT
allowing us to associated with a subsystem/controller..

If the qpair gets destroyed before the CONNECT is
received, we can use the qpair->connect_received
flag to decrement current_unassociated_qpairs.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I8bba8da2abfe225b3b9f981cd71b6f49e2b87391
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15693
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2022-12-01 10:57:29 +00:00
Jim Harris
f3e197ff18 nvmf: add qpair->connect_received
Currently we use qpair->ctrlr at qpair destroy
time to decide if we need to decrement the
qpair's poll group's qpair count.  But this is
not correct - these counters get incremented
when the CONNECT is received, but qpair->ctrlr
doesn't get set until later.

So add a new connect_received bool to the spdk_nvmf_qpair.
Use this instead to determine when we should decrement
the poll group qpair counters.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I174a0fda36c4558171953bf58f2f5117bc074f76
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15692
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2022-12-01 10:57:29 +00:00
John Levon
478c0fa852 lib/nvmf: don't report invalid identify controller CNS
At least recent Linux guest VMs send SPDK_NVME_IDENTIFY_CTRLR_IOCS as a
matter of course. While this isn't supported in lib/nvmf, as this
doesn't represent an error, reduce the log level of the error message so
we don't spam the logs.

Signed-off-by: John Levon <john.levon@nutanix.com>
Change-Id: I095de3e4331b3912cbc457da6d722b9883ec7884
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15646
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Community-CI: Mellanox Build Bot
2022-11-30 08:51:00 +00:00
GangCao
cebb63a7a7 lib/virtio: add the ctx NULL check before dereferencing it
Issue is found in the virtio_pci_scsi_dev_create() whose
error path is setting the vdev->ctx to NULL before the
destruct operation.

Change-Id: I4ab0fbe300f7413ad4503833088856aa3f4c0734
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15676
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-11-30 08:50:48 +00:00
Artur Paszkiewicz
fed1f52b9e nvmef: don't set optimal I/O boundary if write_unit_size != 1
Optimal I/O boundary causes I/O to be split in the nvme driver. This is
a problem for writes if write_unit_size > 1 because the split I/O may
not match the write_unit_size.

Fixes: #2791
Change-Id: I437e6cb6d8e2415658d5b46539feeacb5363fd46
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15627
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-11-30 08:50:29 +00:00
Evgeniy Kochetov
8305e49b07 nvmf: Add copy command support
NVMf target reports copy command support if all bdevs in the subsystem
support copy IO type. Maximum copy size is reported for each namespace
independently in namespace identify data. For now we support just one
source range.

Note, that command support in the controller is initialized once on
controller create. If another namespace which doesn't support copy
command is added to the subsystem later, it will not be reflected in
the controller data structure and will not be communicated to the
initiator. Attempt to execute copy command on such namespace will
fail. This issue is not specific to copy command and applies also to
write zeroes and unmap (dataset management) commands.

Signed-off-by: Evgeniy Kochetov <evgeniik@nvidia.com>
Change-Id: I5f06564eb43d66d2852bf7eeda8b17830c53c9bc
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14350
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2022-11-30 08:50:06 +00:00
Thanos Makatos
6be6e9f298 nvmf/vfio-user: drop thread from struct nvmf_vfio_user_cq
The correct SPDK thread is already contained in the poll group.

Signed-off-by: Thanos Makatos <thanos.makatos@nutanix.com>
Change-Id: I4eefe2ba60c77c01a866a693bccbb8affc8262ed
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15546
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: John Levon <levon@movementarian.org>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-11-30 08:47:31 +00:00
Thanos Makatos
79abd0f034 nvmf/vfio-user: use define instead of hardcoded value
Change-Id: Ia24ba290da3476d452974bfe08e2e93ae44f954e
Signed-off-by: Thanos Makatos <thanos.makatos@nutanix.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15544
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: John Levon <levon@movementarian.org>
2022-11-30 08:47:31 +00:00
Thanos Makatos
954b145ba1 nvmf/vfio-user: add poll group stats
This patch adds some basic stats for nvmf/vfio-user poll groups.

Signed-off-by: Thanos Makatos <thanos.makatos@nutanix.com>
Change-Id: Ifd9621a8dd4f5f89713582ee5c7b408ff49f43bb
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15390
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: John Levon <levon@movementarian.org>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-11-30 08:47:31 +00:00
melon.masou
565a44628d iscsi: fix segfault when r2t
Fixes #2781

This patch fixes two issue causing segfault on r2t:
1. pdu buffer is allocated from immediate_data_pool, but data_buf_len is set as data_out_pool
2. task->desired_data_transfer_length is rewrite by iscsi_send_r2t, which causes a wrong calculated pdu->data_buf_len

Signed-off-by: melon.masou <melon.masou@outlook.com>
Change-Id: I151859afff7104f29ad7f0ec57a8479d88b742bd
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15542
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-11-29 17:21:18 +00:00
GangCao
99a43e75ed lib/sock: use_after_free of the group_impl point
Change-Id: I9d19e469b4c84b09de5a3938238687f7650452ef
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15619
Reviewed-by: wanghailiang <hailiangx.e.wang@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2022-11-29 08:30:29 +00:00
Richael Zhuang
f192c11bbf bdev: support to get histogram per channel
Added new API 'spdk_bdev_histogram_get_channel' to get histogram of
a specified channel for a bdev. A callback function is passed to it
to process the histogram.

Change-Id: If5d56cbb5fe6c39cda7882f887dcc9c6afa769ac
Signed-off-by: Richael Zhuang <richael.zhuang@arm.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15539
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
2022-11-29 08:28:57 +00:00
wanghailiangx
0da97a15cc lib/bdev: print num_blocks and the write_unit_size in SPDK_ERRLOG
Print out the specific values in this SPDK_ERRLOG,
this can help to find where the error is.

Change-Id: I2a38aa2d4270e0bbf554ddb348a73d40967d1b16
Signed-off-by: wanghailiangx <hailiangx.e.wang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15618
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: GangCao <gang.cao@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Mellanox Build Bot
2022-11-28 09:46:12 +00:00
GangCao
c85df53551 lib/virtio: handle double free of virtio_dev device
Change-Id: I76a3f9125d05aa6ca0c31e8220036cf853a24619
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15617
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Mellanox Build Bot
2022-11-25 08:14:25 +00:00
Ben Walker
85478eccc9 thread: Fix error handling in spdk_interrupt_register
If the calloc failed, the fd was left in the fd_group.

Change-Id: Ie68426a13d342756c20315656f0309440fda6e02
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15475
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Thanos Makatos <thanos.makatos@nutanix.com>
Reviewed-by: John Levon <levon@movementarian.org>
2022-11-24 10:08:31 +00:00
Mike Gerdts
8dbaca1300 bdev: use spinlock instead of mutex
SPDK threads generally run on dedicated cores and locks should be rarely
contended. Thus, putting a thread to sleep while waiting on a mutex does
not free up CPU cycles for other pthreads or processes. Even when
running in interrupt mode, lock contention should be low enough that
spinlocks are a net win by avoiding context switches.

Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: I6e2e78b2835bbadb56bbec34918d998d75280dfd
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15438
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-11-24 10:08:17 +00:00
Jim Harris
2be196c609 nvme/pcie: validate that mptr is iova contiguous
Also add unit tests that explicitly test this
condition.  They fail without the nvme driver changes
in this patch.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iaa369be341eb4eba394f248990e56dce001d3940
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15579
Reviewed-by: Mariusz Barczak <mariusz.barczak@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-11-23 08:23:15 +00:00
Jim Harris
1d2700d4c1 event: check that all non-app threads have exited at shutdown
For now, just print a loud warning when this case is
violated.  We will add a hard assertion and cause the app
to exit with error status in a later release.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ic9226f76a4729820f13a2728bea977b6a54f48ee
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15513
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2022-11-23 08:22:04 +00:00
Jim Harris
8203e68e24 thread: add spdk_thread_is_running()
This function can be useful to query if a thread
had spdk_thread_exit() called on it yet.  Internally
we have both EXITING and EXITED state - so
!spdk_thread_is_running() can be used to detect a
thread that is in either of those states.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I2f6fb024a6b1bc895fdc5132c722abc10f5d30f9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15512
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2022-11-23 08:22:04 +00:00
Jim Harris
98ceddb47c rocksdb: remove spdk_thread
This was an accidental remnant from the original
check-in, when we did not have a clear differentiation
between the event and thread libraries.

The rocksdb plugin code will send events to an
lcore - not an SPDK thread.  But originally the two
were combined though an API called spdk_allocate_thread.
Once the differentiation was clearly made, we moved to
using spdk_event_allocate() to send events to a specific
lcore, but never removed the spdk_thread.

So now let's just remove the spdk_thread_create since
it is not needed.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I5c6a3c304b7b4183eee90038367fdea7ebd7280f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15504
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2022-11-23 08:22:04 +00:00
Jim Harris
0d3b54825e subsystem: assert all subsystems initialized on app thread
This requires creating and setting SPDK threads in the
subsystem unit tests as well.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I31acfb1d7e418f011acc9b48933032d8bf8a1c53
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15511
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2022-11-23 08:22:04 +00:00
Jim Harris
327d1c988d vhost: defer vhost_dev_unregister until scsi tgts removed
Currently when a vhost-scsi controller is removed, it
calls spdk_vhost_scsi_dev_remove_tgt on all remaining
targets, and then immediately calls vhost_dev_unregister.
But this path goes into vhost_user_dev_unregister which
immediately returns with error if there are any pending
async operations - and there are since scsi_dev_remove_tgt
is asynchronous.

So instead add the vhost_dev_unregister call to
remove_scsi_tgt, so that the unregister only happens
after the last ref goes away.

This requires changing vhost_fini() to no longer
assume that spdk_vhost_dev_remove() will immediately
unregister the device, since it now happens
asynchronously.  Previously vhost_fini() was making
this assumption erroneously - it would call g_fini_cb
without actually checking that the devices had been
unregistered.  Because of that incorrect assumption,
we need to do both the vhost and vhost-scsi changes
in the same patch.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I9577901266975447f9acfe53475221113f02fea3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15510
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2022-11-23 08:22:04 +00:00
Jim Harris
85d70c03c5 thread: don't move to EXITED if there are pending messages
At end of spdk_thread_poll(), if thread is in EXITING state,
we call thread_exit() to see if the thread can move to
EXITED state.  If there are any pollers, io_channels
or pending device unregistrations in progress, thread_exit()
will keep the thread in EXITING mode for this iteration.

But a thread may post messages to itself during this cleanup
process, so thread_exit() should also check if there are
any messages on its queue.

Found during testing of spdk_thread lifetime patch set.
rbd bdev module will send messages to itself like this
during cleanup.  Without this change, rbd module testing
with bdevperf would cause an spdk_thread to move to
EXITED state prematurely.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ie611026a67b7fa48640ae83be03e29a9c64883a2
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15533
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2022-11-23 08:22:04 +00:00
Jim Harris
b35aceb8cf iscsi: unregister login_timer when destroying connection
If a connection is established and we receive a bad
PDU before successful login, the login_timer would not
get unregistered.  So ensure the login_timer is always
unregistered in _iscsi_conn_destruct().

Found with Calsoft tests during new spdk_thread_exit()
assertion testing.  Lack of unregistration would result
in its associated spdk_thread being unable to exit
cleanly due to the unexpired timer.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I79d427512f7829ad76bf89155e0e14c7bce3a7d7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15499
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-11-23 08:22:04 +00:00
Jim Harris
090b8af12b thread: add spdk_thread_get_app_thread
The "app thread" will always be the first
thread created using spdk_thread_create().  There
are many operations throughout SPDK that implicitly
expect to happen in the context of this app thread,
so by formalizing it we can start to make assertions
on this to help clarify and simplify locking and
synchronization through the code base.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I7133b58c311710f1d132ee5f09500ffeb4168b15
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15497
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-11-23 08:22:04 +00:00
Jim Harris
db18916f29 thread: move _free_thread() earlier in file
Next patch will add a new caller to this function.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I54374c0af3a4a0fdcc5ac9ca25e2c7ef03e99829
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15576
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-11-23 08:22:04 +00:00
Changpeng Liu
b45556e2b2 include/bdev_module.h: add SPDK_ prefix to macros
`BDEV_IO_NUM_CHILD_IOV` and `BDEV_RESET_IO_DRAIN_RECOMMENDED_VALUE`
are public macro definitions without `SPDK_` prefix, so we add the
`SPDK_` prefix to them.

Change-Id: I4be86459f0b6ba3a4636a2c8130b2f12757ea2da
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15425
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-11-22 10:03:57 +00:00
yupeng
c0c333e2ed bdev: provide all available bdevs when loop bdevs
The bdev hot remove might be an async process. The bdev_open will
return an error during the hot remove process. If someone invoke the
bdev_get_bdevs API when a bdev is in the middle of a hot remove
process, the spdk_for_each_bdev function will stop its loop when a
bdev_open return an error. Thus the bdev_get_bdevs will only return
partual bdevs or even return an empty list if the hot remove bdev is
the first bdev in the loop. When spdk_for_each_bdev and
spdk_for_each_bdev_leaf loop for each bdevs, if a bdev returns an
error, we skip that bdev instead of stop the whole loop.

Signed-off-by: Peng Yu <yupeng0921@gmail.com>
Change-Id: Ib35b817e23e47569fc5762a883b4ff8e322ae173
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15322
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Mike Gerdts <mgerdts@nvidia.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2022-11-22 10:03:48 +00:00
Thanos Makatos
70f185ea51 json: add spdk_json_write_named_double
Signed-off-by: Thanos Makatos <thanos.makatos@nutanix.com>
Suggested-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I2439cd739240fb2d95c5cdaccc557ba9a8f6501b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15490
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: John Levon <levon@movementarian.org>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-11-22 10:01:43 +00:00
Thanos Makatos
4475295e15 nvmf/vfio-user: add some unlikely on the hot path
Signed-off-by: Thanos Makatos <thanos.makatos@nutanix.com>
Change-Id: Ib7977f34fc2fc312f0a502405dcd1b5266a22d3f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15430
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: John Levon <levon@movementarian.org>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-11-22 10:01:43 +00:00
Thanos Makatos
6b71006dfe nvmf/vfio-user: refactor nvmf_vfio_user_prop_req_rsp
Signed-off-by: Thanos Makatos <thanos.makatos@nutanix.com>
Change-Id: Id6b0a4bc12aa8799fdb1ce1b286c308c9a79083b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15389
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: John Levon <levon@movementarian.org>
2022-11-22 10:01:43 +00:00
Thanos Makatos
82b2c1923f nvmf/vfio-user: refactor duplicate code
Change-Id: If501002e9ed110f77a4ece9f026ecfc4e53dee27
Signed-off-by: Thanos Makatos <thanos.makatos@nutanix.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15388
Reviewed-by: John Levon <levon@movementarian.org>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-11-22 10:01:43 +00:00
Thanos Makatos
a7885283b3 nvmf/vfio-user: delete CQ on vfio-user client disconnect
If the guest performs a hard shutdown we're not deleting the CQs:
nvmf_vfio_user_close_qpair calls delete_sq_done, which won't delete
the CQ because vu_ctrlr->reset_shn is false.

Signed-off-by: Thanos Makatos <thanos.makatos@nutanix.com>
Change-Id: I383fb985340a0d9d0eb7fea7403372cbdc55a089
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15387
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: John Levon <levon@movementarian.org>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-11-22 10:01:43 +00:00
Thanos Makatos
e398dcdadb nvmf/vfio-user: don't use uninitialized refcount for admin CQ
Signed-off-by: Thanos Makatos <thanos.makatos@nutanix.com>
Change-Id: I16d511ac10b8ba4dfb2f7a7e5c144e2f2fe1bad5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15386
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: John Levon <levon@movementarian.org>
2022-11-22 10:01:43 +00:00
Thanos Makatos
25440c3bdb nvmf/vfio-user: don't blindly drain poll group eventfd
This eventfd may be passed by libvfio-user to the remote process which
might remove the EFD_NONBLOCK flag, in which case we would block
indefinitely.

Signed-off-by: Thanos Makatos <thanos.makatos@nutanix.com>
Change-Id: If9826cd700b4a7b3458a0a8278a96322d99ac08e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15385
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: John Levon <levon@movementarian.org>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2022-11-22 10:01:43 +00:00
Thanos Makatos
7f23638550 util: add function spdk_fd_group_get_epoll_event
This patch introduces function spdk_fd_group_get_epoll_event, which
returns the epoll(7) event that caused the file descriptor group
callback function to execute. Rather than changing the signature of
spdk_fd_fn in order to pass the struct epoll_event, which would result
in a gigantic patch where there vast majority of users would simply have
to ignore the new argument, we introduce this new API that allows to
return the epoll_event only when really needed.

Signed-off-by: Thanos Makatos <thanos.makatos@nutanix.com>
Suggested-by: John Levon <john.levon@nutanix.com>
Change-Id: I3debe1382d1c2bfec6ae4fea274ee38ed0b135fe
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14935
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: John Levon <levon@movementarian.org>
2022-11-22 10:01:43 +00:00
Kozlowski Mateusz
304f0802d1 lib/ftl: Fix segfault in recovery path of unmap
The ftl_md_get_buffer_size returns the buffer size in bytes, so we
should divide by the block size, instead of this smaller value. Risks
touching bad memory during dirty shutdown recovery, especially in >16TiB
drives.

Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Signed-off-by: Mariusz Barczak <mariusz.barczak@intel.com>
Change-Id: I4095b00a79a1bdbce5046dc46349a9670e41b18e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15259
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-11-22 10:01:30 +00:00
Kozlowski Mateusz
6a26cb6053 lib/ftl: Fix findings of static code analysis
A metadata region without mirror should have the INVALID enum set,
otherwise it risks touching invalid parts of the array.

The sb_shm_md not being set to NULL could cause the code to touch this
freed pointer in the error path in ftl_md_create -> ftl_md_create_shm ->
ftl_md_invalidate_shm calls.

Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Signed-off-by: Mariusz Barczak <mariusz.barczak@intel.com>
Change-Id: I7fe9694dad535de5f6b2a4af27400fa125480605
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15258
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-11-22 10:01:30 +00:00
Kozlowski Mateusz
646b851e75 lib/ftl: Update FTL IO activity statistics
Bumping the IO activity statistics during relocation, compaction, L2P
cache processing and user IO handling. This makes sure poller busy
counter is more accurate.

Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Change-Id: Iabf8ec7ca41c01d7a00d3a70825b8d5283ab2bf1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15257
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Mellanox Build Bot
2022-11-22 10:01:30 +00:00
Kozlowski Mateusz
a7f4a2db7f lib/ftl: Validate l2p_dram_limit parameter
Disallow 0 value as parameter - avoids a segmentation fault.

Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Signed-off-by: Mateusz Brenk <mateusz.brenk@intel.com>
Change-Id: I492256ff621da3be11239d2fd705d8cc54bfe7b7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15256
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2022-11-22 10:01:30 +00:00
Konrad Sztyber
72a6cd5381 nvme: execute hotplug monitor even if hotplug_fd < 0
NVMe controllers can be marked as removed even if we cannot receive
uevents (e.g. by the VMD driver), so we should process them regardless
of hotplug_fd.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Iaaf13a136929200e824f7a6dd3b5584998801630
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15547
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tom Nabarro <tom.nabarro@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-11-21 16:15:44 +00:00
Konrad Sztyber
0a672ea974 rpc: print device type in framework_get_pci_devices
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I2d3825ffcce098909745ba949cdde3eb7f71c703
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15545
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tom Nabarro <tom.nabarro@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-11-21 16:15:44 +00:00
Konrad Sztyber
806c100595 rpc: extend bdf buffer in framework_get_pci_devices
The previous 14B buffer was too small for VMD devices.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ib3984f7104fadbb2fbf7ec56932675d73eda1456
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15532
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tom Nabarro <tom.nabarro@intel.com>
2022-11-21 16:15:44 +00:00
Konrad Sztyber
86ba16c39c build: compile API functions with missing deps
We should always build all function that are part of the API, even if
some of the libraries they depend on are missing.  In that case, they
can return an error instead.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I72b450b3a1d62e222bd843e45be547d926414775
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15414
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-11-18 08:40:05 +00:00
Krzysztof Karas
a1c7ae2d3f bdev: remove generation of UUIDs for bdevs that do not provide one
Remove automatic generation of UUIDs for bdevs
that do not provide this value themselves.
This is to clarify whether this field can be
depended upon.

Modified match files to reflect change in UUID
generation.

Disabled nullglob shell option, as it deletes
empty arrays during word splitting. Bdevs with no
aliases would instead of "[]", have nullpointer
printed, which makes resulting JSON invalid.

Part of enhancement proposed in #2516.

Change-Id: Ic1d5f8f8d001ae1a219e876aef2a19b1ff0b2f2c
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15150
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Mellanox Build Bot
2022-11-18 08:38:13 +00:00
wanghailiangx
f6a256c013 lib/accel: set RPC accel_get_opc_assignments as SPDK_RPC_RUNTIME
Add the processing of returning 0 for spdk_accel_get_opc_module_name(),
and remove SPDK_RPC_STARTUP, because this will cause core dumped
when run nvmf_tgt with --wait-for-rpc and no RPC framework_start_init.

Fixes issue: 2770

Change-Id: I1c53ccb8caa52f2eaa0b8b560a021bded49d8fed
Signed-off-by: wanghailiangx <hailiangx.e.wang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15377
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2022-11-17 08:57:53 +00:00
Shuhei Matsumoto
2356d1d6f3 bdev: Add helper functions to allocate/free bdev_get_iostat_ctx
Add helper functions, bdev_iostat_ctx_alloc() and bdev_iostat_ctx_free()
for the bdev_get_iostat RPC.

The following patches will allocate spdk_bdev_io_stat dynamically for
bdev_get_iostat_ctx.

This is a preparation for that.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: Ib71d6fb92d8134d2282507e62874f19045b630b7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15442
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2022-11-17 08:56:49 +00:00
Shuhei Matsumoto
7c687dfcbd bdev: Clarify bdev_ctx and rpc_ctx for bdev_get_iostat RPC
The bdev_get_iostat RPC uses two types of contexts, one to manage the
progress of the bdev_get_iostat RPC and another to call
spdk_bdev_get_device_stat().

However, this was hard to find from the source code.

To make us easier to find this, rename the former by rpc_ctx and the
latter by bdev_ctx. Then rename related functions and variables accordingly.

Furthermore, relocate request and decoder declaration to improve
readability.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I3472c87fe4ec1f5981a49ef79148534fbb1d46c4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15349
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-11-17 08:56:49 +00:00
Shuhei Matsumoto
038fb90350 bdev: Move down RPC parameters and decoders for bdev_get_iostat RPC
RPC parameters and decoders for the bdev_get_iostat RPC are used only
by rpc_bdev_get_iostat(). Locating RPC parameters and decoders close to
rpc_bdev_get_iostat() clarifies it. Furthermore, this will simplify code
review for the next patch.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I1b1b428e3eb3bb4422e490c5f4324f0e40f9710f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15416
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-11-17 08:56:49 +00:00
Shuhei Matsumoto
0ac95a684b bdev: Consolidate two TRACE_BDEV_IO_START calls into a single call
For I/Os controlled by QoS, TRACE_BDEV_IO_DONE is collected after
redirecting to the original thread. Hence, TRACE_BDEV_IO_START should
be collected on the original thread too.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I15411be823450ee5ddaa7582509a7aa068476fc5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14824
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-11-17 08:56:49 +00:00