spdk_nvme_ctrlr_get_registers() calls nvme_get_transport()
to get a reference for a transport, whose registers should
be returned, but nvme_get_transport() explicitly returns
NULL, if the transport does not exist. This would result
in dereferencing a NULL pointer on line 862.
To remedy that, if no transport was found, return NULL.
Additionally change "THis" to "This" on line 46.
Change-Id: I3944925659991e9424e2177b5c940b2e2626d1f4
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17532
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
When removing a listener, for example with
nvmf_subsystem_remove_listener RPC, we use the concept of a
"listen trid" to determine which existing connections
should be disconnected.
This listen trid has the trtype, adrfam, traddr and trsvcid
defined, but *not* the subnqn. We use the subsystem pointer
itself to match the subsystem.
nvmf_stop_listen_disconnect_qpairs gets the listen trid
for each qpair, compares it to the trid passed by the
RPC, and if it matches, then it compares the subsystem
pointers and will disconnect the qpair if it matches.
The problem is that the spdk_nvmf_qpair_get_listen_trid
path does not initialize the subnqn to an empty string,
and in this case the caller does not initialize it either.
So sometimes the subnqn on the stack used to get the
qpair's listen trid ends up with some garbage as the subnqn
string, which causes the transport_id_compare to fail, and
then the qpair won't get disconnected even if the other
trid fields and subsystem pointers match.
For the failover.sh test, this means that the qpair doesn't
get disconnected, so we never go down the reset path
on the initiator side and don't see the "Resetting" strings
expected in the log.
This similarly impacts the host/timeout.sh test, which is
also fixed by this patch. There were multiple failing
signatures, all related to remove_listener not working
correctly due to this bug.
While the get_listen_trid path is the one that caused
these bugs, the get_local_trid and get_peer_trid paths
have similar problems, so they are similarly fixed in
this patch.
Fixes issue #2862.
Fixes issue #2595.
Fixes issue #2865.
Fixes issue #2864.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I36eb519cd1f434d50eebf724ecd6dbc2528288c3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17788
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Mike Gerdts <mgerdts@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: <sebastian.brzezinka@intel.com>
This introduces an examine_config callback that triggers hotplug of
missing esnap devices.
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: I5ced2ff26bfd393d2df4fd4718700be30eb48063
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16626
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
spdk_lvol_get_by_uuid() allows lookup of lvols by the lvol's uuid.
spdk_lvol_get_by_names() allows lookup of lvols by the lvol's lvstore
name and lvol name.
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: Id165a3d17b76e5dde0616091dee5dff8327f44d0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17546
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Add an interator that calls a callback for each clone of a snapshot
volume. This follows the typical pattern of stopping iteration when the
callback returns non-zero.
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: If88ad769b72a19ba0993303e89da107db8a6adfc
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17545
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This introduces spdk_lvs_notify_hotplug() to trigger the lvstore to call
the appropriate lvstore's esnap_bs_dev_create() callback for each esnap
clone lvol that is missing the device identified by esnap_id.
Change-Id: I0e2eb26375c62043b0f895197b24d6e056905aa2
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16428
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
If an lvol is opened in degraded mode, keep track of the missing esnap
IDs and which lvols need them. A future commit will make use of this
information to bring lvols out of degraded mode when their external
snapshot device appears.
Change-Id: I55c16ad042a73e46e225369bfff2631958a2ed46
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16427
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
spdk_blob_is_clone() should return true only for normal clones. To
detect esnap clones, use spdk_blob_is_esnap_clone(). This also clarifies
documentation of spdk_blob_is_esnap_clone() to match the implementation.
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: I9993ab60c1a097531a46fb6760124a632f6857cd
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17544
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
The health of clones of esnap clones depends on the health of the esnap
clone. This allows recursion through a chain of clones so that degraded
state propagates up from any back_bs_dev that is degraded.
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: Iadd879d589f6ce4d0b654945db065d304b0c8357
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17517
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
In preparation for supporting degraded lvols, spdk_blob_is_degraded() is
added. To support this, bs_dev gains an optional is_degraded() callback.
spdk_blob_is_degraded() returns false so long as no bs_dev that the blob
depends on is degraded. Depended upon bs_devs include the blobstore's
device and the blob's back_bs_dev.
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: Ib02227f5735b00038ed30923813e1d5b57deb1ab
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17516
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
While getting memory domains, vbdev_lvol will need to be able to access
the bdev that acts as the lvol's external snapshot. The introduction of
spdk_blob_get_esnap_bs_dev() facilitates this access.
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: I604c957a468392d40b824c3d2afb00cbfe89cd21
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16429
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
For operations that have differently sized input/output buffers (e.g.
compress, decompress), the size of the src buffer is recorded.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I1ee47a2e678ac1b5172ad3d8da6ab548e1aa3631
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17624
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
It is now possible to append an operation calculating crc32c to an accel
sequence. A crc32c operation needs special care when it's part of a
sequence, because it doesn't have a destination buffer. It means that
we can remove copy operations following crc32c only when it's possible
to change the dst buffer of the operation preceding crc32c.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I29204ce52d635162d2202136609f8f8f33db312d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17427
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
This will reduce the amount of changes in the following patch which
makes this function recursive.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: If8da6ae52d78358b66b2d9303413a9723687a767
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17568
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
g_stats_lock is an spdk_spin_lock that is initialized as the module is
loading. With this change, it is destroyed as the module finishes.
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: I5263547f6d0e8981765d59665bd826cf07a6f83e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17681
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
This ensures that there are no more outstanding operations, so we can
safely free any global resources.
Fixes#2987
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Iac423b4f2a1183278d1db20f96c1a3b1bb657f85
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17767
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Mike Gerdts <mgerdts@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Issue #2983 shows a case where we seem to get a
device remove notification from DPDK (via vfio
path) after we have already detached the device
explicitly by SPDK.
This issue has proven difficult to reproduce
outside of the one observed failure so far, so
adding a couple of ERRLOGs into this path to help
confirm the this theory should it happen again.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I0fda4229fe150ca17417b227e8587cd7fbda6692
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17631
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
When using `__lsan_do_recoverable_leak_check` (e.g when fuzzing),
to check for leaks during runtime. Leak sanitizer can not follow
reference of memory that is allocated on heap (e.g. calloc)
and then stored on hugepage causing lsan to incorrectly report
direct leak.
Fixes#2967
Signed-off-by: Sebastian Brzezinka <sebastian.brzezinka@intel.com>
Change-Id: I3511e117a07ca8daa96f19bf1437c0d788b64cb1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17682
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Amir Haroush <amir.haroush@huawei.com>
This was arbitrarily picked as 2 seconds in commit
0e3dbd. But for extremely high connection count
use cases, such as nvme-perf with several cores
and high connection count per core, this 2 second
time window can get exceeded.
So increase this to 10 seconds, but only for qpairs
that are being connected asynchronously.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I906ca9e6561b778613c80b739a20bd72c807216c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17619
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
lvs_load() function verifies if options passed to it
are valid, but doesn't return, if they are not (only error
is logged and callback is called with -EINVAL code). Now
it is corrected and the function ends after the error
is reported.
Change-Id: I19b0b22466b6980345477f62084d27ef13414752
Signed-off-by: Marcin Spiewak <marcin.spiewak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17582
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Mike Gerdts <mgerdts@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
When changing src/dst buffers, we copied memory domain pointers, but we
didn't copy memory domain context, which is obviously incorrect. It was
probably missed, because we never append a copy with non-NULL memory
domain. Added a unit test case to verify this behavior.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ic174e0e72c33d3f437f0faddd3405638049f0c74
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17425
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This file should be external to enable out-of-tree accel modules.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I2e973d0e88d7145d0fc9714f56db48486b00f3b7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17419
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
It always returns 0 and any errors are reported in the callback. Making
it void simplifies error handling.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I0d4299a2789a688eae38d76de46d1baf27cbbd8f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17194
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Deprecation notice for pmem was removed, as libreduce will
still use it until pmem is supported.
Change-Id: I7555dbf20a408a67fac8a6e7b2eaa23edf985eec
Signed-off-by: Marcin Spiewak <marcin.spiewak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17538
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
To make sure we don't lose statistics of destroyed channels, they're now
added to a global stats structure when a channel is destroyed.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ic3b4d285b83267ac06fad1e83721c1b15cc8ec8a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17567
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
The RPC allows the user to retrieve accel framework's statistics.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I5cd1b45686504c08eda50513ad1dae2f8d65013b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17191
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This patch adds support for collecting statistics in accel framework.
Currently, it counts the following events:
1. The number and the type of executed/failed operations.
2. The number of executed/failed accel sequences.
For now, these statistics are only collected and there's no way of
retrieving (or resetting) them - that will be added in the following
patches.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Id211067eb810e7b7d30c756a01b35eb5019c57e7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17190
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
It'll allow for setting accel-specific options. For now, it makes the
size of iobuf caches configurable.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Iaf505cc5e98dc6411453d9964250a4ba22267d79
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17188
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
In case SPDK is build with shared libraries and there's no
LD_LIBRARY_PATH around, linker will complain about missing .sos
similar to:
/usr/bin/ld.bfd: warning: librte_meter.so.23, needed by
/root/spdk/dpdk/build/lib/librte_ethdev.so, not found (try using -rpath
or -rpath-link)
We can't see that under CI since autotest_common.sh always makes sure
the LD_LIBRARY_PATH is properly set.
Add the -rpath to make the build less spammy.
Change-Id: I1d9d1775b2aa24e65cc4b776c2549457b0d7aac3
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17492
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
In commit 23baa67, we will start virtio device only once,
and update the VQ's information in SET_VRING_KICK message
context, so when multi-queues are enabled, SPDK doesn't
register VQ's interrupt handler, here we add it when enable
VQ.
Fix issue #2940.
Change-Id: I29dbd7bf0b81b23c2e47e37c467952cc5887b5bf
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17354
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
bdev modules should have call spdk_bdev_io_complete twice
for the same IO. We can help find cases where this happens
by adding an assert in spdk_bdev_io_complete - confirming
that the current status is still PENDING, before changing
it to the status passed by the caller.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Id8a044a94113f1ac5e3c8d86e426654bfa8d5c5a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17330
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reset the status for a bdev_io that fails with NOMEM status
back to PENDING immediately when it is put on the nomem_io
list, instead of waiting until it gets submitted again.
This helps keep the bdev_io states consistent, so that if
we need to complete these IO for abort reasons later, that
we aren't completing IO that already have a non-PENDING
state.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I9532095141209ed6f7af362b52c689da62e755ce
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17335
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
Handling this callback is quite complex and may lead to
various problems. In most of places, the actual event
when qpair is dosconnected is not importnat for the
app logic. Only in shutdown path we need to be sure
that all qpairs are disconnected, it can be achieved
by checking poll_group::qpairs list
Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: I453961299f67342c1193dc622685aefb46bfceb6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17165
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Michael Haeuptle <michaelhaeuptle@gmail.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
If the qpair is already in the process of disconnect,
the spdk_nvmf_qpair_disconnect API now return -EINPROGRESS
and doesn't call the callback passed by the user.
Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: If996b0496bf15729654d18771756b736e41812ae
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17164
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michael Haeuptle <michaelhaeuptle@gmail.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Current implementation of spdk_nvmf_qpair_disconnect
saves and calls user's callback correctly only on
the first call. If this function is called when
qpair is already in the process of disconnect, the
cb_fn is called immediately, that may lead to stack
overflow.
In most of places this function is called with
cb_fn = NULL, that means that the real qpair disconnect
is not important for the app logic. Only in several
places (nvmf tgt shutdown flow) that is important to
wait for all qpairs to be disconnected.
Taking into account complexity related to possible stack
overflow, do not pass the cb_fn to spdk_nvmf_qpair_disconnect.
Instead, wait until a list of qpairs is empty in shutdown path.
Next patches will change spdk_nvmf_qpair_disconnect behaviour
when disconnect is in progress and deprecate cb_fn and ctx
parameters.
Fixes issue #2765
Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: Ie8d49c88cc009b774b45adab3e37c4dde4395549
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17163
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Michael Haeuptle <michaelhaeuptle@gmail.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This patch removes usage of cb_fn argument of
spdk_nvmf_qpair_disconnect API. Instead of relying
on the callback, post a completion on delete SQ
command when transport qpair_fini is called.
Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: I68dec97ea94e89f48a8667da82f88b5e24fc0d88
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17168
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Make sure UUID is present for every bdev, even ephemeral ones.
Furthermore, this change removes assumption that bdev UUID
may remain empty.
Change-Id: I924c1ba9dedfe88a05044bb1073f28085735b1c1
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17106
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Commit aaba5d introduced a build warning with some
compilers. While fixing it, I realized the function was
difficult to immediately understand. So in addition to fixing
the build warning, I also made the following changes:
* Improved names for local variables
* Use TAILQ_INIT for local TAILQ instead of TAILQ_HEAD_INITIALIZER.
* Add comments explaining more clearly what the nested loops are
doing.
* Use TAILQ_SWAP instead of a FOREACH + REMOVE + INSERT.
Fixes: aaba5d ("subsystem: Gather list changed conditions.")
Fixes issue #2978.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ic8740b5706537938d62a0acfac62625b2424b85f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17496
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Mike Gerdts <mgerdts@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
rte_rcu is available on all versions of DPDK supported by SPDK. It is
also required by quite a few DPDK libraries. So just include
it always, it's a small library so let's not try to over-complicate by
trying to figure out exactly when it's needed.
This change fixes linking issue when crypto enabled (and vhost not).
Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: Ibdd6acb5a25c401b462022bbd94bd380690640d0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17514
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Fixed function is used to determine if it is possible to get iobuf
from the pool. To make sure that buf size alignment requirement is
satisifed value returned shall include alignment value but subtracted
by one.
e.g.
transaction size length = 64k
buffer alignment = 1 byte (no alignment requirement)
metadata length = 0
Without the fix the function returned 64k + 1, now it returns 64k
which is correct behavior and allows to proceed with further command
processing (if max buffer size limit is set to 64k only).
Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: I09104ad21b3652ba1aa5c3805a04b1c6549d04ac
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17513
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
In the presence of hardware offload (for data digest) we may not be
able to immediately release all PDUs to free a connection. Add a
state to wait for them to finish.
Fixes#2862
Change-Id: I5ecbdad394c0296af6f5c2310d7867dd9de154cb
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16637
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Fixes#2812
This patch adds support for '--lcores <map_list>'
parameter in spdk.
This parameter allow mapping of the lcores
to CPU IDs, if the system contains CPUs with IDs
greater or equal to 128 (RTE_MAX_LCORE). Such CPUs
can not be directly included in core mask
specified in '-m <mask>' parameter, as the dpdk
rejects cores if IDs are greater than 127.
The only way to use them in spdk is to map lcore
to CPU using --lcores parameters specified
in command line.
--lcores and -m parameters are mutually
exclusive. Please use only one of them.
Examples:
build/bin/nvmf_tgt --lcores 0@130
build/bin/nvmf_tgt --lcores 0@150,1@151
build/bin/nvmf_tgt --lcores "(5-7)@(10-12)"
build/bin/nvmf_tgt --lcores "(5-7)@(136,138,140)"
Change-Id: Ia92be4499c8daaa936b1a4357d52ae303d6f3eb1
Signed-off-by: Marcin Spiewak <marcin.spiewak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17403
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
This patch adds support for lcore mapping list, which
is needed by spdk if someone wants to use CPUs with IDs
greater than RTE_MAX_LCORE (128). For such CPUs it
is impossible to include them in the core mask (passed
to dpdk as '-c <mask>') as the dpdk doesn't allow
IDs greater than RTE_MAX_LCORE. Therefore they
must be mapped to lower lcore values using
'--lcores <maping_list>' passed to dpdk
Change-Id: If68f15cef2bca9e42a3457bf35477793b58ec53d
Signed-off-by: Marcin Spiewak <marcin.spiewak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17399
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
A host will consider a QID as reusable once it disconnects
from the target. But our target does not immediately
free the QID's bit from the ctrlr->qpair_mask - it waits
until after a message is sent to the ctrlr's thread.
So this opens up a small window where the host makes
a valid connection with a recently free QID, but the
target rejects it.
When this happens, we will now start a 100us poller, and
recheck again. This will give those messages time to
execute in this case, and avoid unnecessarily rejecting
the CONNECT command.
Tested with local patch that injects 10us delay before
clearing bit in qpair_mask, along with fused_ordering
test that allocates and frees qpair in quick succession.
Also tested with unit tests added in this patch.
Fixes issue #2955.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I850b895c29d86be9c5070a0e6126657e7a0578fe
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17362
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Just remove the duplicated Code and make the
Conditions for g_subsystems list to subsystems_list
together.
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Change-Id: I011b550b83d32580bfd25130dab9e44bcbdc1daf
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13753
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
If user passes --no-huge as part of env_context, do
not add other huge-related options to the EAL command
line. Instead emit an error message and return failure, if
any of them were specified explicitly.
Fixes c833f6aa ("env_dpdk: unlink hugepages if shm_id is not specified")
Fixes issue #2973.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I7aa49e4af5f3c333fa1e7dec4e3f5b4b92e7d414
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17483
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
iSCSI connection closes its socket when it is terminated. After the
socket is closed, the connection cannot access to it. However, the iSCSI
fuzz test terminated a connection while processing a text command. The
connection aborted the text command and the corresponding completion
callback accessed the closed socket. This unexpected access caused a
NULL pointer access.
Add a check if conn->sock is not NULL to iscsi_conn_params_update()
to avoid such NULL pointer access. The return type of the most iSCSI
library functions are void. Here, it is enough not to return 0. Hence,
use -ENXIO simply to indicate there is no available socket.
Fixes the issue #2958
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I2c1f58a63ee0a40561a17f81d4b4264061f411f6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17353
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Michal Berger <michal.berger@intel.com>
When bs_create_blob() is creating the internal xattr for the esnap ID,
it errors out if the ID is too long. This error path neglected to set
the return value. It now returns -EINVAL in this case.
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: I6d756da47f41fb554cd6782add63378e81735118
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17292
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Move cq_is_full() closer to its caller post_completion() and along with
fixing comments.
Signed-off-by: Swapnil Ingle <swapnil.ingle@nutanix.com>
Change-Id: I93262d1805f0f9f075c6946ed97cd3006ffba130
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16415
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
From the issue report in #2507
that comparing blob maybe be NULL.
So add assert, that in CI may catch this issue.
And other funtions add this also.
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Change-Id: I98179ec76f2b6785b6921c37373204021c0669b6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12737
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This will make it much easier to mock this library for use in unit
tests.
Change-Id: I7dc835865f75f9e29e8b709a634d30053ada2055
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16296
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This makes it much easier to mock this code in unit tests without having
to mock up the entire thread library.
Change-Id: Ic3d9cb826ae71af780a06f88669c37cef2c9a4ae
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16173
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Now that DPDK v23.03.0 has been released, the version on the main branch
points to the next release, v23.07.0-rc0, so we need to adjust the
version check to enable testing against the main branch.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I37d165111c446612d573c19199e4ace6aa24d191
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17480
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Since there were no ABI changes in the interfaces used by SPDK, the
v22.11 functions are reused.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Iff73405eec197f7ed1752366b6b38c28710a73ec
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17479
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
This allows the pipe to fill up entirely, instead of reserving 1 byte.
This tends to keep copies from the pipe aligned over time.
Change-Id: I4801d62fa839165efef61ea1c83f602931ee7018
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16990
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
If an lvol is opened by way of spdk_bs_blob_open() or spontaneously due
to blobstore activity (e.g. the blobstore is loading), the
esnap_bs_dev_create callback is called with blob_ctx equal to NULL.
Under ideal circumstances, blob_ctx refers to the lvol that is being
opened.
With this change, a NULL blob_ctx triggers lvs_esnap_bs_dev_create() to
look through the lvstore's volumes to find the one that uses a blob id
that matches the id of the blob that is passed in. Now, lvol library
consumers that need to support snapshots can count on both bs_ctx (lvs)
and blob_ctx (lvol) to be non-NULL.
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: Iaaa9cc27664e28e54f0fbd75afe1d6ffbad92580
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16588
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Once an lvol's blob is closed, the lvol should not retain a reference.
Dereferencing lvol->blob could result in a use after free.
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: Ia96a5a488fc96d642aa4cde83c6efa5cff88b068
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16426
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
As an lvstore is opening it calls spdk_bs_load(), which briefly opens
each blob and has no use for external snapshots. Since there is no point
in opening them at this time, don't open them. Once the blobstore has
been loaded, update lvs->load_esnaps so that external snapshots are
opened as the lvols open their blobs.
Change-Id: Ib16c8474300ff4b106aad0baa5b8b38332c23b01
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16424
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This provides the lib/lvol wrapper around blobstore's external
snapshots. Later commits make this work with vbdev_lvol.
The blobstore external snapshot implementation stores an opaque
identifier in an internal xattr. Lvstore uses this to store the
stringified UUID of the bdev that will act as the external snapshot.
This is used by the newly introduced spdk_lvol_create_esnap_clone() to
store the bdev UUID in the blob's metadata.
Change-Id: I58c7b32b656ad1d21a446e3b91e59e655efac7e4
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14977
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
This updates spdk_lvs_opts to be consistent with opther options
structures in that it can now be extended with additional fields.
The fields in spdk_lvs_opts are now documented as well.
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: Ibd93c3a4aa1d2a33ac550d7056a69afece4dc592
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16172
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This prepares for upcoming patch to enable retrying
CONNECT commands when a duplicate QID is detected.
Most important part is moving the spdk_nvmf_request_complete
call into the ctrlr_add_qpair function. This facilitates
deferring the spdk_nvmf_request_complete call when we want
to do a retry.
Note: for adding admin qpair, we now call
spdk_nvmf_request_complete instead of _nvmf_request_complete,
but this is actually no functional change.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I1282be00441851ee2ed3c2dd281e68b4475d3d28
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17361
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
The max qid check is only needed in the add_io_qpair
function, since an admin qpair could never fail this
check.
Upcoming patches will retry a duplicate QID check
after a short period of time. This patch helps
make the upcoming code path more clear, since there
is no need to do this max qid check more than once.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I95f69fc39ae7989d51d36b0d04b5d9a4087a7c4a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17360
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Instead of adding the spdk_fd_group's fd to the reactor fd group, use
the new nesting functionality.
Change-Id: I00727e836da6ba191d5bf778613f31956c9baacf
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15477
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Community-CI: Mellanox Build Bot
These provide a way to nest one fd_group into another in a more
efficient manner than just adding the fd_group's fd to the parent. It
also keeps track of which events belong to which group, so the unnest
operation can be implemented.
Change-Id: I63d63365f1160cce8b4b6388a0ea2003ef424b9e
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15473
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
It is fine to prefetch an invalid address, but ASAN
doesn't like it. So move the prefetch slightly to
make ASAN happy.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ib51ab8890e5fe91d30057f65e1399cfc9dd1dd49
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17432
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This function was intended to be deleted as unused, however it can be
useful for debug and test capabilities.
Its declaration was left in header file, so just adding implementation
for PCIE and VFIO USER transports.
Signed-off-by: Szulik, Maciej <maciej.szulik@intel.com>
Change-Id: I670acb53c2f88a844525a0ecea27143b055f117b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17400
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
This patch makes functions related to Asynchronous Event and error
handling public, so that they can be used in custom nvmf transport
compiled out of SPDK tree.
Signed-off-by: Szulik, Maciej <maciej.szulik@intel.com>
Change-Id: I253bb7cfc98ea3012c179a709a3337c36b36cb0e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17237
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Fixed issue indicated by Klocwork scan. 'name', which potentially
might be NULL, is passed as function parameter. Now the function
name will not point to NULL, and will be the same in interrupt
structure and in event handler.
Change-Id: I5588821139d11288a96f5041703d5b7b71890ad6
Signed-off-by: Marcin Spiewak <marcin.spiewak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17356
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
The nvmf generic transport code creates a mempool of
I/O buffers, as well as its own per-thread cache
of those buffers. The mempool was being created
with a non-zero mempool cache, effectively duplicating
work - we had a cache in the mempool and then another
in the transport layer.
So patch 019cbb9 removed the mempool cache, but the
tcp transport was significantly affected by it. It
uses a default 32 buffers per thread cache which is
very small, it was actually mostly relying on the
mempool cache (which was 512). Performance regression
tests caught this problem, and Karol verified that
specifying a higher buf_cache_size fixed the problem.
So change both the tcp and rdma transports to specify
UINT32_MAX as the default buf_cache_size. If the
user does not override this when creating the transport,
it will be dynamically sized based on the size of
the buffer pool and the number of poll groups.
Fixes: 019cbb9 ("nvmf: disable data buf mempool cache")
Fixes issue #2934.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Idd43e99312d59940ca68402299e264cc187bfccd
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17203
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
Allow transports to specify a default UINT32_MAX
as the buf_cache_size. If user does not override this
when creating the transport, calculate the buf_cache_size
dynamically using the number of poll groups and the
size of the buffer pool (num_shared_buffers). We will
allocate 75% of the buffers for the caches, meaning
the buf_cache_size will be calculated as:
(num_shared_buffers * 3 / 4) / num_poll_groups
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I97768aea701060bbe0ff1925e5322229fa8d051c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17334
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
When buf_cache_size is 0, just return early. This
allows us to un-indent a large section of code.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I167da677fdcd0504c6f2bfdb8b1a818155642f66
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17333
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This will be useful in upcoming patch, where we
use the number of poll groups to dynamically pick
the buf_cache_size for each transport poll group.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Id166098244287c56f12cdd88ba27a17fa34a4348
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17331
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Kernel `ublk_drv` driver will do memcpy to this data buffer in
unit of page size, for a simple 4KiB I/O, it may call memcpy
twice if the data buffer isn't page aligned. Moreover, SPDK
may also has double buffers with this case, so here, we use
page aligned data buffer at initialization.
Change-Id: Ica86a9702283327a2bb491e38990b8c00bc77f57
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17283
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
If CONFIG_ISAL and CONFIG_ISAL_CRYPTO are both defined, the build was
only including the LOCAL_SYS_LIBS for libisal_crypto. This fixes that
bug using the same technique used in other Makeifles.
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: I4c0869d60742cd6bdb0812d67db3abbfa7e69122
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17345
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Fixed issue indicated by Klocwork scan. 'desc->bdev'
is assigned to 'bdev' ptr, before verification that
'desc' is not NULL
Change-Id: I36e63c27b4d3220e85524133a0ec0e3521770875
Signed-off-by: Marcin Spiewak <marcin.spiewak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17350
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This refactors the code paths that call lvs_load() to allocate the
spdk_lvol_store structure before calling lvs_load(). Previously this
allocation was done in lvs_load_cb(). This is being done because a later
patch requires a pointer to the structure to be passed to lvs_load via
the spdk_bs_opts structure.
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: I2e942d1f7525fa5a16cd34b1b4b3a0a821e13006
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17220
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
The blob's parent_id and allocate_all examined and/or modified in a
two places bs_inflate_blob_open_cpl(). This transforms the two if
statements scattered around the function into a switch statement to make
it easier to understand how these two values are related.
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: I2cff2d07a0089b52678035b2ece60db6a5f67a8e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17178
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
There are multiple locations where a struct lvol_store is allocated.
This invites inconsistency in initialization, which will become more of
a problem as esnap clones have additional initialization.
Now all struct lvol_store allocations should be done with lvs_alloc().
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: I07a2f274475375072f80c25ed67cb1fb802cc4e1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16231
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
There are several places where new lvols are created and each reproduces
much of the same code. Esnap clones will add yet another in lvol.c and
more in unit tests. This introduces lvol_alloc() to minimize the chance
of unintended skew over time.
A side effect of this is that snapshots and clones now inherit clear
method from their parent. Previously they would fall back to the
default. The old behavior seems to be accidental, hence the change.
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: Ibf6f79c567e92354ea73e6589c736b1b946731a0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14976
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
The thin_provision member of struct spdk_lvol is set but never used.
When needed, an lvol's thin provision state is obtained by looking at
the lvol's blob. This removes the unused thin_provision member.
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: I5a2048b5334a26772a25a0bd238e42d3aeb63b49
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17173
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
When an esnap clone blob's external snapshot arrives after the blob is
opened, it can now be hot-added to the blob. Presumably the new device
replaces a place-holder device that did not really atteempt IO.
Change-Id: I622feb84efa66628debf44f7e7cb88b6a012db6d
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16232
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This adds the ability to abort IOs as esnap bs_dev channels are being
destroyed.
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: Ia63d4cbef5cd4c84dc8d5e2e9e407bacd961385f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16423
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
As the blobstore is being unlaoded, async esnap channel destructions may
be in flight. In such a case, spdk_bs_unload() needs to defer the unload
of the blobstore until channel destructions are complete.
The following commands lead to the illustrated states.
bdev_malloc_create -b malloc0
bdev_lvol_clone_bdev lvs1 malloc0 eclone
.---------. .--------.
| malloc0 |<--| eclone |
`---------' `--------'
bdev_lvol_snapshot lvs1/eclone snap
.---------. .------. .--------.
| malloc0 |<--| snap |<--| eclone |
`---------' `------' `--------'
bdev_lvol_clone lvs1/snap eclone
.--------.
,-| eclone |
.---------. .------.<-' `--------'
| malloc0 |<--| snap |
`---------' `------'<-. .-------.
`-| clone |
`-------'
As the blobstore is preparing to be unloaded spdk_blob_unload(snap) is
called once for eclone, once for clone, and once for snap. The last of
these calls happens just before spdk_bs_unload() is called.
spdk_blob_unload() needs to destroy channels on each thread. During this
thread iteration, spdk_bs_unload() starts. The work performed in the
iteration maintains a reference to the blob, and as such it
spdk_bs_unload() cannot do its work until the iteration is complete.
Change-Id: Id9b92ad73341fb3437441146110055c84ee6dc52
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14975
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This adds support for inflate and decouple for esnap clones. Since there
are no immediate consumers that will provide back_bs_dev->is_zeroes()
that can return true, a shortcut is taken in that inflate and decouple
of esnap clones are the same.
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: I4d2e6565126991acd650f073ce876466334e986d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11574
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
An esnap clone needs special handling as snapshots are created and
removed. In particular: the following must exist on the blob that
directly references the external snapshot and must be removed from
others:
- Ensure SPDK_BLOB_EXTERNAL_SNAPSHOT invalid flag exists only on the
esnap clone.
- Ensure BLOB_EXTERNAL_SNAPSHOT_ID internal xattr exists only on the
esnap clone.
- Clean up any esnap IO channels on a blob that is no longer an esnap
clone due to snapshot creation or removal.
See the diagrams and description in blob_esnap_clone_snapshot() in
blob_ut.c for details.
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: Ie4125d64d5bac9cfa7d6c7cc9a543d72a169f6ee
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11573
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
The channel passed to blob IO operations is useful for tracking
operations within the blobstore and the bs_dev that the blobstore
resides on. Esnap clone blobs perform reads from other bs_devs and
require per-thread, per-bs_dev channels.
This commit augments struct spdk_bs_channel with a tree containing
channels for the external snapshot bs_devs. The tree is indexed by blob
ID. These "esnap channels" are lazily created on the first read from an
external snapshot via each bs_channel. They are removed as bs_channels
are destroyed and blobs are closed.
Change-Id: I97aebe5a2f3584bfbf3a10ede8f3128448d30d6e
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14974
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
As per https://github.com/DPDK/dpdk/commit/71998eb61ff
Change-Id: Ie4e5a38976145e1037ef45593b4dc4265091482d
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17322
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Let us consider the following process:
1. one fabric connect request A comes but the subsystem is paused
due to adding/removing ns or other operations, so this request A
will be put into sgroup->queued until the subsystem becomes active;
2. the subsystem is paused for a long time until the connect timeout,
related qpair is destroyed, the sgroup->queued will not be cleaned
because qpair's ctrlr is NULL;
3. if a new request B comes, it is more likely to be allocated to the
same memory as the previous fabric command request. And it will be
put into sgroup->queued again, where has already exists the exactly
same pointer with request B.
This leads to the pointer hanging problem and it will cause infinitely
loop when traversing sgroup->queued!
So this patch avoids the ptr-hanging problem by checking and cleaning
all sgroups queued req whose qpair is the being destroyed qpair in
_nvmf_qpair_destroy when ctrlr is NULL.
This problem is already described in issue #2133.
Signed-off-by: Peng Lian<peng.lian@smartx.com>
Change-Id: I909d673b5050f21fa193914cc4ffe6634232fa7d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17147
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Add an API to easily determine if a blob is an esnap clone, similar to
what already exists for snapshot, clone, and thin_provisioned.
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: Ie07cd09b30513893e82f1c85e94a24a93c79d71e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16862
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
When a sequence is used to perform IO on an esnap clone, differenent
channels will be needed for the blobstore device and the esnap device.
No special esnap handling is required when a sequence is used to perform
IO directly on the blobstore device.
This commit splits bs_sequence_start() into bs_sequence_start_bs() and
bs_sequence_start_blob() to handle these two scenarios. A later commit
introduces special handling of ensap clone blobs.
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: I3a6f46640cdb7fdc380bf557736638f1b39f05e3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17172
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
