When I/O error resiliency is supported, most DNR parameters for internal
APIs were cleared. However, for some cases, especially for the reset I/O
command, the upper layer wants the NVMe driver to return I/O errors
immediately without retry even if the upper layer enables I/O error retry.
To satisfy such requirement, add an abort_dnr variable to the spdk_nvme_qpair
structure and internal abort APIs use the abort_dnr variable. A public API
spdk_nvme_qpair_set_abort_dnr() can change the abort_dnr variable dynamically.
The public spdk_nvme_transport_ops structure is not changed to avoid
premature changes.
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I486a1b3ad8411f9fa261a2bf3a45aea9da292e9c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17099
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Previously, if a reconnect timer was registered when a reset request
came, the reset request failed with -EBUSY. However, this means the
reset request was queued for a long time until the reconnect timer was
expired.
When a reconnect timer is registered, reset is not actually in progress.
Hence, a new reset request can cancel the reconnect timer and can start
reconnection safely.
Add a unit test case to verify this change.
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: Ied8dd0ad822d2fd6829d88cd56cb36bd4fad13f9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16823
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
The llvm_precompile function checks for the CLANG version available on the machine
using bash regex and searches for fuzzer libraries in a path based on the full CLANG
version number. (e.g. /usr/lib64/clang/15.0.3/...)
However, on the newest Fedora distribution, the path has changed and fuzzer libraries
couldn't be found. Currently, CLANG libraries path contains only major version number
(/usr/lib64/clang/16)
To address this issue, the function has been updated to search only for the major
CLANG version number instead of the full version number. Instead of using clang_version,
the function now uses clang_num because in every Fedora distribution there is directory
or symlink that points to the right CLANG version.
e.g. symlinks
/usr/lib64/clang/13 -> /usr/lib64/clang/13.0.1
/usr/lib64/clang/15 -> /usr/lib64/clang/15.0.3
or directory:
/usr/lib64/clang/16
Fixes#3000
Signed-off-by: Kamil Godzwon <kamilx.godzwon@intel.com>
Change-Id: Iaf0dedc2bb3956cf06796e2eb60a5fa6f492b780
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17907
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
on some hosts, it might take 1 or 2 seconds for the
mapper device to appear on /dev
in this case, the test will fail
because we check if the device exists immediately.
by giving it chance to get up the test will pass.
Signed-off-by: Amir Haroush <amir.haroush@huawei.com>
Signed-off-by: Shai Fultheim <shai.fultheim@huawei.com>
Change-Id: I996d84861333d29d5c9370a2c5a471e7962d91b1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17912
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This is mostly a wrapper around spdk_blob_is_degraded(), but it also
performs a NULL check on lvol->blob. Since an lvol without a blob cannot
perform IO, this condition returns true.
The two callers of spdk_blob_is_degraded() in vbdev_lvol.c have been
updated to use spdk_lvol_is_degraded().
Change-Id: I11dc682a26d971c8854aeab280c8199fced358c3
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17896
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
As an lvolstore is being destroyed, _vbdev_lvs_remove() starts an
interation through the lvols to delete each one, ultimately leading to
the destruction of the lvolstore with a call to lvs_free(). The callback
passed to vbdev_lvs_destruct() is always called asynchronously via
spdk_io_device_unregister() in bs_free().
When the lvolstore resides on bdevs that perform async IO (i.e. most
bdevs other than malloc), this gives a small window when the lvol bdev
is not registered but a lookup with spdk_lvol_get_by_uuid() or
spdk_lvol_get_by_names() will succeed. If rpc_bdev_lvol_delete() runs
during this window, it can get a reference to an lvol that has just been
unregistered and lvol->blob may be NULL. This lvol is then passed to
vbdev_lvol_destroy().
Before this fix, vbdev_lvol_destroy() would call:
spdk_blob_is_degraded(lvol->blob);
Which would then lead to a NULL pointer dereference, as
spdk_blob_is_degraded() assumes a valid blob is passed. While a NULL
check would avoid this particular problem, a NULL blob is not
necessarily caused by the condition described above. It would better to
flag the lvstore's destruction before returning from
vbdev_lvs_destruct() and use that flag to prevent operations on the
lvolstore that is being deleted. Such a flag already exists in the form
of 'lvs_bdev->req != NULL', but that is set too late to close this race.
This fix introduces lvs_bdev->removal_in_progress which is set prior to
returning from vbdev_lvs_unload() and vbdev_lvs_destruct(). It is
checked by vbdev_lvol_destroy() before trying to destroy the lvol. Now,
any lvol destruction initiated by something other than
vbdev_lvs_destruct() while an lvolstore unload or destroy is in progress
will fail with -ENODEV.
Fixes issue: #2998
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: I4d861879097703b0d8e3180e6de7ad6898f340fd
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17891
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This automatically cleans up aio files left over from earlier aborted
runs. This helps streamline development of new tests and should have no
impact on CI.
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: Id65f60cdfc9969fda1dcdd17e60643ad87f45de7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17898
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This is a followup to commit f4dc558245
which strove to log blob IDs as hex to make small blob IDs more
recognizable. That commit missed a few cases where the blob ID is logged
as decimal.
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: I75d1b5973ee7e812f7caf0e826d3edbcba126743
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17641
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
we don't have dependency files for OCF sources/headers.
for example, if someone 'touch metadata_collision.h'
it will not compile anything.
with this fix, it will compile all the relevant files.
Signed-off-by: Amir Haroush <amir.haroush@huawei.com>
Signed-off-by: Shai Fultheim <shai.fultheim@huawei.com>
Change-Id: I35b1c1f80a60f4be59cdca95f68bbafc7a212774
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17914
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
the default indent is 3 so we must set it to 2
as our md files are all indented with 2.
Signed-off-by: Amir Haroush <amir.haroush@huawei.com>
Signed-off-by: Shai Fultheim <shai.fultheim@huawei.com>
Change-Id: I76c501311b6a4443dc6fc655894487b762d67abb
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17913
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
LLVMFuzzerRunDriver does not allow to specify minimum input length,
return immediately when data insufficient.
Signed-off-by: Sebastian Brzezinka <sebastian.brzezinka@intel.com>
Change-Id: I306e1774b17b04108f2454b2fdaadb4d912bd274
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17884
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Write_zeroes command fallback had used its own split logic but multiple
writes had been serialized.
Use the unified split logic also for the write_zeroes command fallback.
This not only improves the performance but also simplifies the code.
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I955870947ae036482871453b4870f06f6f7f947b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17902
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
As same as copy command, calculation of max write_zeroes size for
fallback case includes division and is costly. The result is constant
for each bdev. Hence, we can calculate it only once and store it into
bdev->max_write_zeroes at bdev registration. However, in unit tests,
bdev->blocklen and bdev->md_len can be changed dynamically. Hence,
adjust bdev->max_write_zeroes for such changes.
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I16e4980e7a283caa6c995a7dc61f7e77585d464e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17911
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
ZERO_BUFFER_SIZE is in bytes but it is easier to calculate max
write_zeroes in blocks first and then get the minimum between max
write_zeroes in blocks and remaining_num_blocks rather than converting
remaining_num_blocks to num_bytes. This is helpful to store the result
into bdev->max_write_zeroes for fallback case.
We have one small fix in this patch. As we recently fixed
bdev_io_get_max_buf_len(), to get aligned length,
spdk_bdev_get_buf_align() - 1 is correct.
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I104bc837c9eee1303664bfdb3559b0e840d6f0e5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17910
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
The generic bdev layer has a fallback meachanism for the copy command
used when the backend bdev module does not support it. However, its max
size is limited. To remove the limitation, the fallback supports split by
using the unified split logic rather than following the write zeroes
command.
bdev_copy_should_split() and bdev_copy_split() use spdk_bdev_get_max_copy()
rather then referring bdev->max_copy to include the fallback case.
Then, spdk_bdev_copy_blocks() does the following.
If the copy size is large and should be split, use the generic split
logic regardless of whether copy is supported or not.
If copy is supported, send the copy request, or if copy is not
supported, emulate it using regulard read and write requests.
Add unit test case to verify this addition.
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: Iaf51db56bb4b95f99a0ea7a0237d8fa8ae039a54
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17073
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
As name suffix, _done has been used more often than _complete for
fallback function names. 100 chars per line is suggested implicitly.
Do these small clean up in this patch.
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: Id14dd3f09be8fd49b947b7a8f8b87108fb56c346
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17900
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Calculation of max copy size for fallback case includes division and is
costly. The result is constant for each bdev. Hence we can calculate it
only once and store it into bdev->max_copy at bdev registration.
Calculation of max copy size for fallback case is almost same as
calculation of max write zero size for fallback case. To reuse the
calculation, the helper function is named as bdev_get_max_write() and
has a num_bytes parameter.
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: Iac83a1f16b908d8b36b51d9c51782de40313b6c8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17909
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
The following patches will change spdk_bdev_register() to access iobuf
and bdev's blocklen and blockcnt.
Hence, we have to configure these correctly for alltest cases.
Move ut_init/fini_bdev() up in a file. Add missing ut_init/fini_bdev()
and allocate/free_bdev() calls for some test cases. Add blockcnt and
blocklen to allocate_vbdev().
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: Iccbb1cfe4dcdc4496f15304b5362d76d5296607f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17908
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
As we recently fixed bdev_io_get_max_buf_len(), to get aligned length,
spdk_bdev_get_buf_align() - 1 is correct.
_bdev_get_block_size_with_md() considers both interleaved metadata and
separate metadata cases. It is simpler to use
_bdev_get_block_size_with_md().
The copy command fallback uses write command. As the write zeroes
fallback does, bdev->write_unit_size should be considered.
Fix all in this patch.
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I88fe1b250289f2bab7b541523e8be931eeb8150c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17899
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
If spdk_spin_lock() is called on an uninitialized spinlock, it will
deadlock. This commit detects whether a lock is initialized and aborts
instead of deadlocking.
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: Ie7497633091edd4127c06ca0530e9a1dff530d1b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16002
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Debug builds have information about when each spinlock was initialized,
last locked and last unlocked. This commit logs that information when
a spinlock operation aborts.
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: I11232f4000f04d222dcaaed44c46303b7ea6cf6b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16001
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
In debug builds, SPDK spinlocks will have stack traces that track where
they were allocated, last locked, and last unlocked. This adds gdb
pretty printers to make that information easily visible. See the updates
in doc/gdb_macros.md for details.
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: I4f903c588d9384c4005eec01348fa5c2d3cab5db
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16000
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
To help debug spinlocks, capture stack traces as spinlocks are used.
Future commits in this series will make debugging with these stack
traces easier.
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: I597b730ca771ea3c5b831f5ba4058d359215f7f6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15998
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
This reverts commit 32908cbfc8.
OCF deprecation notice has removed as
Huawei is picking up support for the OCF project.
Signed-off-by: Amir Haroush <amir.haroush@huawei.com>
Signed-off-by: Shai Fultheim <shai.fultheim@huawei.com>
Change-Id: I007e80bc74dc50cfa9b8cde97fc6fdc9608d7ebd
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17894
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This reverts commit c5224a96ae.
OCF deprecation notice has removed as
Huawei is picking up support for the OCF project.
Signed-off-by: Amir Haroush <amir.haroush@huawei.com>
Signed-off-by: Shai Fultheim <shai.fultheim@huawei.com>
Change-Id: I80ebfe75eaa1a9b96249ed578fcaff6e9576928f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17893
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
In releases of SPDK prior to v23.01, GPT bdevs had a random UUID. This
ended with commit a1c7ae2d3f, which is OK
because a non-persistent UUID is not all that useful.
Per Table 5.6 in Section 5.3.3 of UEFI Spec 2.3, each partition has a
16-byte UniquePartitionGUID:
GUID that is unique for every partition entry. Every partition ever
created will have a unique GUID. This GUID must be assigned when the
GPT Partition Entry is created. The GPT Partition Entry is created
whenever the NumberOfPartitionEntries in the GPT Header is increased
to include a larger range of addresses.
With this change, GPT bdevs use this unique partition GUID as the bdev's
UUID.
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: Id8e8aa9e7903d31f199e8cfdb487e45ce1524d7b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17351
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
This introduces spdk_bdev_part_construct_ext(), which takes an options
structure as an optional parameter. The options structure has one
option: uuid.
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: I5e9fdc8e88b78b303e60a0e721d7a74854ac37a9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17835
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
rc might be not initialized and it was not correct to
use it in this place.
Fixes 6b7cca1542 accel/dpdk_cryptodev: Handle OP_STATUS_SUCCESS
Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: Ifd2b3032afd6830bd851adb61f68ae4fa9621d33
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17656
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Accel buffer is released when encrypt operation
completes, however it doesn't mean that base
bdev finishes writing encrypted data. As result,
accel buffer might be reused in another IO, that
leads to data corruption.
Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: I1acf7c30da2f92989ecc44e96b00f7609058ec5a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17655
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
This is done in preparation for retrying IOs on ENOMEM when pushing
bounce data. Also, rename md_buffer to md_buf to keep the naming
consistent with other code which uses this abbreviation.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I014f178a45a2a751ecca40d119f45bf323f37d0c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17762
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
The IOs will now be retried after ENOMEM is received when doing memory
domain pull or appending an accel copy. The retries are performed using
the mechanism that's already in place for IOs completed with
SPDK_BDEV_IO_STATUS_NOMEM.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I284643bf9971338094e14617974f7511f745f24e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17761
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
The IOs with an outstanding memory domain push/pull or accel sequence
finish operation are now added to the io_outstanding counter. It'll be
necessary to correctly calculate nomem_threshold when handling ENOMEM
from those operations.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ice1fb94f1c9054a3a96312a0960ac5085d0b21bc
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17760
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
The leading underscore usually indicate that a function providing the
actual implementation for something that's called from some other
wrapper function without the leading underscore. That is not the case
for these functions, so this patch removes the leading underscores.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I6e1186b156116249ee53a3845ae99ba87db5122b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17868
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
In the next patches we'll need to increment the io_outstanding from a
few more places, so it'll be good to have a dedicated function for that.
Also, move _bdev_io_decrement_outstanding() up, so that both functions
are near each other.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I1af5dbe288f7f701c8ba5e85406f02330ae21a39
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17759
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
There are some common operations that need to be done each time a
sequence is executed (and more will be added in the following patches),
so it makes sense to have a common callback. data_transfer_cpl is used
for executing user's callbacks since it's unused at this point.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I4570acbdbe158512d13c31c0ee0c7bb7bf62d18c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17678
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
The IOs are now kept on the io_memory_domain queue only if they have an
outstanding pull/push operation. It'll make it easier to support
retrying pull/push in case of ENOMEM.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: If5a54fac532206ee8472bacf364a5ef6cde8edea
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17677
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This is a preparation for reusing the code handling nomem_io for
other type of NOMEM errors (e.g. from pull/push/append_copy). This
patch doesn't actually change anything functionally - only IOs completed
by a module with SPDK_BDEV_IO_STATUS_NOMEM status are retried.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I12ecb2efcf2d2cdf75b302f9f766b4c16ac99f3e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17676
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
The actual memory domain push already only happened for successfully
completed requests, but the code would go still go through
_bdev_io_push_bounce_data_buffer(), which could cause issues for IOs
completed with NOMEM, because the bounce buffer would be released in
_bdev_io_complete_push_bounce_done().
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Id1af1e31cb416e91bf11101a5ce7919530245e1e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17866
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
The sequence is associated with parent IO, so that's the IO that should
be used when executing a sequence.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ifcdb06094b38a5eaee1691e5aa8de1c8dc9d01a6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17865
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
After we create the GPT, we change the partition type
GUID to the associated SPDK value. The current
comment just says "change the GUID" which is
ambiguous because there are multiple GUIDs associated
with each partition.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Id821c5c5bbd7a72d84d5ddf4d91d633307f2235b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17855
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This function disconnects any qpairs that match both
the listen trid and the subsystem pointer. If the
specified subsystem is NULL, it will just disconnect
all qpairs matching the listen trid.
But there are cases where a qpair doesn't yet have an
associated subsystem - for example, before a CONNECT
is received.
Currently we would always disconnect such a qpair, even
if a subsystem pointer is passed. Presumably this check
was added to ensure we don't dereference qpair->ctrlr
when it is NULL but it was added incorrectly.
Also while here, move and improve the comment about
skipping qpairs.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I8b7988b22799de2a069be692f4a5b4da59c2bad4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17854
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Otherwise, it's impossible to stop the app before its run time expires,
because the accel library waits until its IO channels are released which
would only happen at the end.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I7438b474f4f6d6bcb4bf6aad02ccae9f511f1b51
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17768
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
While userspace probes have a high overhead when enabled due
to the trap, it is still cleaner and slightly more efficient
to not have all of the SPDK_DTRACE_PROBE macros implicitly
capture the tsc counter as an argument.
So rename the existing SPDK_DTRACE_PROBE macros to
SPDK_DTRACE_PROBE_TICKS, and create new SPDK_DTRACE_PROBE
macros without the implicit ticks argument.
Note this does cause slight breakage if there is any
out-of-tree code that using SPDK_DTRACE_PROBE previously,
and programs written against those probes would need to
adjust their arguments. But the likelihood of such code
existing is practically nil, so I'm just renaming the
macros to their ideal state.
All of the nvmf SPDK_DTRACE_PROBE calls are changed to
use the new _TICKS variants. The event one is left
without _TICKS - we have no in-tree scripts that use
the tsc for that event.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Icb965b7b8f13c23d671263326029acb88c82d9df
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17669
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Mike Gerdts <mgerdts@nvidia.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
