Commit Graph

7097 Commits

Author SHA1 Message Date
Artur Paszkiewicz
84f8594c80 test/bdev_raid: add raid5f tests
Change-Id: Iebecbe09c8808c553ccaec2973852d1c9dd115bf
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
2023-05-09 17:58:11 +08:00
Artur Paszkiewicz
43a6a14e1b test/bdev_raid: test more than 2 base bdevs in raid
Change-Id: I3b105a33dc89fd654a21de548f5aba7fa29c6ee7
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
2023-05-09 17:58:11 +08:00
Artur Paszkiewicz
cb56a42775 test/bdev: add raid1 tests
Change-Id: Ib7c8f8818be9da24e829a4d4470671cd39234543
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
2023-05-09 17:58:11 +08:00
Artur Paszkiewicz
3728239ab5 test/bdev_raid: simplify test cleanup
There is a trap set up to kill the spdk process in case of error, there
is no need to delete any bdevs before that.

Change-Id: Ic80e2a48453f718fbc42cabe88d86eefa35c95db
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
2023-05-09 17:58:11 +08:00
Artur Paszkiewicz
882ecb55a8 util/uuid: add API to test/set null uuid
Refactor the code to use these new functions.

Change-Id: I21ee7e9a96f30fbd60106add5e8b071e86bf93c9
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
2023-05-09 17:58:11 +08:00
Krzysztof Smolinski
52726c9b6e test/raid1: unit tests for read balancing
Signed-off-by: Krzysztof Smolinski <krzysztof.smolinski@intel.com>
Change-Id: Ic42bfe8be2d0d76e804a82b4837391384f01f8f8
2023-05-09 17:58:11 +08:00
Krzysztof Smolinski
4d783e7255 raid1: read balancing
Reads for raid1 bdevs are balanced. Algorithm tries to evenly distribute
load by sending read I/O to all base bdevs following round-robin, but
skiping base bdev that processed most data so far.

Signed-off-by: Krzysztof Smolinski <krzysztof.smolinski@intel.com>
Change-Id: I7d85411a6421bd7352031efb562ee95f2c612011
2023-05-09 17:58:11 +08:00
Artur Paszkiewicz
951d766289 raid5f: degraded io support
Change-Id: I1af2ffc3fe1f41b798e15b5194ab5695923737ef
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
2023-05-09 17:58:11 +08:00
Artur Paszkiewicz
d0c7517b84 ut/raid5f: assume raid_io_info is always one stripe
Support for multi-stripe requests is not used anymore so remove it.

Change-Id: I8f28817763452674c8a183c640800f3a4b4b3653
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
2023-05-09 17:58:11 +08:00
Artur Paszkiewicz
284fd419cb ut/raid5f: simplify read tests
Remove struct test_request_conf and instead do some basic reads on each
chunk. Also remove the io_info splitting because it is not used now.

Change-Id: I4b945b40598670f6ab84fb8066278877fee7fb75
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
2023-05-09 17:58:11 +08:00
Artur Paszkiewicz
f71c255827 module/raid: add raid_bdev ptr to raid_base_bdev_info
This allows to simplify some code where raid_bdev and base_info are
needed.

Change-Id: I40395204fdcdd0487bdecec1cd47efb347f1310a
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
2023-05-09 17:58:11 +08:00
Artur Paszkiewicz
79eccac059 module/raid: suspend/resume IO
Add functions to suspend and resume IO on all channels. This will be
used to safely change the device state in case of e.g. removing a base
bdev.

Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Change-Id: I203c1899bde15101e0c2bc8da7a1066a2fee6dd2
2023-05-09 17:58:11 +08:00
Krzysztof Smolinski
8c591e2d4f module/raid: data offset and data size implementation
When raid bdev is created with superblock parameter then all data on
this bdev should be shifted by some offset. Such space at the beginning
of bdev will be used to store on-disk raid metadata.

Signed-off-by: Krzysztof Smolinski <krzysztof.smolinski@intel.com>
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Change-Id: I2545a2b00a651ef5332ca1757da0110a63914a43
2023-05-09 17:58:11 +08:00
Krzysztof Smolinski
19c79b0d01 rpc/bdev_raid_create: added superblock parameter
Introduction of superblock parameter for bdev_raid_create rpc. This
parameter determines whether raid bdev should be created with support
for on-disk metadata (support for raid on-disk metadata is going to be
implemented in the future).

Signed-off-by: Krzysztof Smolinski <krzysztof.smolinski@intel.com>
Change-Id: Ie8c64f837dd7eb3ba788b7c5d7bc98e8f1368ba7
2023-05-09 17:58:11 +08:00
Artur Paszkiewicz
ba752093db raid5f: convert to use accel framework for xor
Change-Id: Id8fb521549342564bcf4288d74337fb4dd41fa03
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
2023-05-09 17:58:11 +08:00
Konrad Sztyber
e5dbea5967 test/bdev: extend chaining test with bdev layer ENOMEM case
The test already checked ENOMEM handling, but it only used bdevs that
support chaining (crypto, malloc), so bdev layer didn't need to execute
any accel operations.  So, to force bdev layer to do that, a passthru
bdev was added, as it doesn't support chaining.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I322a65ccebb0f144c759692fff285cfd44bbab4b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17766
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2023-05-09 17:58:11 +08:00
Konrad Sztyber
32683a847a bdev: retry IOs on ENOMEM when pushing bounce data/md
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ia7634b570eb7d04c22003337a46630d152171157
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17764
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2023-05-09 17:58:11 +08:00
Jim Harris
c387766501 nvmf: use iterator APIs in nvmf_tgt_create_poll_group()
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I4d9a5dd4655edb8315503e7551aec1926d1cc017
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17959
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
2023-05-09 17:58:11 +08:00
Jim Harris
6c760ec607 nvme: remove deprecated spdk_nvme_ctrlr_prepare_for_reset()
Note that the prepare_for_reset flag in spdk_nvme_ctrlr is
still needed - it's just set now in the nvme_ctrlr_disconnect
path instead of this deprecated and now removed API.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I0a6aa1c72767eb67a84b8928a986e06cbac88240
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17936
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2023-05-09 17:58:11 +08:00
Shuhei Matsumoto
610265c9fa bdev/nvme: Reset I/O disables retry when destroying I/O qpairs
As the RBD bdev module does, the upper layer wants the reset command
to abort or complete all I/Os submitted before the reset command.

To satisfy this requirement, return all aborted I/Os by deleting I/O
qpairs to the upper layer without retry. To return all aborted I/Os
by deleting I/O qpairs, enable DNR for I/O qpairs. These I/O qpairs
are deleted and recreated. Hence, we do not have to disable DNR.

No more I/O comes at a reset I/O because the generic bdev layer already
blocks I/O submission. However, some I/Os may be queued for retry even
after deleting I/O qpairs. Hence, abort all queued I/Os for the bdev
before completing the reset I/O.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I9830026ef5f2b9c28aee92e6ce4018ed8541c808
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16836
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2023-05-09 17:58:11 +08:00
Shuhei Matsumoto
8bc0582f6e nvme: The upper layer controls DNR dynamically for I/O aborts
When I/O error resiliency is supported, most DNR parameters for internal
APIs were cleared. However, for some cases, especially for the reset I/O
command, the upper layer wants the NVMe driver to return I/O errors
immediately without retry even if the upper layer enables I/O error retry.

To satisfy such requirement, add an abort_dnr variable to the spdk_nvme_qpair
structure and internal abort APIs use the abort_dnr variable. A public API
spdk_nvme_qpair_set_abort_dnr() can change the abort_dnr variable dynamically.

The public spdk_nvme_transport_ops structure is not changed to avoid
premature changes.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I486a1b3ad8411f9fa261a2bf3a45aea9da292e9c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17099
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2023-05-09 17:58:11 +08:00
Shuhei Matsumoto
7ea8a5aae5 bdev/nvme: Reset I/O cancels reconnect timer and starts reconnection
Previously, if a reconnect timer was registered when a reset request
came, the reset request failed with -EBUSY. However, this means the
reset request was queued for a long time until the reconnect timer was
expired.

When a reconnect timer is registered, reset is not actually in progress.
Hence, a new reset request can cancel the reconnect timer and can start
reconnection safely.

Add a unit test case to verify this change.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: Ied8dd0ad822d2fd6829d88cd56cb36bd4fad13f9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16823
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2023-05-09 17:58:11 +08:00
Kamil Godzwon
d117053679 test/autobuild: Update llvm_precompile function to handle newer CLANG versions
The llvm_precompile function checks for the CLANG version available on the machine
using bash regex and searches for fuzzer libraries in a path based on the full CLANG
version number. (e.g. /usr/lib64/clang/15.0.3/...)

However, on the newest Fedora distribution, the path has changed and fuzzer libraries
couldn't be found. Currently, CLANG libraries path contains only major version number
(/usr/lib64/clang/16)

To address this issue, the function has been updated to search only for the major
CLANG version number instead of the full version number. Instead of using clang_version,
the function now uses clang_num because in every Fedora distribution there is directory
or symlink that points to the right CLANG version.

e.g. symlinks
/usr/lib64/clang/13 -> /usr/lib64/clang/13.0.1
/usr/lib64/clang/15 -> /usr/lib64/clang/15.0.3

or directory:
/usr/lib64/clang/16

Fixes #3000

Signed-off-by: Kamil Godzwon <kamilx.godzwon@intel.com>
Change-Id: Iaf0dedc2bb3956cf06796e2eb60a5fa6f492b780
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17907
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2023-05-09 17:58:11 +08:00
Amir Haroush
6be9d04cb5 test/setup: Fix dm_mount test for slow hosts
on some hosts, it might take 1 or 2 seconds for the
mapper device to appear on /dev
in this case, the test will fail
because we check if the device exists immediately.
by giving it chance to get up the test will pass.

Signed-off-by: Amir Haroush <amir.haroush@huawei.com>
Signed-off-by: Shai Fultheim <shai.fultheim@huawei.com>
Change-Id: I996d84861333d29d5c9370a2c5a471e7962d91b1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17912
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2023-05-09 17:58:11 +08:00
Mike Gerdts
c9b802ca49 lvol: add spdk_lvol_is_degraded
This is mostly a wrapper around spdk_blob_is_degraded(), but it also
performs a NULL check on lvol->blob. Since an lvol without a blob cannot
perform IO, this condition returns true.

The two callers of spdk_blob_is_degraded() in vbdev_lvol.c have been
updated to use spdk_lvol_is_degraded().

Change-Id: I11dc682a26d971c8854aeab280c8199fced358c3
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17896
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2023-05-09 17:58:11 +08:00
Mike Gerdts
b240c2b103 lvol: lvol destruction race leads to null deref
As an lvolstore is being destroyed, _vbdev_lvs_remove() starts an
interation through the lvols to delete each one, ultimately leading to
the destruction of the lvolstore with a call to lvs_free(). The callback
passed to vbdev_lvs_destruct() is always called asynchronously via
spdk_io_device_unregister() in bs_free().

When the lvolstore resides on bdevs that perform async IO (i.e. most
bdevs other than malloc), this gives a small window when the lvol bdev
is not registered but a lookup with spdk_lvol_get_by_uuid() or
spdk_lvol_get_by_names() will succeed. If rpc_bdev_lvol_delete() runs
during this window, it can get a reference to an lvol that has just been
unregistered and lvol->blob may be NULL. This lvol is then passed to
vbdev_lvol_destroy().

Before this fix, vbdev_lvol_destroy() would call:

   spdk_blob_is_degraded(lvol->blob);

Which would then lead to a NULL pointer dereference, as
spdk_blob_is_degraded() assumes a valid blob is passed. While a NULL
check would avoid this particular problem, a NULL blob is not
necessarily caused by the condition described above. It would better to
flag the lvstore's destruction before returning from
vbdev_lvs_destruct() and use that flag to prevent operations on the
lvolstore that is being deleted. Such a flag already exists in the form
of 'lvs_bdev->req != NULL', but that is set too late to close this race.

This fix introduces lvs_bdev->removal_in_progress which is set prior to
returning from vbdev_lvs_unload() and vbdev_lvs_destruct(). It is
checked by vbdev_lvol_destroy() before trying to destroy the lvol.  Now,
any lvol destruction initiated by something other than
vbdev_lvs_destruct() while an lvolstore unload or destroy is in progress
will fail with -ENODEV.

Fixes issue: #2998

Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: I4d861879097703b0d8e3180e6de7ad6898f340fd
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17891
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2023-05-09 17:58:11 +08:00
Mike Gerdts
ee5ea993a0 test/lvol: unlink aio files at start of test
This automatically cleans up aio files left over from earlier aborted
runs. This helps streamline development of new tests and should have no
impact on CI.

Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: Id65f60cdfc9969fda1dcdd17e60643ad87f45de7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17898
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2023-05-09 17:58:11 +08:00
Sebastian Brzezinka
2b954ddae4 app/fuzz: discard randoms of insufficient length
LLVMFuzzerRunDriver does not allow to specify minimum input length,
return immediately when data insufficient.

Signed-off-by: Sebastian Brzezinka <sebastian.brzezinka@intel.com>
Change-Id: I306e1774b17b04108f2454b2fdaadb4d912bd274
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17884
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2023-05-09 17:58:11 +08:00
Shuhei Matsumoto
9454d16810 bdev: Calculate max_write_zeroes once at bdev registration for fallback case
As same as copy command, calculation of max write_zeroes size for
fallback case includes division and is costly. The result is constant
for each bdev. Hence, we can calculate it only once and store it into
bdev->max_write_zeroes at bdev registration. However, in unit tests,
bdev->blocklen and bdev->md_len can be changed dynamically. Hence,
adjust bdev->max_write_zeroes for such changes.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I16e4980e7a283caa6c995a7dc61f7e77585d464e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17911
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2023-05-09 17:58:11 +08:00
Shuhei Matsumoto
95975beb1c bdev: Copy command fallback supports split to make copy size unlimited
The generic bdev layer has a fallback meachanism for the copy command
used when the backend bdev module does not support it. However, its max
size is limited. To remove the limitation, the fallback supports split by
using the unified split logic rather than following the write zeroes
command.

bdev_copy_should_split() and bdev_copy_split() use spdk_bdev_get_max_copy()
rather then referring bdev->max_copy to include the fallback case.

Then, spdk_bdev_copy_blocks() does the following.

If the copy size is large and should be split, use the generic split
logic regardless of whether copy is supported or not.
If copy is supported, send the copy request, or if copy is not
supported, emulate it using regulard read and write requests.

Add unit test case to verify this addition.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: Iaf51db56bb4b95f99a0ea7a0237d8fa8ae039a54
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17073
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2023-05-09 17:58:11 +08:00
Shuhei Matsumoto
fa2c6446fa bdev: Calculate max_copy once at bdev registration for fallback case
Calculation of max copy size for fallback case includes division and is
costly. The result is constant for each bdev. Hence we can calculate it
only once and store it into bdev->max_copy at bdev registration.
Calculation of max copy size for fallback case is almost same as
calculation of max write zero size for fallback case. To reuse the
calculation, the helper function is named as bdev_get_max_write() and
has a num_bytes parameter.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: Iac83a1f16b908d8b36b51d9c51782de40313b6c8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17909
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2023-05-09 17:58:11 +08:00
Shuhei Matsumoto
05580566e8 ut/bdev: Configure bdev size and iobuf for all test cases
The following patches will change spdk_bdev_register() to access iobuf
and bdev's blocklen and blockcnt.
Hence, we have to configure these correctly for alltest cases.

Move ut_init/fini_bdev() up in a file. Add missing ut_init/fini_bdev()
and allocate/free_bdev() calls for some test cases. Add blockcnt and
blocklen to allocate_vbdev().

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: Iccbb1cfe4dcdc4496f15304b5362d76d5296607f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17908
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2023-05-09 17:58:11 +08:00
Mike Gerdts
57fd506c57 bdev_gpt: use unique partition GUID as bdev UUID
In releases of SPDK prior to v23.01, GPT bdevs had a random UUID. This
ended with commit a1c7ae2d3f, which is OK
because a non-persistent UUID is not all that useful.

Per Table 5.6 in Section 5.3.3 of UEFI Spec 2.3, each partition has a
16-byte UniquePartitionGUID:

  GUID that is unique for every partition entry. Every partition ever
  created will have a unique GUID. This GUID must be assigned when the
  GPT Partition Entry is created.  The GPT Partition Entry is created
  whenever the NumberOfPartitionEntries in the GPT Header is increased
  to include a larger range of addresses.

With this change, GPT bdevs use this unique partition GUID as the bdev's
UUID.

Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: Id8e8aa9e7903d31f199e8cfdb487e45ce1524d7b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17351
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
2023-05-09 17:58:11 +08:00
Mike Gerdts
9b687d7753 bdev_part: allow UUID to be specified
This introduces spdk_bdev_part_construct_ext(), which takes an options
structure as an optional parameter. The options structure has one
option: uuid.

Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: I5e9fdc8e88b78b303e60a0e721d7a74854ac37a9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17835
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2023-05-09 17:58:11 +08:00
Konrad Sztyber
d89b59610d bdev: retry IOs on ENOMEM from pull/append_copy
The IOs will now be retried after ENOMEM is received when doing memory
domain pull or appending an accel copy.  The retries are performed using
the mechanism that's already in place for IOs completed with
SPDK_BDEV_IO_STATUS_NOMEM.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I284643bf9971338094e14617974f7511f745f24e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17761
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2023-05-09 17:58:11 +08:00
Jim Harris
02799aa571 test: clarify GPT related comment in blockdev.sh
After we create the GPT, we change the partition type
GUID to the associated SPDK value.  The current
comment just says "change the GUID" which is
ambiguous because there are multiple GUIDs associated
with each partition.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Id821c5c5bbd7a72d84d5ddf4d91d633307f2235b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17855
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2023-05-09 17:58:11 +08:00
Mike Gerdts
57b47f209f lvol: esnap clones must end on cluster boundary
When regular lvols are created, their size is rounded up to the next
cluster boundary. This is not acceptable for esnap clones as this means
that the clone may be silently grown larger than external snapshot. This
can cause a variety of problems for the consumer of an esnap clone lvol.

While the better long-term solution is to allow lvol sizes to fall on
any block boundary, the implementation of that needs to be suprisingly
complex to support creation and deletion of snapshots and clones of
esnap clones, inflation, and backward compatibility.

For now, it is best to put in a restriction on the esnap clone size
during creation so as to not hit problems long after creation. Since
lvols are generally expected to be large relative to the cluster size,
it is somewhat unlikely that this restriction will be a significant
limitation.

Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: Id7a628f852a40c8ec2b7146504183943d723deba
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17607
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2023-05-09 17:58:11 +08:00
Mike Gerdts
dcd012e8d0 vbdev_lvol: esnap memdomain support
Return the total number of memory domains supported by the blobstore and
any external snapshot bdev.

Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: I2f8afba6b31e689b8f942e2cf36906a0a30f38c8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16430
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2023-05-09 17:58:11 +08:00
Michal Berger
cf69d9af82 test/spdkcli: Wait until spdkcli_clear_config settled
Similarly to https://review.spdk.io/gerrit/c/spdk/spdk/+/17559, this
part may also be affected by:

Error in cmd: load_config /root/spdk/test/spdkcli/config.json (
load_config /root/spdk/test/spdkcli/config.json
request:
{
  "action_on_timeout": "none",
  "timeout_us": 0,
  "timeout_admin_us": 0,
  "keep_alive_timeout_ms": 10000,
  "transport_retry_count": 4,
  "arbitration_burst": 0,
  "low_priority_weight": 0,
  "medium_priority_weight": 0,
  "high_priority_weight": 0,
  "nvme_adminq_poll_period_us": 10000,
  "nvme_ioq_poll_period_us": 0,
  "io_queue_requests": 1024,
  "delay_cmd_submit": true,
  "bdev_retry_count": 3,
  "transport_ack_timeout": 0,
  "ctrlr_loss_timeout_sec": 0,
  "reconnect_delay_sec": 0,
  "fast_io_fail_timeout_sec": 0,
  "generate_uuids": false,
  "transport_tos": 0,
  "io_path_stat": false,
  "method": "bdev_nvme_set_options",
  "req_id": 31
}
Got JSON-RPC error response
response:
{
  "code": -1,
  "message": "RPC not permitted with nvme controllers already attached"
}

so make sure the nvme controller was fully detached.

Change-Id: Iaebed7b640d96fbabfc694dfebc5a725902caad2
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17850
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2023-05-09 17:58:11 +08:00
Michal Berger
d4a939dc87 test/nvme: Lock FDP test to FDP-capable nvme only
Change-Id: I199394c54914c99448153134ceb6e67a9c003f94
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17773
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2023-05-09 17:58:11 +08:00
Michal Berger
b8d6f31308 test/nvme: Add helper functions to detect FDP-capable nvme
Change-Id: I817ffbfcb3bca154cad86ca70465a923610cbabb
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17772
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2023-05-09 17:58:11 +08:00
Michal Berger
3023e70585 pkgdep/git: Bump QEMU to latest 8.0.0 release
Needed for testing latest FDP feature.

Change-Id: I83a0b46c716d6658efa4f2723c4c40e617f40cf7
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17770
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2023-05-09 17:58:11 +08:00
Michal Berger
df90fdb9c2 tests: Skip block devices marked as hidden
These devices don't come with their major:minor dev, hence they won't
pop up under /dev, i.e. are not really usable.

Change-Id: I49b39ccbedcdd1bfe37964819e15b769af22cab6
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17774
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
2023-05-09 17:58:11 +08:00
Ankit Kumar
5741feb45f test/nvme: add test application to verify fdp functionality
TP4146 introduced support for flexible data placement, which is
a data placement directive.

This application will test the new I/O management commands,
write with directives, log pages and set/get features.

Signed-off-by: Ankit Kumar <ankit.kumar@samsung.com>
Change-Id: I2d68625d9a180afb5a6e85e59738c2713ce965a8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16521
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michal Berger <michal.berger@intel.com>
2023-05-09 17:58:11 +08:00
Anil Veerabhadrappa
445c813151 ut/fc: Cleanup transport cleanup tests
This change fixes the following assert in FC UT,

   transport.c:329: spdk_nvmf_transport_create: Assertion `nvmf_get_transport_ops(transport_name) && nvmf_get_transport_ops(transport_name)->create' failed.

Change-Id: I57a9c6e83f07656b207d74bbadeb82e5efb5fa32
Signed-off-by: Anil Veerabhadrappa <anil.veerabhadrappa@broadcom.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17717
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2023-05-09 17:58:11 +08:00
Anil Veerabhadrappa
e57fb83bf9 ut/fc: add missing spdk_mempool_lookup stub
This patch fixes the following error:
        fc_ut.o: In function `nvmf_transport_create_async_done':
        spdk/lib/nvmf/transport.c:203: undefined reference to `spdk_mempool_lookup'
        collect2: error: ld returned 1 exit status

Change-Id: I6e81a8d62cfcc70bed6efe6ac807739d77ef89aa
Signed-off-by: Anil Veerabhadrappa <anil.veerabhadrappa@broadcom.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17716
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2023-05-09 17:58:11 +08:00
Jim Harris
7f292dd457 test/nvmf: fully re-enable host/timeout.sh
Now that the bug with the remove_listener path has been
fixed, we can re-enable this part of the test.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I249011b20ffe468ed499766e4333e7bf9007a962
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17797
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: <sebastian.brzezinka@intel.com>
2023-05-09 17:58:11 +08:00
Mike Gerdts
0dc9169a44 vbdev_lvol: allow degraded lvols to be deleted
An esnap clone is now deletable when its external snapshot is missing.
Likewise, the tree of degraded lvols rooted at a degraded esnap clone
can also be deleted, subject to the normal restrictions.

Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: I711ae25d57f5625a955d1f4cdb2839dd0a6cb095
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17549
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2023-05-09 17:58:11 +08:00
Mike Gerdts
a0790ea1af vbdev_lvol: load esnaps via examine_config
This introduces an examine_config callback that triggers hotplug of
missing esnap devices.

Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: I5ced2ff26bfd393d2df4fd4718700be30eb48063
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16626
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2023-05-09 17:58:11 +08:00
Mike Gerdts
0fa543f209 vbdev_lvol: degraded open of esnap clones
If an esnap clone is missing its snapshot the lvol should still open in
degraded mode. A degraded lvol will not have a bdev registered and as
such cannot perform any IO.

Change-Id: I736194650dfcf1eb78214c8896c31acc7a946b54
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16425
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2023-05-09 17:58:11 +08:00