Commit Graph

19839 Commits

Author SHA1 Message Date
Damiano Cipriani
3e7c605bbc module/raid: preliminary work for add base bdev
Refactored the code creating raid_bdev_module_get_min_operational,
distinguished intentional removal from failure in superblock base
bdev state.

Signed-off-by: Damiano Cipriani <damiano.cipriani@suse.com>
2023-09-19 09:57:48 +02:00
Damiano Cipriani
4c06a1e600 module/raid: add callback for IO resume completion
Function raid_bdev_resume takes as input parameters also a function
pointer and a context to be called at the end of the resume operation.
If no callback should be called, NULL parameters can be passed.

Signed-off-by: Damiano Cipriani <damiano.cipriani@suse.com>
2023-09-19 09:57:48 +02:00
Damiano Cipriani
4542f9b140 lib/json: format fix of append double quote
A couple of format fix shown by check_format.sh in the commit
"Always append one double quote char in the end of string".

Signed-off-by: Damiano Cipriani <damiano.cipriani@suse.com>
2023-06-30 23:47:17 +08:00
Derek Su
69f656824e Always append one double quote char in the end of string
The returned value write_string_or_name() is not really checked, so one double quote char error
is not identified and the broken json string is directly sent to the client.

To address the issue, the workaround is always appending one double quote char in the end of string.

TODO:
1. Find the root cause
2. Check the returned value of write_string_or_name()

Longhorn 6190

Signed-off-by: Derek Su <derek.su@suse.com>
2023-06-27 09:00:15 +08:00
Damiano Cipriani
c1d0ea9a39 lvol/blob: fix get shallow copy status
Function spdk_blob_get_shallow_copy_result has been added to
spdk_blob.map file because it is a public API function.

Signed-off-by: Damiano Cipriani <damiano.cipriani@suse.com>
2023-06-27 08:58:19 +08:00
Damiano Cipriani
575a9a8669 lvol/blob: get shallow copy status of ended ops
Shallow copy status RPC now returns info of ongoing and even
of ended operations.

Signed-off-by: Damiano Cipriani <damiano.cipriani@suse.com>
2023-06-12 17:13:54 +08:00
Derek Su
59f61e6fd0 Update SPDK_LVOL_NAME_MAX and SPDK_LVS_NAME_MAX to 256
Signed-off-by: Derek Su <derek.su@suse.com>
2023-06-12 16:45:03 +08:00
Damiano Cipriani
c22a9a7683
Merge pull request #5 from DamiaSan/lvol_shallow_copy
lvol shallow copy
2023-06-06 17:18:41 +02:00
Damiano Cipriani
c34d4d490d
lvol/blob: add shallow copy over a given device
A shallow copy will copy over the destination device only the
cluster allocated to the blob/lvol discarding those belonging
to the blob/lvol parent snapshot. blob/lvol must be read only.

Signed-off-by: Damiano Cipriani <damiano.cipriani@suse.com>
2023-06-06 15:58:36 +02:00
Damiano Cipriani
980f535d38 module/raid: fix rebase with master
This is a fix of a rebase operation done over the branch
created from Gerrit 16167 patch

Signed-off-by: Damiano Cipriani <damiano.cipriani@suse.com>
2023-05-10 19:59:21 +08:00
Damiano Cipriani
6322b4ce40 module/raid: increase raid name size in superblock
The size of raid name inside raid superblock structure is
increased to 64 chars to let the creation of raid bdev with longer name.

Signed-off-by: Damiano Cipriani <damiano.cipriani@suse.com>
2023-05-09 17:58:11 +08:00
Damiano Cipriani
01e9afd39e raid1: allow creation with a single base bdev
The minimun number of raid1 base bdevs is set to 1, so a raid1 bdev
can be created with only one base bdev.

Signed-off-by: Damiano Cipriani <damiano.cipriani@suse.com>
2023-05-09 17:58:11 +08:00
Artur Paszkiewicz
e325fbafec module/raid: allow assembly of a degraded raid
Add num_base_bdevs_operational to raid_bdev and use it to determine the
required number of base bdevs.

Change-Id: I31b39cc8ea708b6cdce748f015949e4c9fdeb3cd
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
2023-05-09 17:58:11 +08:00
Artur Paszkiewicz
ea4b2f6d75 module/raid: update superblock on base bdev removal
Change-Id: I713053a4928139fdf8aa43ebf47a743bec3d5054
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
2023-05-09 17:58:11 +08:00
Artur Paszkiewicz
432d1a99cc module/raid: show base bdev details in json
Change-Id: I0da3e91e7736bc651e284f68238ace864def87b2
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
2023-05-09 17:58:11 +08:00
Artur Paszkiewicz
e61c1c51be module/raid: check for existing superblock on a base_bdev
When adding a new base bdev to a raid bdev (currently only when creating
a new raid bdev) make sure that there is no existing superblock
stored on the base bdev. This prevents accidentally overwriting a base
bdev belonging to a different raid array.

Change-Id: Id5f6c7e3ed7223f6a8fc7455f75831fbbcac7e43
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
2023-05-09 17:58:11 +08:00
Artur Paszkiewicz
957acd43b9 module/raid: assemble raid bdev from superblock
Change the bdev_raid examine procedure to read the superblock from the
examined base bdev. If a valid superblock is found, re-create the
raid_bdev from it.

Change-Id: I4bd589647a207a216ecf0dec9baf11c5d691f5d5
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
2023-05-09 17:58:11 +08:00
Artur Paszkiewicz
668c5a769e module/raid: write initial superblock
When creating the raid_bdev with enabled superblock option, write the
superblock to the base bdevs before bringing the array online.

Change-Id: I24659202ef3bbe6c87ca8603d514bd81660c9b41
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
2023-05-09 17:58:11 +08:00
Artur Paszkiewicz
84f8594c80 test/bdev_raid: add raid5f tests
Change-Id: Iebecbe09c8808c553ccaec2973852d1c9dd115bf
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
2023-05-09 17:58:11 +08:00
Artur Paszkiewicz
43a6a14e1b test/bdev_raid: test more than 2 base bdevs in raid
Change-Id: I3b105a33dc89fd654a21de548f5aba7fa29c6ee7
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
2023-05-09 17:58:11 +08:00
Artur Paszkiewicz
cb56a42775 test/bdev: add raid1 tests
Change-Id: Ib7c8f8818be9da24e829a4d4470671cd39234543
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
2023-05-09 17:58:11 +08:00
Artur Paszkiewicz
3728239ab5 test/bdev_raid: simplify test cleanup
There is a trap set up to kill the spdk process in case of error, there
is no need to delete any bdevs before that.

Change-Id: Ic80e2a48453f718fbc42cabe88d86eefa35c95db
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
2023-05-09 17:58:11 +08:00
Artur Paszkiewicz
69946f9eac module/raid: use raid_bdev_free() for cleanup in raid_bdev_create()
Change-Id: I73e283d4f5d98f2bb95d0dbab7e3abbb007134e4
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
2023-05-09 17:58:11 +08:00
Artur Paszkiewicz
ff9d703946 module/passthru: add uuid option for creating passthru bdev
Change-Id: I1a298161018553feea00248568f2ea786a08ff64
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
2023-05-09 17:58:11 +08:00
Artur Paszkiewicz
882ecb55a8 util/uuid: add API to test/set null uuid
Refactor the code to use these new functions.

Change-Id: I21ee7e9a96f30fbd60106add5e8b071e86bf93c9
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
2023-05-09 17:58:11 +08:00
Krzysztof Smolinski
52726c9b6e test/raid1: unit tests for read balancing
Signed-off-by: Krzysztof Smolinski <krzysztof.smolinski@intel.com>
Change-Id: Ic42bfe8be2d0d76e804a82b4837391384f01f8f8
2023-05-09 17:58:11 +08:00
Krzysztof Smolinski
4d783e7255 raid1: read balancing
Reads for raid1 bdevs are balanced. Algorithm tries to evenly distribute
load by sending read I/O to all base bdevs following round-robin, but
skiping base bdev that processed most data so far.

Signed-off-by: Krzysztof Smolinski <krzysztof.smolinski@intel.com>
Change-Id: I7d85411a6421bd7352031efb562ee95f2c612011
2023-05-09 17:58:11 +08:00
Artur Paszkiewicz
951d766289 raid5f: degraded io support
Change-Id: I1af2ffc3fe1f41b798e15b5194ab5695923737ef
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
2023-05-09 17:58:11 +08:00
Artur Paszkiewicz
d0c7517b84 ut/raid5f: assume raid_io_info is always one stripe
Support for multi-stripe requests is not used anymore so remove it.

Change-Id: I8f28817763452674c8a183c640800f3a4b4b3653
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
2023-05-09 17:58:11 +08:00
Artur Paszkiewicz
284fd419cb ut/raid5f: simplify read tests
Remove struct test_request_conf and instead do some basic reads on each
chunk. Also remove the io_info splitting because it is not used now.

Change-Id: I4b945b40598670f6ab84fb8066278877fee7fb75
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
2023-05-09 17:58:11 +08:00
Krzysztof Smolinski
6680fdf818 module/raid: bdev_raid_remove_base_bdev rpc
Signed-off-by: Krzysztof Smolinski <krzysztof.smolinski@intel.com>
Change-Id: I4829f6cd0c10bfcd2c6893cf9412fc974c4b338c
2023-05-09 17:58:11 +08:00
Artur Paszkiewicz
cdf0959bc4 raid1: degraded mode support
Properly handle IO when one or more base bdevs are missing.

Change-Id: I51161b01a625c20da5156d7db1c5e5d9b62ce298
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
2023-05-09 17:58:11 +08:00
Artur Paszkiewicz
65f2af1dcb module/raid: continue operation after base bdev removal
Don't stop the raid bdev if the minimum number of base bdevs are
available.

When removing a base bdev, first suspend the raid bdev and then perform
the actual removal/cleanup. Finally, resume the raid bdev.

Change-Id: Ie010d3760c32b0dad455a5a2a0ab7adcc602edf9
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
2023-05-09 17:58:11 +08:00
Artur Paszkiewicz
f71c255827 module/raid: add raid_bdev ptr to raid_base_bdev_info
This allows to simplify some code where raid_bdev and base_info are
needed.

Change-Id: I40395204fdcdd0487bdecec1cd47efb347f1310a
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
2023-05-09 17:58:11 +08:00
Artur Paszkiewicz
79eccac059 module/raid: suspend/resume IO
Add functions to suspend and resume IO on all channels. This will be
used to safely change the device state in case of e.g. removing a base
bdev.

Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Change-Id: I203c1899bde15101e0c2bc8da7a1066a2fee6dd2
2023-05-09 17:58:11 +08:00
Krzysztof Smolinski
8c591e2d4f module/raid: data offset and data size implementation
When raid bdev is created with superblock parameter then all data on
this bdev should be shifted by some offset. Such space at the beginning
of bdev will be used to store on-disk raid metadata.

Signed-off-by: Krzysztof Smolinski <krzysztof.smolinski@intel.com>
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Change-Id: I2545a2b00a651ef5332ca1757da0110a63914a43
2023-05-09 17:58:11 +08:00
Krzysztof Smolinski
19c79b0d01 rpc/bdev_raid_create: added superblock parameter
Introduction of superblock parameter for bdev_raid_create rpc. This
parameter determines whether raid bdev should be created with support
for on-disk metadata (support for raid on-disk metadata is going to be
implemented in the future).

Signed-off-by: Krzysztof Smolinski <krzysztof.smolinski@intel.com>
Change-Id: Ie8c64f837dd7eb3ba788b7c5d7bc98e8f1368ba7
2023-05-09 17:58:11 +08:00
Artur Paszkiewicz
ba752093db raid5f: convert to use accel framework for xor
Change-Id: Id8fb521549342564bcf4288d74337fb4dd41fa03
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
2023-05-09 17:58:11 +08:00
Konrad Sztyber
e5dbea5967 test/bdev: extend chaining test with bdev layer ENOMEM case
The test already checked ENOMEM handling, but it only used bdevs that
support chaining (crypto, malloc), so bdev layer didn't need to execute
any accel operations.  So, to force bdev layer to do that, a passthru
bdev was added, as it doesn't support chaining.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I322a65ccebb0f144c759692fff285cfd44bbab4b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17766
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2023-05-09 17:58:11 +08:00
Konrad Sztyber
62ce9dbcf6 bdev: remove handle_no_momem from push/seq cb
The IOs are never completed with NOMEM from push/sequence callbacks and
NOMEM IOs are already retried in internal callbacks, so there's no point
in calling _bdev_io_handle_no_mem().

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Iecc2a41f2a394836f62d541e6235277f333f226b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17765
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
2023-05-09 17:58:11 +08:00
Konrad Sztyber
8e66d976cc bdev: rename (pull|push)_done callbacks
The functions that were passed as callbacks for the memory domain
pull/push calls were prefixed with an underscore, which doesn't really
explain the difference between the corresponding functions without an
underscore.  So, they're now renamed to *_and_track() to emphasize that
they additionally responsible for tracking IOs.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ia9e56230fe244d2c64d729e97445fae105418a76
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17931
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2023-05-09 17:58:11 +08:00
Konrad Sztyber
32683a847a bdev: retry IOs on ENOMEM when pushing bounce data/md
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ia7634b570eb7d04c22003337a46630d152171157
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17764
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2023-05-09 17:58:11 +08:00
Konrad Sztyber
45f02f07f0 bdev: enqueue IOs on the memory domain queue only when pushing
The IOs don't need to be put onto the io_memory_domain queue if there's
no need for memory domain push.  This makes push_data consistent with
other memory domain operations (pull_data, pull_md, push_md).

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I85d95f6ce580a15b23f56ab5101e49236f341cb1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17763
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2023-05-09 17:58:11 +08:00
Jim Harris
8de18d3ccb nvmf: use iterator APIs in nvmf_tgt_destroy_cb
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I27b1b851fc8f47150670636cb65ccba40d1a57d6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17961
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Community-CI: Mellanox Build Bot
2023-05-09 17:58:11 +08:00
Jim Harris
c4ce596187 nvmf: refactor nvmf_tgt_destroy_cb
This preps for some upcoming patches as well as
removing two levels of indentation.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I4f685c1e44ec4aa261e68af1786cfc110f451ed5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17960
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2023-05-09 17:58:11 +08:00
Jim Harris
c387766501 nvmf: use iterator APIs in nvmf_tgt_create_poll_group()
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I4d9a5dd4655edb8315503e7551aec1926d1cc017
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17959
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
2023-05-09 17:58:11 +08:00
Jim Harris
37d0433fc3 nvmf: use iterator APIs to generate discovery log
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iedd1c0a92e8b5f839ad4905d8063a04ec47f3d9b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17938
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2023-05-09 17:58:11 +08:00
Jim Harris
35a3cfd51e examples/nvme: fix reconnect memory leaks
1) In submit_single_io(), if an I/O fails to submit,
   we need to free the associated task structure,
   otherwise it gets leaked.
2) When draining I/O, just always check_io() instead
   of only doing it when current_queue_depth > 0.
   This is the simplest of ensuring that we cleanup
   the ns_ctx (including freeing the IO qpairs and
   the qpair pointer array) if the current_queue_depth
   is already 0 when starting to drain.

Fixes issue #2995.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I53f336c6a11ff63782dc81c087a58feca0e8a5d7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17873
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2023-05-09 17:58:11 +08:00
Jim Harris
711b9ca207 nvme: remove unnecessary initialization value
spdk_nvme_trid_populate_transport() inits
trstring to an empty string, but that value
is never used - it always gets overwritten by
a different value before it gets used.

Found by scan-build.

Fixes issue #3003.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I2f5f9bedd39fc540df758ad3e6719ba992552896
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17872
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
2023-05-09 17:58:11 +08:00
Jim Harris
6c760ec607 nvme: remove deprecated spdk_nvme_ctrlr_prepare_for_reset()
Note that the prepare_for_reset flag in spdk_nvme_ctrlr is
still needed - it's just set now in the nvme_ctrlr_disconnect
path instead of this deprecated and now removed API.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I0a6aa1c72767eb67a84b8928a986e06cbac88240
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17936
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2023-05-09 17:58:11 +08:00