Commit Graph

1662 Commits

Author SHA1 Message Date
Konrad Sztyber
5b18e904d2 bdev/crypto: remove quiescing reset code
After changing to spdk_accel_append_*, this code is no longer necessary,
as accel operations are now executed by the base bdev/bdev layer.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I2e31249dfdf7abeee420744a5cc75b8273ecbad9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17023
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2023-03-13 21:02:27 +00:00
Konrad Sztyber
85cf8d273f bdev/crypto: use accel appends for encryption
Similarly to reads, writes path is now also using the spdk_accel_append*
interface for performing encrypt operation.

Additionally, this patch also changes the way aux buffer is allocated -
spdk_bdev_io_get_aux_buf() was replaced with spdk_accel_get_buf().  This
ensures that the actual data buffer will be only allocated if it's
actually needed.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I3cd1d4f5753a95709d7b81de23d9227102a74261
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17022
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
2023-03-13 21:02:27 +00:00
Konrad Sztyber
5d860c18d4 bdev/crypto: use accel appends for decrypting data
The new spdk_accel_apend_decrypt() interface is now used for all read
requests for decrypting data.  This makes it possible to chain decrypt
operations with other operations in a sequence.  Support for encryption
in the write path will be added in the following patches.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I2f65044eeaf92232d839ddd166ae50889b3f386d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17021
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
2023-03-13 21:02:27 +00:00
Tomasz Zawadzki
7b56cc45f1 module/uring: report actual error code from getaddrinfo()
getaddrinfo() does not use errno to report failures, instead
does it with return value.

This patch makes sure that proper error is reported.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I56569f991ae55d4b113f280394bb9ab375465a13
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17096
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2023-03-10 16:44:37 +00:00
Marcin Spiewak
b42295dbc3 bdev/pmem: Removed bdev pmem implementation
Removed bdev pmem implementation and makefiles
Removed bdev pmem rpc calls
Updated Python scripts
Updated match files for tests
Updated doc files
pkgdep and doc files will be updated when pmem
dependency is removed from libreduce

Change-Id: Ia1be5046159e04cd933ac13a0d5791e6d28219da
Signed-off-by: Marcin Spiewak <marcin.spiewak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17070
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2023-03-09 09:21:23 +00:00
Jim Harris
c5224a96ae ocf: clarify deprecation notice
Huawei is working on picking up support for OCF and the
associated SPDK bdev module, after Intel discontinued
support.  So clarify various docs and strings to
reflect this for now.  If everything goes according
to plan, deprecation notice will get removed before
23.05 release.  If not, SPDK will just keep the
deprecation notice for 23.05, keep the code in-tree,
and give another release cycle to get things worked
out.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I9ac6bf2f509005956440b41516ca00b0c1997361
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17085
Community-CI: Mellanox Build Bot
Reviewed-by: Amir Haroush <amir.haroush@huawei.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2023-03-08 08:46:41 +00:00
Tomasz Zawadzki
2f08dc7f7f bdev/delay: add uuid option
This patch aligns delay bdev with other bdev modules
that allow passing specific uuid during their creation.
See malloc or null bdev modules.

Change-Id: Icc5e816dd42b2e55e8d57712f2d4f125e486fdab
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16840
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2023-03-08 08:46:16 +00:00
Konrad Sztyber
07be7ca0ad bdev/nvme: use non-ext IO functions if possible
Patch 55f947933 ("bdev: remove spdk_bdev_ext_io_opts from spdk_bdev_io")
changed the way bdev_nvme submits IO to the NVMe driver causing
performance degradation for requests with iovcnt = 1, as they also had
to go through the path that executes the reset_sgl/next_sge callbacks.

This patch reverts those changes back to the original code checking
iovcnt and using the non-SGL functions if possible.

Suggested-by: Jim Harris <james.r.harris@intel.com>
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I5e7c6620d38b7690ff862d8cd0075afacc578217
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16961
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2023-03-07 11:49:20 +00:00
Mike Gerdts
ab2eff0798 blob_bdev: take read-only-many claims on ro devs
If the bs_dev was opened read-write, continue to take a
read-many-write-one claim. If it was opened read-only, take a
read-many-write-none claim.

Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: I25d977c6961f962423899fb891ec912cd847930a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16282
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2023-03-03 11:25:35 +00:00
Mike Gerdts
bd5a784719 blob_bdev: support read-only devices
External snapshots, which will be introduced in a later commit, will
need read-only blob_bdev instances. This support is partially needed to
support underlying devices that are naturally read-only and partially to
provide an extra layer of protection against accidental writes.

Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: Ibcb28d00ad644a6053aa5f4de15471c2cd8e348a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14968
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2023-03-03 11:25:35 +00:00
Marcin Spiewak
2903189c40 lib/accel: removed references to ACCEL_FLAG_PERSISTENT
- Removed ACCEL_FLAG_PERSISTENT and related code

Change-Id: Icc8890ad7143fab4bd4b544e82acdf9b84c0a1eb
Signed-off-by: Marcin Spiewak <marcin.spiewak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16951
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2023-03-03 11:20:03 +00:00
Swapnil Ingle
d1b2a5c436 bdev_nvme: start accel_comp_poll poller lazily
With bdev_nvme accel_comp_poll is always created, even if it is not
really used e.g bdev_nvme with vfio-user transport.
accel_comp_poll poller is registered when accel_channel is created, in
order to start it lazily this patch creates accel_channel during first
usage.

Signed-off-by: Swapnil Ingle <swapnil.ingle@nutanix.com>
Change-Id: I5c710a038606ada00c9bfde6a4d9691ee8f6d204
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16957
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2023-03-02 08:36:03 +00:00
Jim Harris
4037f0671b sock/posix: don't add 1 to pipe size
There's no need to add 1 to the pipe size, it
actually results in unaligned accesses after we've
cycled through the pipe buffer for the first time.

Fixes: eb98488 ("sock/posix: Internally buffer reads.")

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iff79d40694dab1afe128ccac0c7c7a28cd827a3d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16924
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2023-03-01 08:56:14 +00:00
Sebastian Brzezinka
2d590e74b4 bdev/lvol: add param size_in_mib to replace size in bytes
`rpc_bdev_lvol_resize` and `rpc_bdev_lvol_create` can now use size
in MiB instead of `size` in bytes. This change make param `size`
deprecated and using both `size` and `size_in_mib` return error.

Since `bdev_lvol_resize` and `bdev_lvol_create` use size in MiB,
name of param should reflect it, previously used param `size` is
positional therefore there is no need to keep it as deprecated.

This patch fix issue: #2346

Signed-off-by: Sebastian Brzezinka <sebastian.brzezinka@intel.com>
Change-Id: Ibbe2c056bb63d9f82dee91c87fdd501ce441d5f8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16901
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2023-03-01 08:55:43 +00:00
Krzysztof Karas
609f8d4379 bdev_nvme: adjust generation of UUID to conform to the standards
Change the way we generate UUIDs to use uuid_generate_sha1()
from libuuid.

Fixes #2788.

Change-Id: Id750a7de74fb37bc3ab3ab06b3254b49d73f4be3
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15935
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
2023-02-28 08:57:01 +00:00
Shuhei Matsumoto
511daff348 bdev/nvme: Factor out start operations from failover() and delete_ctrlr()
The following patches will add mutex for bdev_nvme_delete() to improve
handling race conditions. This refactoring make such change easier.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I8971684a708bce89872f08f75db86eb3b723f380
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16821
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2023-02-28 08:56:45 +00:00
Shuhei Matsumoto
f38d2f6a8e bdev/nvme: Delete alternate paths and an active path separately
If alternate paths and an active path are treated separately in
_bdev_nvme_delete(), it will be much easier to protect a path list
from delete operation by mutex.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: Ie3bed095fd92b80c0487ef7b136953ad03a174eb
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16820
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2023-02-28 08:56:45 +00:00
Shuhei Matsumoto
422f5f3d11 bdev/nvme: Factor out deleting a ctrlr from bdev_nvme_delete()
Factor out path deletion operation for one nvme_ctrlr into _bdev_nvme_delete().

_bdev_nvme_delete() traverses all remaining nvme_ctrlrs even after one
nvme_ctrlr was successfully deleted. To remember the successfule
deletion, use two return value variables, rc and _rc. For _rc returned by
_bdev_nvme_delete(), copy _rc to rc if _rc is zero, return _rc immediately
if _rc is not -ENXIO, or do nothing otherwise.

This improves the readability and makes us easier to add more changes.

Additionally, for _bdev_nvme_reset_io(), it is simpler to overwrite rc
to 0 if rc is -EBUSY rather than to return 0 if rc is -EBUSY. This is
very small change and hence done together.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I57eafd22918c8c976b9c51bedb9e4369976c1d5c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16819
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2023-02-28 08:56:45 +00:00
Shuhei Matsumoto
8173930051 bdev/nvme: Rename _bdev_nvme_delete() by bdev_nvme_delete_ctrlr()
The next patch wants to use _bdev_nvme_delete() as a function name.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: Iba103868e74795bf42c443a377c77e9c073329fb
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16818
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2023-02-28 08:56:45 +00:00
Shuhei Matsumoto
4757c05fa1 bdev/nvme: Factor out check if path should be deleted into a helper function
This improves the readability and makes us easier to add more changes.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I55d7925d70f2a204f65a81f3fc44cf96b69c3ebe
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16709
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2023-02-28 08:56:45 +00:00
Artur Paszkiewicz
fd7641d575 bdev/lvol: retry lvol open if it fails due to insufficient resources
Fixes: #2910
Change-Id: Id28b5c4e7e002fe81fea774d989d1507d933133e
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16875
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2023-02-24 21:01:11 +00:00
Artur Paszkiewicz
0194aa5bd2 bdev/lvol: eliminate grow examine code duplication
These functions:
 _vbdev_lvs_grow_examine_finish
 _vbdev_lvs_grow_examine_cb
 _vbdev_lvs_grow_examine

are almost exact copies of:
 _vbdev_lvs_examine_finish
 _vbdev_lvs_examine_cb
 vbdev_lvs_examine

Remove the duplicated grow functions. Factor out the common entry point
into _vbdev_lvs_examine() which accepts a callback to perform the
required action (spdk_lvs_load or spdk_lvs_grow).

Change-Id: I9805f29e22812051608873207401e3786dab7103
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16942
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2023-02-24 21:01:11 +00:00
Artur Paszkiewicz
f64fe6fe34 bdev/lvol: remove _vbdev_lvol_examine_close_cb
Remove _vbdev_lvol_examine_close_cb() because it is identical as the
code at the end of _vbdev_lvs_examine_finish().

Remove the INFOLOG message because it is redundant (there's an ERRLOG
just before) and inconsistent with the lvolerrno != 0 case, where there
is no such message.

Change-Id: I8488a8b5fcbaf1895f4953081d98bc13f82be221
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16941
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2023-02-24 21:01:11 +00:00
Panfil, Wojciech
1eb06bd600 malloc_bdev: Add physical block size optional argument
In the past, we didn't use the distinction between logical block size
and physical block size. Now it's possible to optionally set the
physical block size to be different then logical block size. It's useful
for NVMe 512e Advanced Format tests.

Change-Id: I1b596da471031ee90dafc6ba6276cebf769b5ea2
Signed-off-by: Panfil, Wojciech <wojciech.panfil@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16793
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2023-02-22 12:59:10 +00:00
Panfil, Wojciech
7f50da15bc null_bdev: Add physical block size optional argument
In the past we didn't use the distinction between logical block size and physical block
size in null bdev. Now we can optionally set the physical block size to
be different then logical block size. It's useful for testing NVMe 512e
Advanced Format scenarios.

Change-Id: Ib3713af9a4d0ce7fd322170b4d87100cfc91a103
Signed-off-by: Panfil, Wojciech <wojciech.panfil@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16792
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2023-02-22 12:59:10 +00:00
Lukas Stockner
3c5a8ddc41 bdev/rbd: fix double-free when failing to initialize cluster
When bdev_rados_cluster_init encounters an error, it shuts down the cluster
before returning, but since the pointer is still set, bdev_rbd_free will
attempt to shut it down again, which causes a crash.
Therefore, set the pointer to NULL after the first shutdown to indicate
that the cluster object does not exist anymore.

Change-Id: Ie403471e8aba881cb6380e74bd1a4ca8d67cbc68
Signed-off-by: Lukas Stockner <lstockner@genesiscloud.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16203
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
2023-02-21 08:33:52 +00:00
Artur Paszkiewicz
31ce6cd8a3 module/raid: check strip size against device block size
Change-Id: I7a8d0b8f89aa8e075632faa8eac19a26a9110e57
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14872
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2023-02-17 10:15:18 +00:00
Konrad Sztyber
55f9479333 bdev: remove spdk_bdev_ext_io_opts from spdk_bdev_io
The spdk_bdev_ext_io_opts structure is used to pass extra options when
submitting a bdev IO request, without having to modify/add functions to
handle new options.  Additionally, the structure has a size field to
allow adding new fields without breaking the ABI (and thus having to
bump up the major version of a library).

It is also a part of spdk_bdev_io and there are several reasons for
removing it from that structure:

  1. The size field only makes sense in structures that are passed
     through pointers.  And spdk_bdev_ext_io_opts is indeed passed as a
     pointer to spdk_bdev_{readv,writev}_blocks_ext(), however it is
     also embedded in spdk_bdev_io (internal.ext_opts_copy), which is
     also part of the API.  It means that each time a new field is added
     to spdk_bdev_ext_io_opts, the size of spdk_bdev_io will also
     change, so we will need to bump the major version of libspdk_bdev
     anyway, thus making spdk_bdev_ext_io_opts.size useless.
  2. The size field also makes internal.ext_opts cumbersome to use, as
     each time one of its fields is accessed, we need to check the size.
     Currently the code doesn't do that, because all of the existing
     spdk_bdev_ext_io_opts fields were present when this structure was
     initially introduced, but we'd need to do check the size before
     accessing any new fields.
  3. spdk_bdev_ext_io_opts has a metadata field, while spdk_bdev_io
     already has u.bdev.md_buf, which means that we store the same thing
     in several different places in spdk_bdev_io (u.bdev.md_buf,
     u.bdev.ext_opts->metadata, internal.ext_opts->metadata).

Therefore, this patch removes all references to spdk_bdev_ext_io_opts
from spdk_bdev_io and replaces them with fields (memory_domain,
memory_domain_ctx) that were missing in spdk_bdev_io.  Unfortunately,
this change breaks the API and requires changes in bdev modules that
supported spdk_bdev_io.u.bdev.ext_opts.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I49b7524eb84d1d4d7f12b7ab025fec36da1ee01f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16773
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2023-02-16 10:09:35 +00:00
John Levon
70a82d9a95 nvmf: add spdk_nvmf_request_copy_*_buf()
Also deprecate the existing spdk_nvmf_request_data() API, which is
incompatible with iovecs.

Signed-off-by: John Levon <john.levon@nutanix.com>
Change-Id: I44df8ff30a431873a0c2f34b0cdb58df858fd7e3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16200
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2023-02-13 13:50:51 +00:00
Richael Zhuang
d8c8e3f4ee bdev_nvme: update outdated log for reset_io
It has been implemented in commit 56e2d632ce to reset all nvme_ctrlrs
of a bdev controller sequentially. Update the log.

Change-Id: Id87bf99d7e269c8ad0d6b2576e2329455601ec55
Signed-off-by: Richael Zhuang <richael.zhuang@arm.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16653
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2023-02-09 11:32:12 +00:00
Mike Gerdts
f246b2d5c8 blob_bdev: use bdev module claims v2
With the introduction bdev module claims v2, existing consumers should
transition off of v1 claims. This transitions blob bdevs from v1
exclusive writer claims to v2 read write once claims.

Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: I1884585a540fa17ee341430e03de3c4f5d35322b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16168
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2023-02-09 11:20:34 +00:00
Alexey Marchuk
cc8347dc2d bdev/crypto: Use iobuf large_bufsize to set opt IO boundary
We used hardcoded value set by bdev module, however
this value might be changed, so get a real value
from iobug config

Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: Id15b602f699f3dd63aeff11fb063bb62c384663d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16601
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2023-02-03 08:54:34 +00:00
Alexey Marchuk
a54fc9f4a2 bdev/crypto: Free aux buf when crypto op fails
Fixes issue #2886

Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: I19e2ba3763822bc9092fba584d35a6802c520ab7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16600
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
2023-02-03 08:54:34 +00:00
Shuhei Matsumoto
3c5fdc0610 bdev/xnvme: Use bdev_unregister_by_name() to delete a xnvme bdev
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I4551dee96f3241f6004fc3b8ad0381082551db8e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16587
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2023-01-31 21:40:09 +00:00
Shuhei Matsumoto
916fb46915 bdev/daos: Use bdev_unregister_by_name() to delete a daos bdev
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I8e73d0d8e6cfa76f253f5cd49b3bf37d42d44b6f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16557
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2023-01-31 21:40:09 +00:00
Shuhei Matsumoto
841cd8b939 module/blobfs: Use error_response() rather than bool_response(false) for JSON RPCs
For JSON RPC, boolean response with false value may not be regarded as error.
Previously many cases were replaced to use
spdk_jsonrpc_send_error_response() explicitly. Replace one of the
remaining cases in this patch.

rpc_blobfs_detect() uses boolean response for not only true but also
false, but it had a memory leak bug. Fix it together in this patch.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: Iaff21115282f638038345daa986b5c743ba9a3ed
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16621
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2023-01-31 21:40:09 +00:00
Shuhei Matsumoto
c77bc554d2 module/bdev: Use error_response() rather than bool_response(false) for JSON RPCs
For JSON RPC, boolean response with false value may not be regarded as error.
Previously many cases were replaced to use
spdk_jsonrpc_send_error_response() explicitly. However, this pattern did
not become established and new bdev modules used
spdk_jsonrpc_send_bool_response() with false.

Once again, replace remaining cases in this patch.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: Ie8858bc8ff7c36bc3a829977044a91d459db76f2
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16555
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2023-01-31 21:40:09 +00:00
Shuhei Matsumoto
5feab0ddcf bdev/nvme: Remove spdk_bdev_first/next() calls from apply_firmware RPC
The NVMe bdev module has been refined since the bdev_nvme_apply_firmware
RPC was added. It is enough to open any one NVMe bdev even if one NVMe
controller has multiple namespaces. Hence, let's remove
spdk_bdev_first/next() calls from rpc_bdev_nvme_apply_firmware() rather
than replacing these by spdk_for_each_bdev(). Additionally, not only
spdk_bdev_close() but also spdk_put_io_channel() and
spdk_jsonrpc_send_*_response() should be done on the original thread.
Hence, redirect to the original thread always when a NVMe command is
returned, and rename firm_ctx->thread by firm_ctx->orig_thread for
clarification. orig_thread will be better than comment.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I90b1986584b3926980fd265e4fded194eb5a2d00
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16541
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2023-01-31 21:40:09 +00:00
Shuhei Matsumoto
1974117a29 bdev/nvme: apply_firmware RPC embeds request structure into context
It is enough to embed an instance of rpc_apply_firmware structure into
firmware_update_info structure.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I41a480a0b8cd56f4b537512d5f83c5cbd5eee718
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16540
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2023-01-31 10:30:52 +00:00
Shuhei Matsumoto
f6f9cc59c6 bdev/nvme: apply_firmware() calls jsonrpc_send_error_response() directly
Previously, bdev_nvme_apply_firmware() used a temporary character array
to reduce the calls of spdk_jsonrpc_send_error_response().

Change these to call spdk_jsonrpc_send_error_response() directly.

This will simplify the next patch to replace spdk_bdev_first/next() by
spdk_for_each_bdev()ss.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I87ffc9416232dcaba30202ebbe0cb4b185a9563d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16539
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2023-01-31 10:30:52 +00:00
Nathan Claudel
669069da0f lvol: fix jsonrpc error code sign in create_lvstore
`vbdev_lvs_create` returns negative errnos. The sign was flipped here.

Change-Id: I57e0813c2ecd10695395273a1c4ea750ef00b930
Signed-off-by: Nathan Claudel <nathan.claudel@kalrayinc.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15273
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2023-01-30 16:57:14 +00:00
Jim Harris
9e4875c398 bdev_nvme: remove duplicated line from bdev_mdns_client.c
Reported-by: Alexey Marchuk <alexey.marchuk@nvidia.com>
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I075d909304d01f94ffbe150dac1109bf809f3436
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16275
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Boris Glimcher <Boris.Glimcher@emc.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2023-01-30 16:30:01 +00:00
Tomasz Zawadzki
32908cbfc8 deprecation: remove Open CAS Framework
Support OCF is now winding down and will not see further updates.

Change-Id: Ica2026d3a9ad13daf677fcce2b7974eb333971fc
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16392
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2023-01-26 15:16:19 +00:00
Tomasz Zawadzki
23196506da deprecation: Remove support for PMDK
Support for PMDK is winding down, see:
https://pmem.io/blog/2022/11/update-on-pmdk-and-our-long-term-support-strategy/

Change-Id: I8999a31137aa372e757472cfb7d4bf4edf9d2fb6
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16393
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2023-01-26 15:16:19 +00:00
Jim Harris
3a39d90b03 bdev_gpt: add new SPDK partition type for off-by-one fix
The gpt bdev module has an off-by-one error.  When it
calculates the size of the partition, it simply does
"end - start", when really it should be "end - start + 1".

We cannot just fix it by changing the math here, any
consumers of the partition may have put down metadata
on the partition based on the old size.

So instead add a new SPDK partition type.  SPDK will keep
the existing off-by-one behavior when it finds the old
partition type, but will use the correct math when finding
the new partition type.

Fixes issue #2801.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I48eb48c781f1968b59e52b4477ca45e9c81eac11
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16298
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Mike Gerdts <mgerdts@nvidia.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2023-01-24 17:19:35 +00:00
buzagg
be299bf533 bdev_nvme: Support multipath across all discovery contexts.
This is required when volume is attached via different storage boxes
for example while migrating it.

Signed-off-by: Gili Buzaglo <gili.buzaglo@kioxia.com>
Change-Id: I680c781ae5de078972d6aef547804d60fff3c643
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16407
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2023-01-24 15:26:28 +00:00
Tomasz Zawadzki
3359bf34d6 so_ver: increase all major versions
To allow SO_MINOR updates on LTS for the whole year it is supported,
the major version for all components needs to be increased.
This is to prevent scenario where two versions exists with matching
versions, but conflicting ABI.
Ex. Next SPDK release adds an API call increasing the minor version,
then LTS needs just a subset of those additions.

Increasing major so version after LTS, allows the future releases
to update versions as needed. Yet allowing LTS to increase minor
version separately.

Disabled test for increasing SO version without ABI change, as
that is goal of this patch. This check shall be removed with SPDK 23.05
release.
Looks like this was left over from prior LTS, to avoid that
make sure it is only skipped when running against v23.01.x as latest
release.

This patch:
- increases SO_VER by 1 for all components
- resets SO_MINOR to 0 for all components
- removes suppressions for ABI tests


Short reference to how the versions were changed:
MAX=$(git grep "SO_VER := " | cut -d" " -f 3 | sort -ubnr | head -1)
for((i=$MAX;i>0;i-=1)); do find . -name "Makefile" -exec \
	sed -i -e "s/SO_VER := $i\$/SO_VER := $(($i+1))/g" {} +;  done
find . -name "Makefile" -exec \
	sed -i -e "s/SO_MINOR := .*/SO_MINOR := 0/g" {} +

Change-Id: I3e5681802c0a5ac6d7d652a18896997cd07cc8bf
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16419
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2023-01-24 08:37:21 +00:00
Alexey Marchuk
52c86e21be accel/dpdk_cryptodev: Remove missed noticelog
This was added during debug/development and
was not deleted by accident.

Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: I146748151623a3ea4808c7b5cc34085d30ca81b4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16388
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2023-01-20 15:40:03 +00:00
paul luse
bb5083a85d bdev/compress: Port to use accel_fw instead of compressdev
directly

This patch removes hardcoded compressdev code from the
vbdev module and instead uses the accel_fw. The port required
a few changes based on how things are plumbed and accessed,
nothing that isn't be too obscure.  CI tests were updated to
run ISAL accel_fw module as well as DPDK compressdev with QAT.

Unit tests for the new module will follow in a separate patch.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I769cbc888658fb846d89f6f0bfeeb1a2a820767e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13610
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2023-01-20 10:26:21 +00:00
Yifan Bian
ed2b53f389 ublk: add configure and event/subsystem
ublk backend could support ublk driver with kernel. Specify
configuration parameter to start it up.

Signed-off-by: Yifan Bian <yifan.bian@intel.com>
Co-authored-by: Xiaodong Liu <xiaodong.liu@intel.com>
Change-Id: I55e7d757e04315b25e9bfab5fdcbb6621be3e29e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15680
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2023-01-20 07:48:25 +00:00