Commit Graph

7062 Commits

Author SHA1 Message Date
Jim Harris
02799aa571 test: clarify GPT related comment in blockdev.sh
After we create the GPT, we change the partition type
GUID to the associated SPDK value.  The current
comment just says "change the GUID" which is
ambiguous because there are multiple GUIDs associated
with each partition.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Id821c5c5bbd7a72d84d5ddf4d91d633307f2235b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17855
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2023-05-09 17:58:11 +08:00
Mike Gerdts
57b47f209f lvol: esnap clones must end on cluster boundary
When regular lvols are created, their size is rounded up to the next
cluster boundary. This is not acceptable for esnap clones as this means
that the clone may be silently grown larger than external snapshot. This
can cause a variety of problems for the consumer of an esnap clone lvol.

While the better long-term solution is to allow lvol sizes to fall on
any block boundary, the implementation of that needs to be suprisingly
complex to support creation and deletion of snapshots and clones of
esnap clones, inflation, and backward compatibility.

For now, it is best to put in a restriction on the esnap clone size
during creation so as to not hit problems long after creation. Since
lvols are generally expected to be large relative to the cluster size,
it is somewhat unlikely that this restriction will be a significant
limitation.

Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: Id7a628f852a40c8ec2b7146504183943d723deba
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17607
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2023-05-09 17:58:11 +08:00
Mike Gerdts
dcd012e8d0 vbdev_lvol: esnap memdomain support
Return the total number of memory domains supported by the blobstore and
any external snapshot bdev.

Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: I2f8afba6b31e689b8f942e2cf36906a0a30f38c8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16430
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2023-05-09 17:58:11 +08:00
Michal Berger
cf69d9af82 test/spdkcli: Wait until spdkcli_clear_config settled
Similarly to https://review.spdk.io/gerrit/c/spdk/spdk/+/17559, this
part may also be affected by:

Error in cmd: load_config /root/spdk/test/spdkcli/config.json (
load_config /root/spdk/test/spdkcli/config.json
request:
{
  "action_on_timeout": "none",
  "timeout_us": 0,
  "timeout_admin_us": 0,
  "keep_alive_timeout_ms": 10000,
  "transport_retry_count": 4,
  "arbitration_burst": 0,
  "low_priority_weight": 0,
  "medium_priority_weight": 0,
  "high_priority_weight": 0,
  "nvme_adminq_poll_period_us": 10000,
  "nvme_ioq_poll_period_us": 0,
  "io_queue_requests": 1024,
  "delay_cmd_submit": true,
  "bdev_retry_count": 3,
  "transport_ack_timeout": 0,
  "ctrlr_loss_timeout_sec": 0,
  "reconnect_delay_sec": 0,
  "fast_io_fail_timeout_sec": 0,
  "generate_uuids": false,
  "transport_tos": 0,
  "io_path_stat": false,
  "method": "bdev_nvme_set_options",
  "req_id": 31
}
Got JSON-RPC error response
response:
{
  "code": -1,
  "message": "RPC not permitted with nvme controllers already attached"
}

so make sure the nvme controller was fully detached.

Change-Id: Iaebed7b640d96fbabfc694dfebc5a725902caad2
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17850
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2023-05-09 17:58:11 +08:00
Michal Berger
d4a939dc87 test/nvme: Lock FDP test to FDP-capable nvme only
Change-Id: I199394c54914c99448153134ceb6e67a9c003f94
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17773
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2023-05-09 17:58:11 +08:00
Michal Berger
b8d6f31308 test/nvme: Add helper functions to detect FDP-capable nvme
Change-Id: I817ffbfcb3bca154cad86ca70465a923610cbabb
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17772
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2023-05-09 17:58:11 +08:00
Michal Berger
3023e70585 pkgdep/git: Bump QEMU to latest 8.0.0 release
Needed for testing latest FDP feature.

Change-Id: I83a0b46c716d6658efa4f2723c4c40e617f40cf7
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17770
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2023-05-09 17:58:11 +08:00
Michal Berger
df90fdb9c2 tests: Skip block devices marked as hidden
These devices don't come with their major:minor dev, hence they won't
pop up under /dev, i.e. are not really usable.

Change-Id: I49b39ccbedcdd1bfe37964819e15b769af22cab6
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17774
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
2023-05-09 17:58:11 +08:00
Ankit Kumar
5741feb45f test/nvme: add test application to verify fdp functionality
TP4146 introduced support for flexible data placement, which is
a data placement directive.

This application will test the new I/O management commands,
write with directives, log pages and set/get features.

Signed-off-by: Ankit Kumar <ankit.kumar@samsung.com>
Change-Id: I2d68625d9a180afb5a6e85e59738c2713ce965a8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16521
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michal Berger <michal.berger@intel.com>
2023-05-09 17:58:11 +08:00
Anil Veerabhadrappa
445c813151 ut/fc: Cleanup transport cleanup tests
This change fixes the following assert in FC UT,

   transport.c:329: spdk_nvmf_transport_create: Assertion `nvmf_get_transport_ops(transport_name) && nvmf_get_transport_ops(transport_name)->create' failed.

Change-Id: I57a9c6e83f07656b207d74bbadeb82e5efb5fa32
Signed-off-by: Anil Veerabhadrappa <anil.veerabhadrappa@broadcom.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17717
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2023-05-09 17:58:11 +08:00
Anil Veerabhadrappa
e57fb83bf9 ut/fc: add missing spdk_mempool_lookup stub
This patch fixes the following error:
        fc_ut.o: In function `nvmf_transport_create_async_done':
        spdk/lib/nvmf/transport.c:203: undefined reference to `spdk_mempool_lookup'
        collect2: error: ld returned 1 exit status

Change-Id: I6e81a8d62cfcc70bed6efe6ac807739d77ef89aa
Signed-off-by: Anil Veerabhadrappa <anil.veerabhadrappa@broadcom.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17716
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2023-05-09 17:58:11 +08:00
Jim Harris
7f292dd457 test/nvmf: fully re-enable host/timeout.sh
Now that the bug with the remove_listener path has been
fixed, we can re-enable this part of the test.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I249011b20ffe468ed499766e4333e7bf9007a962
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17797
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: <sebastian.brzezinka@intel.com>
2023-05-09 17:58:11 +08:00
Mike Gerdts
0dc9169a44 vbdev_lvol: allow degraded lvols to be deleted
An esnap clone is now deletable when its external snapshot is missing.
Likewise, the tree of degraded lvols rooted at a degraded esnap clone
can also be deleted, subject to the normal restrictions.

Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: I711ae25d57f5625a955d1f4cdb2839dd0a6cb095
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17549
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2023-05-09 17:58:11 +08:00
Mike Gerdts
a0790ea1af vbdev_lvol: load esnaps via examine_config
This introduces an examine_config callback that triggers hotplug of
missing esnap devices.

Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: I5ced2ff26bfd393d2df4fd4718700be30eb48063
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16626
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2023-05-09 17:58:11 +08:00
Mike Gerdts
0fa543f209 vbdev_lvol: degraded open of esnap clones
If an esnap clone is missing its snapshot the lvol should still open in
degraded mode. A degraded lvol will not have a bdev registered and as
such cannot perform any IO.

Change-Id: I736194650dfcf1eb78214c8896c31acc7a946b54
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16425
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2023-05-09 17:58:11 +08:00
Mike Gerdts
a4862f5a56 vbdev_lvol: add bdev_lvol_get_lvols RPC
This provides information about logical volumes without providing
information about the bdevs. It is useful for listing the lvols
associated with specific lvol stores and for listing lvols that are
degraded and have no associated bdev.

Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: I795161ac88d9707831d9fcd2079635c7e46ecc42
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17547
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2023-05-09 17:58:11 +08:00
Mike Gerdts
68cde3b770 vbdev_lvol: external snapshot rpc interface
Add RPC interfaces for creation of esnap clone lvols. This also
exercises esnap clone creation and various operations involving
snapshots and clones of esnap clones to ensure that bdev_get_bdevs
reports state correctly.

Change-Id: Ib87d01026ef6e45203c4d9451759885a7be02d87
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14978
Reviewed-by: Michal Berger <michal.berger@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2023-05-09 17:58:11 +08:00
Mike Gerdts
aa209d4ecb test/common: allow tests to use set -u
Now autotest_common.sh is tolerant of tests that use "set -u" so that
they quickly generate useful errors when variables are used but not set.

Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: I5d7709f3029fa8f52affecf68a4b9da97a84589d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16703
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michal Berger <michal.berger@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2023-05-09 17:58:11 +08:00
Mike Gerdts
4ac5c293d6 test/lvol: test esnap clones with real bdevs
This adds test/lvol/esnap for functional tests lvol esnap clone bdevs
without RPCs or reactors.

Change-Id: If62b1bde2b19343af51ba4c11599623556484b0d
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16705
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2023-05-09 17:58:11 +08:00
Mike Gerdts
be54ccc4a9 vbdev_lvol: allow creation of esnap clones
This adds the ability for create esnap clone lvol bdevs.

Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: Ifeef983430153d84d896d282fe914c6671283762
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16590
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2023-05-09 17:58:11 +08:00
Mike Gerdts
f5e42c6c3f vbdev_lvol: create esnap blobstore device
Register an spdk_bs_esnap_dev_create callback when initializing or
loading an lvstore. This is the first of several commits required to add
support enable lvol bdevs to support external snapshots and esnap
clones.

Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: I35c4e61fdbe5b93d65b9374e0ad91cb7fb94d1f4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16589
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2023-05-09 17:58:11 +08:00
Mike Gerdts
7c42a93a35 lvol: add spdk_lvol_get_by_* API
spdk_lvol_get_by_uuid() allows lookup of lvols by the lvol's uuid.

spdk_lvol_get_by_names() allows lookup of lvols by the lvol's lvstore
name and lvol name.

Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: Id165a3d17b76e5dde0616091dee5dff8327f44d0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17546
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2023-05-09 17:58:11 +08:00
Mike Gerdts
b920476b3a lvol: add spdk_lvol_iter_immediate_clones()
Add an interator that calls a callback for each clone of a snapshot
volume. This follows the typical pattern of stopping iteration when the
callback returns non-zero.

Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: If88ad769b72a19ba0993303e89da107db8a6adfc
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17545
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2023-05-09 17:58:11 +08:00
Mike Gerdts
db028ab064 lvol_ut: test esnap hotplug
This exercises spdk_lvs_esnap_notify_hotplug() under a variety of happy
and not-so-happy paths.

Change-Id: I1f4101a082b113dacc7d03f81ca16069acfb458d
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17602
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2023-05-09 17:58:11 +08:00
Mike Gerdts
250acc1792 lvol: hotplug of missing esnaps
This introduces spdk_lvs_notify_hotplug() to trigger the lvstore to call
the appropriate lvstore's esnap_bs_dev_create() callback for each esnap
clone lvol that is missing the device identified by esnap_id.

Change-Id: I0e2eb26375c62043b0f895197b24d6e056905aa2
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16428
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2023-05-09 17:58:11 +08:00
Mike Gerdts
6d71e476ec lvol: keep track of missing external snapshots
If an lvol is opened in degraded mode, keep track of the missing esnap
IDs and which lvols need them. A future commit will make use of this
information to bring lvols out of degraded mode when their external
snapshot device appears.

Change-Id: I55c16ad042a73e46e225369bfff2631958a2ed46
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16427
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2023-05-09 17:58:11 +08:00
Mike Gerdts
2e1b12b45f blob: esnap clones are not clones
spdk_blob_is_clone() should return true only for normal clones. To
detect esnap clones, use spdk_blob_is_esnap_clone(). This also clarifies
documentation of spdk_blob_is_esnap_clone() to match the implementation.

Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: I9993ab60c1a097531a46fb6760124a632f6857cd
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17544
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2023-05-09 17:58:11 +08:00
Mike Gerdts
42e50f66d0 blob: add spdk_blob_is_degraded()
In preparation for supporting degraded lvols, spdk_blob_is_degraded() is
added. To support this, bs_dev gains an optional is_degraded() callback.
spdk_blob_is_degraded() returns false so long as no bs_dev that the blob
depends on is degraded. Depended upon bs_devs include the blobstore's
device and the blob's back_bs_dev.

Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: Ib02227f5735b00038ed30923813e1d5b57deb1ab
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17516
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2023-05-09 17:58:11 +08:00
Mike Gerdts
510b723ba9 blob: add spdk_blob_get_esnap_bs_dev()
While getting memory domains, vbdev_lvol will need to be able to access
the bdev that acts as the lvol's external snapshot. The introduction of
spdk_blob_get_esnap_bs_dev() facilitates this access.

Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: I604c957a468392d40b824c3d2afb00cbfe89cd21
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16429
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2023-05-09 17:58:11 +08:00
Konrad Sztyber
9df8e01c88 accel: append support for crc32c
It is now possible to append an operation calculating crc32c to an accel
sequence.  A crc32c operation needs special care when it's part of a
sequence, because it doesn't have a destination buffer.  It means that
we can remove copy operations following crc32c only when it's possible
to change the dst buffer of the operation preceding crc32c.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I29204ce52d635162d2202136609f8f8f33db312d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17427
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2023-05-09 17:58:11 +08:00
Michal Berger
3f593969ed scripts/pkgdep: Add support for rocky|centos 9
Also, shuffle DAOS pieces a bit to keep repo handling in one place.
Also, also switch ceph repo to an actively supported release, common
and available for both centos|rocky 8|9 (i.e. pacific).

Change-Id: Idb19e4a5ff80770c7d6f9e6db85f983e163958e6
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17661
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2023-05-09 17:58:11 +08:00
Karol Latecki
877bb5dcac test/vhost: increase memory in virtio tests
Increase the memory for spdk virtio initiator
processes using "-s" option.

See https://review.spdk.io/gerrit/c/spdk/spdk/+/17371
22fa84f77a

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I2f425cb547e72e1ac6748e777158427dcf57b9f0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17662
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2023-05-09 17:58:11 +08:00
Jim Harris
90656238b4 test/unit: disable sock unit tests on FreeBSD
There are several failing signatures observed as
part of issue #2943.  So disable the unit tests for
now until they are debugged.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iae54f8bfcd7883c02152abee37410a998da81dd7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17573
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2023-05-09 17:58:11 +08:00
Michal Berger
0a845fec27 test/vhost: Create wrapper around wipefs
Call sync each time, as an extra step, to make sure all the writes on the underlying device completed. This is needed, as on occasion parted (called right after wipefs) fails to create pt complaining that the target device (and its partitions) are still in use.

Change-Id: I959d9b36a1588ec3754335995e3e8bc5057bfeb7
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17498
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2023-05-09 17:58:11 +08:00
Marcin Spiewak
5ea00c32d6 lvol_ut: add test for invalid options
Add unit test for calling spdk_lvs_load_ext()/lvs_load()
with invalid options (opts_size is 0).

Change-Id: I9c48b972066cf977304e3efa936827d1ef1b5250
Signed-off-by: Marcin Spiewak <marcin.spiewak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17584
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Mike Gerdts <mgerdts@nvidia.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2023-05-09 17:58:11 +08:00
Konrad Sztyber
35b1d8c7da accel: copy memory domain context when merging tasks
When changing src/dst buffers, we copied memory domain pointers, but we
didn't copy memory domain context, which is obviously incorrect.  It was
probably missed, because we never append a copy with non-NULL memory
domain.  Added a unit test case to verify this behavior.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ic174e0e72c33d3f437f0faddd3405638049f0c74
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17425
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2023-05-09 17:58:11 +08:00
Konrad Sztyber
5147911114 accel: move accel_module.h to include/spdk
This file should be external to enable out-of-tree accel modules.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I2e973d0e88d7145d0fc9714f56db48486b00f3b7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17419
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2023-05-09 17:58:11 +08:00
Konrad Sztyber
072703d510 bdev/crypto: complete IOs on ENOMEM from accel
spdk_bdev_queue_io_wait() can only be used when one of bdev submission
functions returns ENOMEM (i.e. there are no more spdk_bdev_ios on that
IO channel).  Using it in any other case, e.g. on spdk_accel_append_*()
returning ENOMEM, will most likely result in failure.  Therefore, to
avoid that, the IOs are completed with NOMEM status relying on the bdev
layer to retry them.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ie0f03496e5d3180c481815b3f1b021e74ae2f46d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17319
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2023-05-09 17:58:11 +08:00
Konrad Sztyber
c812fe0948 bdev/malloc: report accel sequence support
This actually allows malloc bdev to chain multiple accel operations
together.  And, since the last operation will always be a copy, accel
should remove that copy by modifying previous operation's dst/src.

On my system, it improved bdevperf performance (single core, qd=4,
bs=128k, bdev_crypto on top of bdev_malloc, crypto_sw):

randread: 5668M/s -> 8201M/s
randwrite: 5148M/s -> 7856M/s

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I5b9173fa70a42ee56f56c496a34037d46d2f420f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17202
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2023-05-09 17:58:11 +08:00
Konrad Sztyber
16cfc9a656 bdev/malloc: use appends for write requests
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ief6c873a5f65274a25b67bc3f2811d8f3e4a33b3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17200
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2023-05-09 17:58:11 +08:00
Konrad Sztyber
c66b6c283c bdev/malloc: use appends for read requests
This only changes the interface bdev_malloc uses for scheduling the copy
to appends, but it won't chain those copies to an existing sequence, as
bdev_malloc doesn't report support for accel sequences yet.  That will
be changed in one of the subsequent patches.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I6db2c79b15cb96a1b07c6cf5514004c76b9d2e92
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17198
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2023-05-09 17:58:11 +08:00
Konrad Sztyber
81b5b941e9 accel: make spdk_accel_sequence_finish() void
It always returns 0 and any errors are reported in the callback.  Making
it void simplifies error handling.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I0d4299a2789a688eae38d76de46d1baf27cbbd8f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17194
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
2023-05-09 17:58:11 +08:00
Konrad Sztyber
967937e213 test/bdev: accel chaining test
This test sends several read/write requests and verifies the expected
number of accel operations have been executed.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Idda46ef00dc5bcc0a176d3dfb39f3f3861964741
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17193
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2023-05-09 17:58:11 +08:00
Michal Berger
f28978a106 test/packaging: Export LD_LIBRARY_PATH, PKG_CONFIG_PATH setup
095f40630e missed the autobuild dependencies while enabling the rpm
test against the external DPDK build. Without it, DPDK is not able
to properly configure itself against ipsec and isa-l libs.

Change-Id: Ia4307f0d0f9c1f82f6f80ca06113a5289c2916ed
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17576
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Kamil Godzwon <kamilx.godzwon@intel.com>
2023-05-09 17:58:11 +08:00
Michal Berger
5ff8838dda test/spdkcli: Include errors from a failed command
Dump it to stdin to make the debugging easier.

Change-Id: I9b13d0a77e45aa84ec2a55b7b982225592f2566d
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17560
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2023-05-09 17:58:11 +08:00
Michal Berger
f714562f5a test/spdkcli: Wait long enough for the nvme ctrl to be gone
Some nvmes need more time to attach|detach to|from, hence having a
static sleep is not ideal depending on what type of the nvme was
picked up for the test. Instead, simply wait until the list of
nvme ctrls is empty after the cleanup.

Change-Id: I2fc2630020436d0e1f6b01a5ce60aea56e7bf8ec
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17559
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2023-05-09 17:58:11 +08:00
Jim Harris
20e5d5388e test: wait_for_examine and delay in reap_unregistered_poller.sh
We need to give the thread library some time to reap
the unregistered poller - it is a bit of a delayed
process.  We have to wait for examine to finish on
the aio bdev, then the poll group gets destroyed and
the pollers unregistered.  This marks the pollers as
UNREGISTERED, but they aren't actually reaped until
next time the thread is polled.

Fixes issue #2980.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I1e32c50ea9b28ea2d5560ddc9b2f68fa81e708d9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17575
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Mike Gerdts <mgerdts@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2023-05-09 17:58:11 +08:00
Konrad Sztyber
35993ac9de accel: add accel_set_options
It'll allow for setting accel-specific options.  For now, it makes the
size of iobuf caches configurable.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Iaf505cc5e98dc6411453d9964250a4ba22267d79
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17188
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2023-05-09 17:58:11 +08:00
Michal Berger
8d3118f156 test/check_so_deps: Adjust printout when $SPDK_ABI_DIR is not set
There's no default defined so it doesn't have to be available in the
env at all. Adjust the echo so we don't include an empty string.

Change-Id: Icaa75915544f9da1adcdcdeafce29f5ae97149ab
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17428
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2023-05-09 17:58:11 +08:00
Michal Berger
fa633b2825 autopackage: Move packaging test to autobuild
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: Ifbe4d98f3d7a4b9970f923acd6d299d9cc02d350
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17206
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2023-05-09 17:58:11 +08:00