Commit Graph

18835 Commits

Author SHA1 Message Date
Kozlowski Mateusz
c8ab874d7c ftl: Add upgrade of superblock from version 2 to version 3
Layout of metadata will be part of the superblock at the end of the upgrade.

Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Change-Id: If888866806e948ee07f0777612da73ab8b7548b1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13385
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-09-20 19:24:26 +00:00
Kozlowski Mateusz
7ff285193f ftl: Add metadata upgrade framework
Added the ability for minor metadata upgrade - updating the internal
fields of metadata structures, without changing the overall layout.

Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Change-Id: Iec98c62b45b099d6d476d486ba7e4ff6b648bb95
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13384
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-09-20 19:24:26 +00:00
Artur Paszkiewicz
44b6d585ca FTL: Add helper functions for superblock upgrade
Adds extra functions which will be used during upgrade (changing
versions) of superblock metadata.

Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Change-Id: I08642deaf509f613cc8b22043dcdded6c329daa9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13383
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-09-20 19:24:26 +00:00
Kozlowski Mateusz
1bc356bb21 ftl: Fix abort in compaction retry path
Don't try to abort when return code is actually 0.

Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Change-Id: Id93a43173ae54324dc61ba419d929fdec4d90264
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14449
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-09-20 19:24:26 +00:00
Ben Walker
75aded8439 spdk_dd: Use FIXED_BUFFERS
This improves performance for io_uring

Change-Id: Ifff7f87e76f7ce3074c25d409dfd2b171c800e19
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12514
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-09-20 16:57:47 +00:00
Ben Walker
c284fa9ec4 spdk_dd: Use FIXED_FILES
This reduces overhead from fget/fput on each operation.

Change-Id: I43ddfd6e4ec9ed210206a5378eff9e7ab2f64f82
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12513
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-09-20 16:57:47 +00:00
Ben Walker
84cec154e8 spdk_dd: Allow io_uring to use O_NONBLOCK
This works fine on recent Linux kernels.

Change-Id: If3bf7feb88666d5c3e4b2f3e5071d6be6937fdf8
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12515
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-09-20 16:57:47 +00:00
Ben Walker
cec27746ae spdk_dd: Report speed correctly
There were several places where the integers were likely to overflow,
plus we can avoid checking the timestamp every time because the poller
already fires on a fixed interval.

Change-Id: I7ec8f40f13b6c7a9c0f3f2de6d0aa23e9479c100
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12574
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-09-20 16:57:47 +00:00
Ben Walker
128083ec42 spdk_dd: If both input and output are block devices, use
IORING_SETUP_IOPOLL

Change-Id: Ie1ab8ab896ce0d2e41290645bf3ee275e1a5c578
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12022
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-09-20 16:57:47 +00:00
Michal Berger
a342847df7 pkgdep: Add clang packages to rhel-based distros
Currently these packages are installed through (sometimes weak)
dependencies. Install them explicitly instead.

Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I5d8f7c92be4a74028ce028eef33ec4ac6d2ad792
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14447
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Kamil Godzwon <kamilx.godzwon@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2022-09-20 16:00:28 +00:00
wanghailiangx
c43a50ea00 test/nvme: Counting io_submitted after starting I/O success
Change-Id: I1e79c40d25dd67c84f583b8aab362e0eb65f522e
Signed-off-by: wanghailiangx <hailiangx.e.wang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14536
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-09-20 10:19:39 +00:00
Jim Harris
b313652b30 env_dpdk/pci: Refactor PCI bus scan
Preparing for potential 22.11 changes, refactor this code using DPDK api:
- a bus device list can be walked through via RTE_DEV_FOREACH,
- a reference to the bus object is directly available under the device,

Signed-off-by: David Marchand <david.marchand@redhat.com>
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Id3a21a6e62dfa1619a92465fac5a82afb9b43cb0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14532
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2022-09-20 10:19:22 +00:00
Jim Harris
36644ef32f env_dpdk: move spdk_pci_driver definition to pci.c
Also remove all pci-related DPDK includes from
env_internal.h, and add rte_bus_pci.h to pci.c
only.

Now pci.c has all references to DPDK pci-related
header files and data structures.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I5f1727d465eaa73cf71d2f3589cecd3ebb83eb85
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14531
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-09-20 10:19:22 +00:00
Jim Harris
2bb7185f1b env_dpdk: add dpdk_pci_device_vtophys()
This moves the only references to the rte_pci_device
data structure from memory.c to pci.c.  This helps
prepare SPDK for possible changes to DPDK around
visibility of these DPDK data structures, making it
easier for SPDK to manage if only one file is
affected.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I26b1907fabd7a6c23701523811abd1ce12606683
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14530
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-09-20 10:19:22 +00:00
Jim Harris
92e63a9cc6 env_dpdk: remove unused SPDK_PCI_DRIVER_MAX_NAME_LEN
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I7b6f8d165b56b079fbab0f9dd4a354bf82533d59
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14529
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-09-20 10:19:22 +00:00
paul luse
dd2c08d2d1 configure/misc: make ISA-L a hard dependency
Following discussion in a recent SPDK community meeting,
it was determined that we no longer need to carry ISA-L as
a user configuration option.  It will be enabled by default.

If running on an architecture that ISA-L isn't fully supported
on, the configure script will disable associated features and
display a warning and will also not build ISA-L. Same case if
there are issues with dependencies.

Note that --without-isal is no longer supported as a configure
option.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Ibd1e5e9454d1b090462c3e757b2f51c52e6cb774
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14393
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-09-20 10:18:54 +00:00
KanKuo
b4f9bb444a UT/bdev/part.c:add the test of function spdk_bdev_part_free()
Signed-off-by: KanKuo <kuox.kan@intel.com>
Change-Id: I99bdb67eea08e27276fca4c8659b71ced076573c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14074
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
2022-09-20 10:17:57 +00:00
Jim Harris
18c8b52afa trace: allocate shm filesize based on number of cores used
Previously we would always allocate the shm file based on
max (128) cores which is unnecessary.  So use
spdk_env APIs to only allocate shm file size based
on the cores we might possible use.

With default settings, an shm file was 135MB before this
change, now an app using cores 0-7 will just use
about 9MB.

A lot of the trace-related code depended on there
*always* being a history for every core, even unused
ones, so a few additional changes were needed,
mainly the trace_parser library.

Tested by starting an app using a 0x4 core mask and
enabling a trace mask, generating some events, then
checking both the size of the shm file and that
spdk_trace works properly with the resulting file.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ie868b3e3658d6f82b2fea37cb87453e8a9e0abc4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14044
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
2022-09-20 10:17:45 +00:00
Changpeng Liu
982c25feef nvmf: add spdk_nvmf_ctrlr_[save|restore]_migr_data() APIs
When doing live migration, there are some spdk_nvmf_ctrlr internal
data structures which need to be saved/restored, these data
structures are designed only for vfio-user transport, for
the purpose to extend them to support other vendor
specific transports, here we move them as public APIs,
users can use SAVE|RESTORE to restore a new nvmf controller
based on original one.

And remove the register from vfio-user transport, these registers
are stored in the common nvmf library.

Change-Id: I9f5847ef427f7064f8e16adcc963dc6b4a35f235
Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11059
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Thanos Makatos <thanos.makatos@nutanix.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-09-20 10:17:24 +00:00
Michal Berger
04dd028292 perf/nvmf: Gather power statistics on the target side
Use collect-bmc-pm to gather power usage stats during fio workloads.

Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I20d1f3009c8a32c3927ad2cf2f932a8e97d3e7c5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14164
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-09-20 10:17:04 +00:00
Michal Berger
b43a0112dd scripts/perf: Add script for collecting system power statistics
This script is a wrapper around ipmitool which attempts to extract
power statistics from the BMC. It's mainly based around DCMI spec
to make sure it's common for most of the platforms that we use
in our CI. If given BMC does not conform to DCMI, there's a fallback
in place which attempts to find proper sensor in the SDR which may
report overall power usage of the underlying system.

By default, current ("NOW") reading is extracted in context of DCMI.
Different intervals may be defined, and when matched with supported
averaging time periods, special version of the command will be used
to extract additional avg, min, max information for given time (e.g.
current, max, min, avg readings from last 1m). For the SDR fallback,
only current reading can be extracted.

When terminated, script dumps all readings + their averaged value
into separate .txt files.

Change-Id: Ifc8c1defe862254864d69de7edf5cce635aff258
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14144
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-09-20 10:17:04 +00:00
Jacek Kalwas
b9717f0109 perf: change qdepth from int to uint32_t
this is due to further change and compiler warning related to
comparison of variables with different signedness

Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: Ia1e5acd85fce27fc53f63597b0e44753b86a7801
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14518
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2022-09-20 00:46:38 +00:00
Michal Berger
c738575823 autotest: Move nvmf's fuzzing outside the main suite
This test is static in nature and not dependent on actual nvmf setup
that comes from the CI - everything is driven via a static json config
that comes with the test itself.

Instead, keep fuzzing directly in autotest.sh, under a top dedicated
test flag SPDK_TEST_FUZZER - this should allow users to run only
fuzzing part without touching the majority of the nvmf suite.

Change-Id: I9c33c46e05f7c4b977b041673fe200d9d2989265
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14287
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
2022-09-20 00:46:11 +00:00
Denis Barakthanov
044bb17525 bdev/daos: support custom DAOS installation dir
./configure --with-daos=[DIR] now accepts DAOS installation directory.

Signed-off-by: Denis Barakhtanov <denis.barahtanov@croit.io>
Change-Id: I16f4e0e41e6f73a32dec452c20aa7b558ca50f33
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14522
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-09-20 00:45:40 +00:00
Sebastian Brzezinka
d326389d96 test/sma: use smaller indention for the JSONs
Both tests sma/vhost_blk.sh and sma/vfiouser_qemu.sh
use two spaces indention for JSON

Signed-off-by: Sebastian Brzezinka <sebastian.brzezinka@intel.com>
Change-Id: I59aedabe40852d45a606c82a875a49a182137f91
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14293
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-09-19 19:43:35 +00:00
Sebastian Brzezinka
9f0d4f33c3 test/sma: vhost_blk create cntrl with crypto bdev
Signed-off-by: Sebastian Brzezinka <sebastian.brzezinka@intel.com>
Change-Id: I72c1340d927101d171509b1a42c4f14c0f7dc3e9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13998
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-09-19 19:43:35 +00:00
Sebastian Brzezinka
7cefe7f62d test/sma: vfiouser attach crypto bdev
The test uses bdev_crypto and vfiouser devices to verify that
volumes with crypto parameters are correctly attached

Signed-off-by: Sebastian Brzezinka <sebastian.brzezinka@intel.com>
Change-Id: I9abed43f3df803870af11d7f28601f5f9390474c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13997
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-09-19 19:43:35 +00:00
Sebastian Brzezinka
a70732c9f1 test/sma: move crypto helper function to common.sh
Signed-off-by: Sebastian Brzezinka <sebastian.brzezinka@intel.com>
Change-Id: I55d58a5959d5b67d99c2e2dfd9ea787a1e31b023
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13996
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-09-19 19:43:35 +00:00
Sebastian Brzezinka
aa55c23dbc sma: crypto support for vhost_blk devices
The vhost_blk devices will now support volumes with encryption enabled.
Reorder 'imports' according to PEP 8.

Change-Id: Ia4ae0832f4cf91578153da7ca0a28a83707ccdca
Signed-off-by: Sebastian Brzezinka <sebastian.brzezinka@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13983
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-09-19 19:43:35 +00:00
Sebastian Brzezinka
62e8255460 sma: crypto support for nvmf/vfiouser devices
The nvmf/vfiouser devices will now support attaching volumes with
encryption enabled. Reorder 'imports' according to PEP 8.

Change-Id: I9862c655c2b57b8bb307f6091d4967afc21c11da
Signed-off-by: Sebastian Brzezinka <sebastian.brzezinka@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13982
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-09-19 19:43:35 +00:00
Konrad Sztyber
30ff5c6f14 test/sma: crypto test
The test uses bdev_crypto and NVMe/TCP devices to verify that volumes
with crypto parameters are correctly attached and any errors are handled
as intended.

It starts up two spdk_tgt processes, one is being orchestrated by SMA
and the other one acts as an NVMe-oF target.  This ensures that we can
verify that SMA terminates discovery service connections whenever a
failure occurs during crypto setup.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I74001c256d4a05d2d8d295fb6fd8375d30ab66a1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13873
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-09-19 19:43:35 +00:00
Sebastian Brzezinka
a9d10dade8 test/common: add with-crypto to sma autotests
Change-Id: I25cff079e021418ec66c6d04cfd5a3daf969ed77
Signed-off-by: Sebastian Brzezinka <sebastian.brzezinka@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14154
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-09-19 19:43:35 +00:00
Konrad Sztyber
2bd552fbfe sma: crypto support through bdev_crypto
This patch adds CryptoEngine implementation using bdev_crypto.  Only a
single crypto drvier can be used at a time and it's configured during
startup in a config file, e.g.:

```
crypto:
  name: 'bdev_crypto'
  params:
    driver: 'crypto_aesni_mb'
```

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I13f4fd1227a02cf9f1bee61a1686904b43c0fc55
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13872
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: <sebastian.brzezinka@intel.com>
2022-09-19 19:43:35 +00:00
Konrad Sztyber
c16dab7e37 sma: crypto support for nvmf/tcp devices
The nvmf/tcp devices will now support attaching volumes with encryption
enabled.  This basically boils down to getting the crypto bdev name
through the CryptoEngine.get_crypto_bdev() interface (instead of just
using volume_id) and specyfing UUID/NGUID when attaching namespaces
to a subsystem.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Iefddbf07675152aa2e042564eb87c457b6995b9b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13871
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: <sebastian.brzezinka@intel.com>
2022-09-19 19:43:35 +00:00
Konrad Sztyber
e1068a862c sma: configure crypto when attaching volumes
Crypto parameters are now checked and crypto is configured when a volume
is attached.  Since configuring crypto can lead to creating new bdevs on
top of the attached volume, each device manager will also need to be
changed to retrieve the bdev through the CryptoEngine.get_crypto_bdev()
interface.  This will be done in subsequent patches.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Idb5f804cf88aa5e34bbee6817acdb8f3a42a2320
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13870
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: <sebastian.brzezinka@intel.com>
2022-09-19 19:43:35 +00:00
Konrad Sztyber
cc3f842cd1 sma: initial crypto definitions
This patch defines the interface for crypto engines, which provide
support for configuring crypto on a given volume.  Only a single crypto
engine can be active at a time and it's selected in the "crypto" section
of the config file.  Similarly to device managers, external crypto
engines can be loaded from plugins.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Id942ef876e070816827d7ad1937eb510a85c8f8d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13869
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: <sebastian.brzezinka@intel.com>
2022-09-19 19:43:35 +00:00
Konrad Sztyber
b30edf643d sma: volume crypto interface
This interface will make it possible for users to configure data at rest
encryption for volumes exposed through SMA.  The crypto parameters are
part of volume parameters specified when a volume is attached.  Once a
volume is attached it is not possible to change it.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I795cfab67523ace47d366d50e33b253737716668
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13868
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-09-19 19:43:35 +00:00
Michal Berger
11d13b85e6 test/nvmf: Test different abort qd sizes under SPDK and kernel targets
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I79ede7670477ed0d8eb38fa3eced4a9b70c64620
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8962
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-09-19 13:13:42 +00:00
Tomasz Zawadzki
dcd1fb796c example/perf: move allocating ns_worker
Next patch in series will add another caller for
the allocate_ns_worker.

Change-Id: I60281b519265b27d661c2b1ca94e09dc68dbbf57
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14414
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2022-09-19 13:13:09 +00:00
Tomasz Zawadzki
b6e9e1abb4 example/perf: comment the ns to worker association
This documents the logic below and motivation behind
(70db0e1)nvme/perf: Add support for multiple cores per device.

Change-Id: I949b794355f871c0f995b946622dbe77295d922b
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14413
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2022-09-19 13:13:09 +00:00
Liu Xiaodong
762db2a4f4 vhost: register memtable once if unchanged
Move memtable register out of start_device, into
post_handler for vhost-msg SET_MEMTABLE;
And unregister memtable in destroy_connection
instead of destroy_device

If memtable info not changed in the msg, then we
don't need to register it multi times.

Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Change-Id: I0f8c76c1ee43b6f981d703beeba92da5dac4dbd6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14263
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2022-09-19 13:12:24 +00:00
Tomasz Zawadzki
69bcb185f3 bdev/xnvme: save xnvme bdevs from runtime
config_json callback is used to preserve all
bdevs that were created during application runtime.

xnvme bdev module was missing this callback.

While here, io_mechanism is now saved in the bdev_xnvme
structure for reference in the config_json.
Haven't seen an option to pull this from xnvme itself.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I1a88ad2bb761f589d214fec8f0690c38572824d6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14116
Reviewed-by: Michal Berger <michal.berger@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-09-19 13:12:07 +00:00
wanghailiangx
69b4e8b17f test/nvme: Replace -s with -o for "io size in bytes"
For most of apps, -o is for "io size in bytes",
-s("io size in bytes") was changed to -o a long time ago.
-s is now for "memory size in MB for DPDK".

Change-Id: I0977d4a7bc30a4f2e135707ff6e77b6254240fb9
Signed-off-by: wanghailiangx <hailiangx.e.wang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14533
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-09-19 13:11:55 +00:00
Krzysztof Karas
905e519adc autotest_common.sh: increase read timeout inside rpc_cmd
Currently we use 5 seconds as a timeout inside rpc_cmd's
"while read" loop, which is enough for a system with
a single NVMe. That value is insufficient for machines
with more devices and causes the timeout, erroring out
the tests (verified with the same number of devices,
as in the CI node with the most NVMes available - 4).

For that reason increase the read timeout to 15 seconds.

Fixes #2641

Change-Id: If7e9fd169a75ea16ec243c63821a4e5d70995063
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14525
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-09-19 13:11:30 +00:00
Jim Harris
88055b422f test/spdkcli: reduce amount of HUGEMEM memory requested
We do not need to force usage of 4096MB of memory for these
tests.  Let's just remove how that is being forced.

This allows us to also remove the setup.sh call from
the spdkcli test common.sh script.  Its only purpose
was to increase the amount of memory allocated to satisfy
the spdkcli tests, but now that the spdkcli tests aren't
forcing so much memory to be allocated, we can remove
that too.

Fixes issue #2694.

Note: these limits were in the original commit for the
test scripts - 7e8206c - but it is not clear why they
were added instead of just relying on DPDK dynamic
memory allocation..

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I2210765e69117205a039aa6bc468efa5d5d537cb
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14520
Reviewed-by: Michal Berger <michal.berger@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-09-19 13:11:06 +00:00
Changpeng Liu
f167986b6c test/vfio-user: skip NVMeoF target coremask for VMs
For vfio-user tests, SPDK NVMeoF target uses "0xF"
as the core mask, so we should avoid to use these
cores when starting VMs.

Fix issue #2600.

Change-Id: I1927cd70670b30d59944d5571bcfc37c9e7e35d6
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14485
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-09-19 12:47:37 +00:00
Konrad Sztyber
35a71c7748 sock/ssl: free SSL resources in case of failures
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ifba581c92269c675b03b67586b1fea0b096f9e95
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14511
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-09-16 13:33:17 +00:00
jun.ran
488d6e84c4 bdev_aio: handle unexpected res=0 correctly
If res = 0, do not pass this value directly to bdev io completion function,
otherwise it will be treated as a successful completion.
Fix issue: 2679

Signed-off-by: jun.ran <ranjunsh@163.com>
Change-Id: Ice525de1bb5b0ada9ac30d3b59e5c731419efe2f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14364
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-09-16 13:32:40 +00:00
Xinrui Mao
c3f628f141 lib/nbd:export bdev flush and trim ability
Fix mkfs fail when using lvol as backend of nbd.Predefined
NBD_FLAG_SEND_FLUSH and NBD_FLAG_SEND_TRIM are defined by default,
so the operations of trim and flush are supported,but in fact lvol
doesn't support trim and flush operations.Therefore add judgement for
NBD_FLAG_SEND_FLUSH and NBD_FLAG_SEND_TRIM to check.

Signed-off-by: Xinrui Mao <xinrui.mao@intel.com>
Change-Id: I3d21034d12a038c8fc694d3383028103239ea6bd
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14099
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2022-09-16 13:32:13 +00:00
Michal Berger
7a4cb7bc1c perf/vhost: Allow to pass multiple FIO configs to the benchmark script
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I96df5f21f04b9196083f91783f8dd9cc4c6920d1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13688
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-09-15 20:15:57 +00:00