Commit Graph

19674 Commits

Author SHA1 Message Date
Jim Harris
90656238b4 test/unit: disable sock unit tests on FreeBSD
There are several failing signatures observed as
part of issue #2943.  So disable the unit tests for
now until they are debugged.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iae54f8bfcd7883c02152abee37410a998da81dd7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17573
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2023-05-09 17:58:11 +08:00
Ben Walker
7ba7715530 idxd: In perf tool, correctly pass fill pattern as a uint64_t
The pattern is 64 bits but we were only passing in 8.

Fixes #2821

Change-Id: I4a4c3f7c18bcb610df9c37edee549255f93f2632
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17686
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2023-05-09 17:58:11 +08:00
Sebastian Brzezinka
3a14cb3a0f lib/env_ocf: place allocator variable on hugepages
When using `__lsan_do_recoverable_leak_check` (e.g when fuzzing),
to check for leaks during runtime. Leak sanitizer can not follow
reference of memory that is allocated on heap (e.g. calloc)
and then stored on hugepage causing lsan to incorrectly report
direct leak.

Fixes #2967

Signed-off-by: Sebastian Brzezinka <sebastian.brzezinka@intel.com>
Change-Id: I3511e117a07ca8daa96f19bf1437c0d788b64cb1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17682
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Amir Haroush <amir.haroush@huawei.com>
2023-05-09 17:58:11 +08:00
Shuhei Matsumoto
5200552452 bdev/nvme: Add max_bdevs parameter for attach_controller RPC
The target subsystem may expose more than 128 namespaces. To support
such subsystem, add a new parameter max_bdevs for the
bdev_nvme_attach_controller RPC.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I8fab20b9c4d52818205e05de6a31dbe0d31a10fe
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17651
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2023-05-09 17:58:11 +08:00
Shuhei Matsumoto
7bb78b8de9 bdev/nvme: Alloc bdev name array dynamically for attach_controller RPC
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I9c1822421563210f6a656553355e29e75c8b0c21
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17650
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2023-05-09 17:58:11 +08:00
Shuhei Matsumoto
e66e194f1f bdev/nvme: Aggregate req and ctx free for attach_controller RPC
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: Iba2091f67a97a59ecad7f0c853491d9cfcad736d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17649
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2023-05-09 17:58:11 +08:00
Jim Harris
f353506f1c examples/nvme/perf: increase opts.num_io_queues when needed
By default we specify 1024 max_io_queues per controller.
But it's possible we need more for high connection count
use cases (i.e. -c 0xFF -P 512 which is 8 * 512 = 4096).
So dynamically configure opts.num_io_queues based on
the corresponding values.

Note: we have to change a couple of globals from int to
uint32_t to avoid signed v. unsigned comparison warnings.
Let's just do that in this patch instead of a separate
one.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iba2d670c224a91e50377e622b154ce43eed94002
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17621
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2023-05-09 17:58:11 +08:00
Jim Harris
a8d86cb313 examples/nvme/perf: pick num_requests based on qpairs per ns
If we want to test something like 512 qpairs, with qd = 8 for
each, you need to specify -q 4096 -P 512.  Then those 4096
I/O are spread across the 512 qpairs, to get qd = 8
for each qpair..

But currently it ends up also allocating 4096 num_io_requests
for each qpair which is a huge waste.  We need to instead
base the num_io_requests on the effective queue depth for
each of the qpairs.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I3ec0f4d9ab94388bf980c0b0439790847161ec12
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17620
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
2023-05-09 17:58:11 +08:00
Jim Harris
168f31f5b0 nvme/tcp: increase timeout for async icreq response
This was arbitrarily picked as 2 seconds in commit
0e3dbd. But for extremely high connection count
use cases, such as nvme-perf with several cores
and high connection count per core, this 2 second
time window can get exceeded.

So increase this to 10 seconds, but only for qpairs
that are being connected asynchronously.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I906ca9e6561b778613c80b739a20bd72c807216c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17619
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2023-05-09 17:58:11 +08:00
Jim Harris
746dbe8633 nvme: fix async_mode comment
async_mode is now supported on PCIe, RDMA and TCP
transports.  So remove the comment about it only
being supported on PCIe transport.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I059e226aa98e702c9caa2886a10ec1212b6f1ada
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17577
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2023-05-09 17:58:11 +08:00
Marcin Spiewak
0abd682a56 ./configure: add 'detect' value to --max-lcores
This patch adds suport for 'detect' option in SPDK's
./configure, allowing configuring of DPDK to detect
current number of cores during SPDK compilation.
This is done by providing --max-lcores=detect as
a parameter to ./configure, which triggers setting
of '-Dmax_lcores=detect' in DPDK_OPTS passed to
dpdkbuild/Makefile.
DPDK then do detection of number of cores in the
system during compilation, and sets RTE_MAX_LCORE
to that value. Meson build system also generates
a message displaying information about number of
cores detected. E.g. for my system:
"
Message: Found 72 cores
"

Example usages:
1) use default value for RTE_MAX_LCORE:
	./configure
2) detect the core number:
	./configure --max-lcores=detect
3) Set RTE_MAX_LCORE to 256:
	./configure --max-lcores=256

Change-Id: I2103c2d917f210aee4d1ef43584b1bd40dbfe43b
Signed-off-by: Marcin Spiewak <marcin.spiewak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17555
Community-CI: Mellanox Build Bot
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2023-05-09 17:58:11 +08:00
Michal Berger
0a845fec27 test/vhost: Create wrapper around wipefs
Call sync each time, as an extra step, to make sure all the writes on the underlying device completed. This is needed, as on occasion parted (called right after wipefs) fails to create pt complaining that the target device (and its partitions) are still in use.

Change-Id: I959d9b36a1588ec3754335995e3e8bc5057bfeb7
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17498
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2023-05-09 17:58:11 +08:00
Marcin Spiewak
5ea00c32d6 lvol_ut: add test for invalid options
Add unit test for calling spdk_lvs_load_ext()/lvs_load()
with invalid options (opts_size is 0).

Change-Id: I9c48b972066cf977304e3efa936827d1ef1b5250
Signed-off-by: Marcin Spiewak <marcin.spiewak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17584
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Mike Gerdts <mgerdts@nvidia.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2023-05-09 17:58:11 +08:00
Marcin Spiewak
97a02a9878 lib/lvol: lvs_load() shall return, if options are invalid
lvs_load() function verifies if options passed to it
are valid, but doesn't return, if they are not (only error
is logged and callback is called with -EINVAL code). Now
it is corrected and the function ends after the error
is reported.

Change-Id: I19b0b22466b6980345477f62084d27ef13414752
Signed-off-by: Marcin Spiewak <marcin.spiewak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17582
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Mike Gerdts <mgerdts@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2023-05-09 17:58:11 +08:00
Konrad Sztyber
1ca880e6e9 accel: move merging dst buffer to a function
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I62b73f1802a9de35767b72c2cc4ee115e895c538
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17426
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2023-05-09 17:58:11 +08:00
Konrad Sztyber
35b1d8c7da accel: copy memory domain context when merging tasks
When changing src/dst buffers, we copied memory domain pointers, but we
didn't copy memory domain context, which is obviously incorrect.  It was
probably missed, because we never append a copy with non-NULL memory
domain.  Added a unit test case to verify this behavior.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ic174e0e72c33d3f437f0faddd3405638049f0c74
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17425
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2023-05-09 17:58:11 +08:00
Konrad Sztyber
5147911114 accel: move accel_module.h to include/spdk
This file should be external to enable out-of-tree accel modules.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I2e973d0e88d7145d0fc9714f56db48486b00f3b7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17419
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2023-05-09 17:58:11 +08:00
Konrad Sztyber
072703d510 bdev/crypto: complete IOs on ENOMEM from accel
spdk_bdev_queue_io_wait() can only be used when one of bdev submission
functions returns ENOMEM (i.e. there are no more spdk_bdev_ios on that
IO channel).  Using it in any other case, e.g. on spdk_accel_append_*()
returning ENOMEM, will most likely result in failure.  Therefore, to
avoid that, the IOs are completed with NOMEM status relying on the bdev
layer to retry them.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ie0f03496e5d3180c481815b3f1b021e74ae2f46d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17319
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2023-05-09 17:58:11 +08:00
Konrad Sztyber
1ac6963542 accel: make number of tasks/seqs/bufs configurable
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I07ebf37ff31ddb888e68e98cf7b9b425c7a4d128
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17318
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2023-05-09 17:58:11 +08:00
Konrad Sztyber
c812fe0948 bdev/malloc: report accel sequence support
This actually allows malloc bdev to chain multiple accel operations
together.  And, since the last operation will always be a copy, accel
should remove that copy by modifying previous operation's dst/src.

On my system, it improved bdevperf performance (single core, qd=4,
bs=128k, bdev_crypto on top of bdev_malloc, crypto_sw):

randread: 5668M/s -> 8201M/s
randwrite: 5148M/s -> 7856M/s

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I5b9173fa70a42ee56f56c496a34037d46d2f420f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17202
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2023-05-09 17:58:11 +08:00
Konrad Sztyber
23844411ff bdev/malloc: report memory domain support
Because the copying is handled by accel, which will do push/pull when
necessary, we can report support for each registered memory domain.

Also, since verifying PI information would require doing a push/pull, we
don't report support for memory domains if bdev has DIF enabled.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Id80f82aaac68e9dec2a6cae81d96a460105161d6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17201
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2023-05-09 17:58:11 +08:00
Konrad Sztyber
16cfc9a656 bdev/malloc: use appends for write requests
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ief6c873a5f65274a25b67bc3f2811d8f3e4a33b3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17200
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2023-05-09 17:58:11 +08:00
Konrad Sztyber
2133c01f21 bdev/malloc: pass bdev_io to bdev_malloc_writev()
Same reason for the change as in bdev_malloc_readv().

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Id52d8639df6a488342346283c90f12a2ba6f5736
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17199
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2023-05-09 17:58:11 +08:00
Konrad Sztyber
c66b6c283c bdev/malloc: use appends for read requests
This only changes the interface bdev_malloc uses for scheduling the copy
to appends, but it won't chain those copies to an existing sequence, as
bdev_malloc doesn't report support for accel sequences yet.  That will
be changed in one of the subsequent patches.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I6db2c79b15cb96a1b07c6cf5514004c76b9d2e92
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17198
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2023-05-09 17:58:11 +08:00
Konrad Sztyber
45d129191d bdev/malloc: pass bdev_io to bdev_malloc_readv()
It reduces the size of the parameter list, which was already pretty
long, and will make it easier to use other bdev_io's fields (e.g. memory
domain, accel sequence).

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I43a9d3a7cbb77915c00879c43540c9ec725c52d2
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17197
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2023-05-09 17:58:11 +08:00
Konrad Sztyber
a2a8f19ce2 bdev/malloc: don't retry failed requests
If a request was marked as failed, we don't want to retry it, so we
shouldn't override its status with NOMEM.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I85a522a7934d2d6f415620b9a323effefb91f299
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17196
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2023-05-09 17:58:11 +08:00
Konrad Sztyber
177a74ee0c bdev/malloc: declare malloc task/disk variables
It gets rid of lots of casts to malloc_task/malloc_disk and makes the
code more readable.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Id50f0cbfa18adf5e7baafd58da03d290d6ba62c6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17195
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2023-05-09 17:58:11 +08:00
Konrad Sztyber
81b5b941e9 accel: make spdk_accel_sequence_finish() void
It always returns 0 and any errors are reported in the callback.  Making
it void simplifies error handling.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I0d4299a2789a688eae38d76de46d1baf27cbbd8f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17194
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
2023-05-09 17:58:11 +08:00
Konrad Sztyber
967937e213 test/bdev: accel chaining test
This test sends several read/write requests and verifies the expected
number of accel operations have been executed.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Idda46ef00dc5bcc0a176d3dfb39f3f3861964741
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17193
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2023-05-09 17:58:11 +08:00
Michal Berger
f28978a106 test/packaging: Export LD_LIBRARY_PATH, PKG_CONFIG_PATH setup
095f40630e missed the autobuild dependencies while enabling the rpm
test against the external DPDK build. Without it, DPDK is not able
to properly configure itself against ipsec and isa-l libs.

Change-Id: Ia4307f0d0f9c1f82f6f80ca06113a5289c2916ed
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17576
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Kamil Godzwon <kamilx.godzwon@intel.com>
2023-05-09 17:58:11 +08:00
Michal Berger
5ff8838dda test/spdkcli: Include errors from a failed command
Dump it to stdin to make the debugging easier.

Change-Id: I9b13d0a77e45aa84ec2a55b7b982225592f2566d
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17560
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2023-05-09 17:58:11 +08:00
Michal Berger
f714562f5a test/spdkcli: Wait long enough for the nvme ctrl to be gone
Some nvmes need more time to attach|detach to|from, hence having a
static sleep is not ideal depending on what type of the nvme was
picked up for the test. Instead, simply wait until the list of
nvme ctrls is empty after the cleanup.

Change-Id: I2fc2630020436d0e1f6b01a5ce60aea56e7bf8ec
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17559
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2023-05-09 17:58:11 +08:00
Jim Harris
20e5d5388e test: wait_for_examine and delay in reap_unregistered_poller.sh
We need to give the thread library some time to reap
the unregistered poller - it is a bit of a delayed
process.  We have to wait for examine to finish on
the aio bdev, then the poll group gets destroyed and
the pollers unregistered.  This marks the pollers as
UNREGISTERED, but they aren't actually reaped until
next time the thread is polled.

Fixes issue #2980.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I1e32c50ea9b28ea2d5560ddc9b2f68fa81e708d9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17575
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Mike Gerdts <mgerdts@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2023-05-09 17:58:11 +08:00
Artur Paszkiewicz
5a952042f2 module/raid: specify memory domain support per raid module
Not all raid modules may support memory domains - raid5f currently does
not. Add a parameter to struct raid_bdev_module to specify that.

Change-Id: I3285c118db846d290837606b3f85ac4b5277de97
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17601
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2023-05-09 17:58:11 +08:00
Marcin Spiewak
7e14ad86b8 libreduce: removing deprecation messages for pmem
Deprecation notice for pmem was removed, as libreduce will
still use it until pmem is supported.

Change-Id: I7555dbf20a408a67fac8a6e7b2eaa23edf985eec
Signed-off-by: Marcin Spiewak <marcin.spiewak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17538
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
2023-05-09 17:58:11 +08:00
Konrad Sztyber
c72148f1a2 accel: keep track of destroyed channels' statistics
To make sure we don't lose statistics of destroyed channels, they're now
added to a global stats structure when a channel is destroyed.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ic3b4d285b83267ac06fad1e83721c1b15cc8ec8a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17567
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
2023-05-09 17:58:11 +08:00
Konrad Sztyber
0f81ba102b accel: add accel_get_stats
The RPC allows the user to retrieve accel framework's statistics.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I5cd1b45686504c08eda50513ad1dae2f8d65013b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17191
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2023-05-09 17:58:11 +08:00
Konrad Sztyber
319dedcb31 accel: collect statistics
This patch adds support for collecting statistics in accel framework.
Currently, it counts the following events:
 1. The number and the type of executed/failed operations.
 2. The number of executed/failed accel sequences.

For now, these statistics are only collected and there's no way of
retrieving (or resetting) them - that will be added in the following
patches.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Id211067eb810e7b7d30c756a01b35eb5019c57e7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17190
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2023-05-09 17:58:11 +08:00
Konrad Sztyber
9150cac94e accel: extract submitting task to a function
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I7d24ab571fb3217917aee53276ccd3d13e1e76c4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17189
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2023-05-09 17:58:11 +08:00
Konrad Sztyber
35993ac9de accel: add accel_set_options
It'll allow for setting accel-specific options.  For now, it makes the
size of iobuf caches configurable.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Iaf505cc5e98dc6411453d9964250a4ba22267d79
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17188
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2023-05-09 17:58:11 +08:00
Michal Berger
6b7ec13c8f scripts/qat_setup: Add support for dh895xcc devices
These can be found under CYP platform.

To that end, refactor qat_setup.sh so it can support devices based on
their dedicated driver rather than the specific device ID - this will
allow for easier addition of new devices in the future.

Also, configure number of VFs based on total number given ctrl
supports - this is exactly what qat_service is doing while enabling
VFs.

Drop warning about old bug in the uio_pci_generic driver - in
practice we don't hit that under CI since quite a long time.

Slap errexit on top to make sure we exit immediately when writing
to sysfs fails.

Last but not least, run blockdev_crypto_qat test without any top
condition - if qat_setup.sh succeeds, the test should be able to pass.

Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I37c4fd319ad7002017f9baf9fdcf3890429aac75
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17086
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
2023-05-09 17:58:11 +08:00
Michal Berger
8d3118f156 test/check_so_deps: Adjust printout when $SPDK_ABI_DIR is not set
There's no default defined so it doesn't have to be available in the
env at all. Adjust the echo so we don't include an empty string.

Change-Id: Icaa75915544f9da1adcdcdeafce29f5ae97149ab
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17428
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2023-05-09 17:58:11 +08:00
Michal Berger
fa633b2825 autopackage: Move packaging test to autobuild
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: Ifbe4d98f3d7a4b9970f923acd6d299d9cc02d350
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17206
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2023-05-09 17:58:11 +08:00
Michal Berger
7cdf6b7377 test/packaging: Move tests out of nightly
Packaging tests will be done under a separate docker job, hence there
will be plenty of time to run them together. Keep DPDK-related builds
in nightly as they are quite sensitive to any changes (especially API
related), hence not very fit for per-patch testing.

Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: Ia1af5b0e86a503f540c32d2e030088d8a24f8847
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16046
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2023-05-09 17:58:11 +08:00
Michal Berger
2099abc819 test/autobuild: Source $spdk_conf before autotest_common.sh
This is to make sure we export all SPDK_* with proper values.

Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I2f01af1a051edcec6a75f99b25b765080abf2a5d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17212
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2023-05-09 17:58:11 +08:00
Michal Berger
3a060b6cd5 lib/env_dpdk: Make sure linker finds $DPDK_LIB_DIR
In case SPDK is build with shared libraries and there's no
LD_LIBRARY_PATH around, linker will complain about missing .sos
similar to:

/usr/bin/ld.bfd: warning: librte_meter.so.23, needed by
/root/spdk/dpdk/build/lib/librte_ethdev.so, not found (try using -rpath
or -rpath-link)

We can't see that under CI since autotest_common.sh always makes sure
the LD_LIBRARY_PATH is properly set.

Add the -rpath to make the build less spammy.

Change-Id: I1d9d1775b2aa24e65cc4b776c2549457b0d7aac3
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17492
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2023-05-09 17:58:11 +08:00
Michal Berger
2994a06067 mk/spdk.common: Use -flto=auto for the LTO builds
This tells lto-wrapper to either use make's jobserver or fallback to
auto guessing number of cpu threads used for the build. Mainly, this
should silence the following warning:

lto-wrapper: warning: using serial compilation of N LTRANS jobs
lto-wrapper: note: see the ‘-flto’ option documentation for more
information

Change-Id: Ib848319c858f4371b94f9264d22449535d25d6da
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17491
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2023-05-09 17:58:11 +08:00
Michal Berger
6585c782e5 test/common: Silence vhost/commo.sh during cleanup
It's too verbose and may send confusing (in context of the actual
cleanup) messages.

Change-Id: I9e86e20afcf567fb54fec3a6cfb9008ad2080a12
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17485
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2023-05-09 17:58:11 +08:00
Changpeng Liu
f3d3338b29 lib/vhost: register VQ interrupt handler when enable VQ
In commit 23baa67, we will start virtio device only once,
and update the VQ's information in SET_VRING_KICK message
context, so when multi-queues are enabled, SPDK doesn't
register VQ's interrupt handler, here we add it when enable
VQ.

Fix issue #2940.

Change-Id: I29dbd7bf0b81b23c2e47e37c467952cc5887b5bf
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17354
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2023-05-09 17:58:11 +08:00
Jim Harris
7062988c66 bdev: assert that internal status is PENDING for completed IO
bdev modules should have call spdk_bdev_io_complete twice
for the same IO.  We can help find cases where this happens
by adding an assert in spdk_bdev_io_complete - confirming
that the current status is still PENDING, before changing
it to the status passed by the caller.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Id8a044a94113f1ac5e3c8d86e426654bfa8d5c5a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17330
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
2023-05-09 17:58:11 +08:00