Commit Graph

19056 Commits

Author SHA1 Message Date
Michal Berger
367fc7a9cf test/bdev: Refactor nbd_function_test()
This function was a bit cluttered so simplify it. Also, it was
assuming nbd module was loaded into the kernel prior running it -
this could silently fail the nbd_all[@] setup in case it wasn't.
Always attempt to load nbd driver before the setup and fail hard
if the driver is not in place.

Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I5884973944eae3e827eaafec16ba72e2cb4f70e7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14994
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-10-27 09:06:37 +00:00
Michal Berger
f17fe62924 test/xnvme: Use null_blk device instead of zram for the backend device
zram doesn't support some of the uring-specific setup that xnvme may
be using, in particular the IORING_SETUP_IOPOLL, so switch to something
more common like null_blk module. For details see:

https://github.com/spdk/spdk/issues/2708

Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I48b91ba356b4054433eb1835fa3e2708c8d2628c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14920
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Kamil Godzwon <kamilx.godzwon@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-10-27 09:06:37 +00:00
Szulik, Maciej
51ae6d4002 nvme/tcp: add max_completion exit condition to loop inside read_pdu
A loop inside 'nvme_tcp_qpair_process_completions' makes
'max_completions' actually behaving like a minimum:
	do {
		rc = nvme_tcp_read_pdu(tqpair, &reaped);
		[...]
	} while (reaped < max_completions);

Before this change 'max_completion' constraint, in its true sense,
was actually not respected and a loop inside 'nvme_tcp_read_pdu'
could be executed indefinitely as long as a recv state changed.

To prevent this behavior, max_completion must be passed to
'nvme_tcp_read_pdu' and used as an additional exit condition.

Signed-off-by: Szulik, Maciej <maciej.szulik@intel.com>
Change-Id: I28da962f4a62f08ddb51915b5d0dae9611a82dee
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15136
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2022-10-26 07:35:21 +00:00
paul luse
497132086b github config: add dependabot.yml
Enables DependaBot scanning on the spdk repo.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I0db032c547d28fae8e6e09a0b3179c3680cf8db2
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15113
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-10-26 07:34:10 +00:00
Richael Zhuang
db8083d908 bdevperf: reformat bdevperf output
Reformat the output of bdevperf and add header to explain unit of
values. With this change, the output example is:

 Device Information          : runtime(s)     IOPS   MiB/s   Fail/s  TO/s
 Job: NVMe0n1 (Core Mask 0x1)
         NVMe0n1             :      10.00 17444.35   68.14     3.00  0.00
 Job: NVMe0n1 (Core Mask 0x2)
         NVMe0n1             :      10.00 15580.05   60.86     3.00  0.00
 =======================================================
 Total                       :            33024.40  129.00     6.00  0.00

Change-Id: I82490d86a2a41fb4c3cbd8beb02a41ccd1b5d55f
Signed-off-by: Richael Zhuang <richael.zhuang@arm.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14917
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-10-26 07:34:03 +00:00
John Levon
36dfcca2b4 nvmf/vfio-user: switch from shadow doorbells when freeing
Some reset/disable paths are freeing the shadow doorbells without
switching the SQs back to BAR0. Fix this up, and add a small cleanup
when initializing the shadow doorbells.

Signed-off-by: John Levon <john.levon@nutanix.com>
Change-Id: Ia5e5b91b7dc696a558eb0ad59cc554abced47cca
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14901
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-10-26 07:32:54 +00:00
John Levon
64db53f1aa nvmf/vfio-user: support multiple poll groups in interrupt mode
To support SQs allocated to a poll group other than the controller's
main poll group, we need to make sure to poll those SQs when we wake up
and handle the controller interrupt. As they will be running in a
separate SPDK thread, we will arrange for all poll groups to wake up
when we receive an interrupt corresponding to a vfio-user message
arriving.

This can mean needless wakeups: we don't (yet) have a mechanism to only
wake up the poll groups that correspond to a particular SQ write.

Additionally, as we don't have any notion of a poll group per
controller, this ends up polling all SQs in the entire poll group, not
just the ones corresponding to the controller we were handling.

As this has potential performance issues in many cases, it defaults to
disabled.

Signed-off-by: John Levon <john.levon@nutanix.com>
Change-Id: I3d9f32625529455f8d55578ae9cd7b84265f67ab
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14120
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-10-26 07:32:54 +00:00
KanKuo
2fb9cd314f UT/nvme_cuse: test for spdk_nvme_cuse_get_ctrlr_name
Signed-off-by: KanKuo <kuox.kan@intel.com>
Change-Id: I425d108d0c1bed6d8302e1fb47a785ddf85e3095
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10432
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Community-CI: Mellanox Build Bot
2022-10-26 07:28:37 +00:00
Jim Harris
6202973033 bdev_nvme: change discovery_poller check to assert
spdk_nvme_probe_poll_async() can only return 0 or -EAGAIN.
But the code currently indicates that other values might be
possible - so change the code to remove that confusion.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I8e7b2e1183f6650043dd751d610d434d626efa7e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15110
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2022-10-25 07:13:08 +00:00
liu.darong
7e17de3d81 bdev/trace: add support to trace with bdev name
Fixes #2585

Signed-off-by: liu.darong <liu.darong@xsky.com>
Change-Id: I3f9b6d4719b5eed004f383e86db8a17b8b0287f5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13823
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-10-25 07:12:52 +00:00
Richael Zhuang
dbd7999f5e bdev/nvme: rename functions and add more logs
Change 'bdev_nvme_compare_trids' to name 'bdev_nvme_check_secondary_trid'
and 'bdev_nvme_compare_namespaces' to 'bdev_nvme_check_secondary_namespace'
which better explain the aim of these function.

RPC bdev_nvme_attach_controller only response "invalid parameter"
if the path is not sucessfully added due to not meeting the conditions
in bdev_nvme_compare_trids. Add more log.

Change-Id: If9ba6ec1d397d49689aa2ed6ab74fbfa96e16afa
Signed-off-by: Richael Zhuang <richael.zhuang@arm.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14251
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2022-10-25 07:12:24 +00:00
Denis Barakhtanov
3d7851e011 bdev/daos: Add object class parameter
A new `oclass` parameter allow to specify DAOS DFS object class that is
responsible for data redundancy and protection.

Examples of the object classes could be found here: https://github.com/daos-stack/daos/blob/master/src/include/daos_obj_class.h

The default value is OC_SX for the max IOPS.

Signed-off-by: Denis Barakhtanov <denis.barahtanov@croit.io>
Change-Id: Ia48681832458c2266eb7c3bcae0df2055e59e309
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15006
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2022-10-25 07:12:06 +00:00
GangCao
03b1b86078 test/bdevperf: set a random seed for rand rw testing
To fix issue: #2726

And also fix the examples/nvme/perf tool.

The srand() only needs to be called once to set the seed of
futher more calls of rand().

Change-Id: I41ab3a46593513516ad11ea7a5b8960b449e9867
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15108
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-10-25 07:11:30 +00:00
Anton
7ba33f49f0 lib/idxd: fix use after free due to stale crc_dst in chained ops
When crc32c is invoked with a multiple entry input iov,
only the last op has crc_dst set in order to write the final
crc value into the user supplied location.

spdk_idxd_process_events() for every successfully completed
CRC op writes the value into *op->crc_dst
UNLESS it is NULL.

The problem is that _idxd_prep_batch_cmd() that allocates
new ops left op->crc_dst uninitialized.

This results in a memory corruption (use after free)
in the following scenario:
1) op A is allocated an crc_dst is set to point to user memory X.
2) Op A is compeleted
3) User memory X is freed.
4) Ops B and C are allocated (chained), C has crc_dst set.
   => B reused op A memory and crc_dst still points to the
   now stale user location (1)
5) B is complered, spdk_idxd_process_events() writes into X
   as B->crc_dst = X.

Fix: _idxd_prep_batch_cmd() should initialize crc_dst to NULL.

Signed-off-by: Anton Eidelman <anton@lightbitslabs.com>
Change-Id: I9e7d57ec43a8fbcb3750906015a5cb7291278c35
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15115
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-10-25 07:10:55 +00:00
paul luse
13597fd4f1 accel_sw: add extra check on compression
We were missing a check when ISAL uses the complete output buffer
on compression to determine whether it was s perfect fit or if
simply not enough buffer was provided.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I73532666f50cb9fbef3c42f6bfb25fc5c7de01c6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14874
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2022-10-25 07:09:37 +00:00
shua.lv
aa22e6f774 include: add extern c to bdev_zone.h
Change-Id: I262dd0bf4871b1e364c8f7de62f20f0ec3bfa500
Signed-off-by: shua.lv <shua.lv@outlook.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13821
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2022-10-25 07:08:57 +00:00
Michal Berger
af5b654df8 test/setup: Adjust use of nullglob, extglob in cleanup_linux()
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I4e5a28f916c5855fefba8594d549b56c63d71013
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15094
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-10-25 06:57:59 +00:00
Michal Berger
8c19851209 scripts/setup: Enable extglob, nullglob in parser's global scope
This done due changes in latest Bash releases (>=5.2) which is
currently shipped with fedora36.

Fixes issue #2740.

Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: Id6e6f00eb4ae8b7f2b110848fd55e710368fcf94
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15081
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: wanghailiang <hailiangx.e.wang@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Kamil Godzwon <kamilx.godzwon@intel.com>
2022-10-25 06:57:59 +00:00
Kamil Godzwon
fd487dbd8b iscsi_tgt/sock.sh: comment out test with openssl ktls option enabled
Do not run test with message send using hello_sock client with
KTLS enabled. CI does not have systems that run with openssl-3
and KTLS and it fails under Fedora36.

This test should be brought back after systems do run openssl-3
on board.

Signed-off-by: Kamil Godzwon <kamilx.godzwon@intel.com>
Change-Id: Id7c46ced745fb4564b7428202a902cb332d30b33
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14830
Reviewed-by: Michal Berger <michal.berger@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-10-21 07:33:15 +00:00
Krzysztof Karas
bdf42664d1 scheduler.md: update static scheduler description
Add a note about not supporting re-enablement of
static scheduler.

Change-Id: I073d904311a88bf812201ca2b1465dcbdc821792
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15056
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-10-21 07:33:06 +00:00
Krzysztof Karas
a74c8c2e8c scheduler: prevent user from switching back to static
Prevent user from switching back to static scheduler after
different scheduler has been selected. Currently we
do not have a way to save initial thread distribution
configuration, so each time user switches from dynamic
scheduler back to static, the SPDK threads may end up on
different reactors. This would cause discrepancy in
performance statistics of SPDK managed by static scheduler.

Change-Id: Ic17a6be55eaea0e1a748f92e01f7075540403637
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15055
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-10-21 07:33:06 +00:00
Denis Barakhtanov
bdc683aaa9 bdev/daos: add resize rpc call
Signed-off-by: Denis Barakhtanov <denis.barahtanov@croit.io>
Change-Id: I71d643733e31eb2229649cff3db610d5bee07796
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14921
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
2022-10-21 07:19:17 +00:00
Jim Harris
a9be4f2c2f trace: add likely/unlikely hints to _spdk_trace_record
This helps generate slightly better code in this function,
which can have a noticeable impact for high trace
event workloads.

Tested with bdevperf, single malloc or null bdev,
qd=32, 512B randreads on a single Xeon core.
Specify "-e bdev" to enable bdev trace events.

Null:
Before: 8.09M/s (123ns per IO)
After: 8.68M/s (115ns per IO)

Malloc:
Before: 4.21M/s (237ns per IO)
After: 4.34M/s (230ns per IO)

Note that each bdev I/O generates two trace events (START
and END) - meaning this change removes 7-8ns of overhead
for every 2 trace events, at least on my system.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I7021b7f9e28b4a7cb16f8a97b4d4004ae165efd2
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15096
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2022-10-21 07:18:37 +00:00
Yuhua
4c6a2e3daa bdev/aio: implement read-only
Support Aio bdev 'readonly' option in RPC call. The read-only flag
can be dumped from the bdev info. Any writes on a read-only aio bdev
will be fail.

Signed-off-by: Yuhua <yuhua@smartx.com>
Change-Id: I939f72479f8953a3678a8df3083ecce0f96844fb
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14955
Reviewed-by: GangCao <gang.cao@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2022-10-21 07:18:15 +00:00
Shuhei Matsumoto
0954302091 sock: Ensure recv/send_buf_size to be larger than g_sock_impl_opts.recv/send_buf_size
In a use case, a custom sock module supports zero copy for read.
The custom sock module wants to keep the recv_buf_size to be sufficiently
large, for example 16MB. However, most upper layers overwrite the recv_buf_size
by a smaller value via spdk_sock_set_recvbuf() later. This is not desirable.

To fix the described issue, change the meaning of impl_opts->recv_buf_size
to be the minimum size, and spdk_sock_set_recvbuf() uses the maximum value
among the requested size, g_spdk_sock_impl_opts->recv_buf_size, and
MIN_SO_RCVBUF_SIZE.

We may have to change the code to initially create a socket. However,
for most cases, the upper layer calls spdk_sock_set_recvbuf() anyway.
Hence this fix will be minimal and enough.

For the use case, it is enough to change recv_buf_size of the posix sock
module. However, the custom sock module may support I/O uring in future.
Hence, change I/O uring sock module together.

Additionally, for consistency, change the meaning of impl_opts->send_buf_size
to be the minimum size, and spdk_sock_set_sendbuf() uses the maximum value
among the requested size, g_spdk_sock_impl_opts->send_buf_size, and
MIN_SO_SNDBUF_SIZE.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I051ba7cb50bc9dcad229e922198b04fe45335219
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14915
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Community-CI: Mellanox Build Bot
2022-10-21 07:17:37 +00:00
GangCao
eab96073cb test/bdevperf: set a random seed for rand rw testing
To fix issue: #2726

And also fix the examples/nvme/perf tool.

Change-Id: I4d1d362b7c1096d27a2d431851462d7bd161df88
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14911
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: wanghailiang <hailiangx.e.wang@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Community-CI: Mellanox Build Bot
2022-10-21 07:16:50 +00:00
KanKuo
2b5f264e7b UT/bdev/part.c:add the test of io_channel
Signed-off-by: KanKuo <kuox.kan@intel.com>
Change-Id: I204fe95d734b21e2b26994bfd0fefb4a6d701d84
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14648
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Community-CI: Mellanox Build Bot
2022-10-21 07:12:54 +00:00
Michal Berger
a4719db420 test/nvmf: Remove redundant steps from fabrics_fuzz
Also, clear generated fuzz logs upon successful test run - they may
get quite big (> 250MB) in size which doesn't play well with the
artifacts limit CI enforces. If the test succeeded then there's no
point in keeping them around for any potential debugging.

Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I240e8217cd50dba258310604c6985d8f6a832756
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15030
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: <sebastian.brzezinka@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2022-10-19 07:51:33 +00:00
Konrad Sztyber
f3f3365688 accel: correct spdk_accel_completion_cb's description
The callback is executed with the opaque argument specified by the user,
not accel_task.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I6f4388d8c4ceb37d4df174a6f4c0529012fe1f28
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15020
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-10-19 07:51:03 +00:00
Michal Berger
28bfb8763b scripts/setup: Don't overwrite already allocated number of hugepages
Don't touch existing hp allocations when they already meet the NRHUGE
requirement. Introduce new environment switch, SHRINK_HUGE, to
override that new behavior.

Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I0cd124b98c3deb7c21b4fbd57529d0995978c2e5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14912
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-10-19 07:49:22 +00:00
Alexey Marchuk
c77b537786 accel: Save overridden options in json config file
Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: Ida2c6f1c460c2b66d2d4159d225036377e488e62
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14856
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
2022-10-19 07:47:58 +00:00
Alexey Marchuk
8d27a5f7ed check_format: Update a hint for Darwin tools
Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: I483cb49c56b02cd9d24dff2c71395a2e1b0a6a80
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14865
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
2022-10-19 07:47:58 +00:00
Konrad Sztyber
8c9ff451b2 test: remove SMA tests from skipped_tests
The SMA tests are now tested on CI, so they no longer need to be on the
skipped_tests list.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I5746da06540efb1d3451a688fec4727d253bfc89
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14151
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Michal Berger <michal.berger@intel.com>
2022-10-19 07:47:20 +00:00
KanKuo
7315b003e0 UT/bdev/bdev_zone.c:add the test of zone_appendv
Signed-off-by: KanKuo <kuox.kan@intel.com>
Change-Id: I9f0fe81802d9eda640131a8bf7aec124b112206a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14694
Reviewed-by: Jun Wen <junx.wen@intel.com>
Reviewed-by: <yex.chen@intel.com>
Reviewed-by: wanghailiang <hailiangx.e.wang@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-10-18 07:30:10 +00:00
Anton Eidelman
c2c8b4ebc7 lib/idxd: fix bug in crc32c with chained ops
When spdk_idxd_submit_crc32c() handles input
with multiple iovs (or multiple ops are generated
due to physically discontinuous buffers),
the first op has the original seed, while the
subsequent ops instruct the hardware to
to fetch the seed from the output of the previous op
(op->hw.crc32c_val):
        void *prev_crc;
        ...
        desc->flags |= IDXD_FLAG_FENCE | IDXD_FLAG_CRC_READ_CRC_SEED;
        desc->crc32c.addr = (uint64_t)prev_crc;  <<< virtual addr

The problem is the prev_crc is a virtual address,
so the hardware (at least with no IOMMU configured)
reports: DSA_COMP_HW_ERR1
spdk_idxd_process_events: Completion status 0x20

Solution:
Set crc32c.addr to the physical address of
the crc32c_val field in the previous desc.
Since desc->completion_addr already holds the physical address
of the dsa_hw_comp_record, we use this with the crc32c_val offset.

Signed-off-by: Anton Eidelman <anton@lightbitslabs.com>
Change-Id: I330e98c2f3fd6da5cb4fc03d0745df09a9ff0e0c
Signed-off-by: Anton Eidelman <anton@lightbitslabs.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14954
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-10-18 07:24:55 +00:00
Konrad Sztyber
c996aff28d build: move protobuf files to a top-level directory
Up until now, the protobuf files were all located under
`python/spdk/sma/proto`.  This was fine, as they were only used in SMA
and we only generated python code.  However, now, we would also like to
to output code for other (e.g. go) languages too, so it no longer make
sense to keep them under `python`.

Only the protobuf files will be stored under `proto`.  The autogenerated
python code will remain under `python`, while the go code will be put in
the `go` directory.

While here, simplified the Makefile by removing a function generating
rules based on a directory, because we keep all protobuf files in the
same location.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I4dcb7b33cd6f2930732f04dee62e35571b09315b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15016
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
2022-10-18 07:23:54 +00:00
Konrad Sztyber
1f3a6b0398 rpc: use rw access when creating RPC lock file
It allows the users to specify the path to the RPC socket on a NFS
mounted filesystem.  This is necessary, because flock(2) on NFS requires
write access to place an exclusive lock.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: If197498ed5bdcb4e02c5f2f2b2c1ef388872c457
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14993
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2022-10-18 07:23:28 +00:00
GangCao
f20b99bbb3 lib/nvme/vfio: destruct ctrlr in failed cases
Change-Id: Ie7d7ab25055c26ea1c2ae4997bf7197a170de989
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15005
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-10-17 12:52:55 +00:00
Michal Berger
f72f154719 autotest: Move the llvm_nvme_fuzz to test/fuzz
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I2ccdd807a6585dd7b24c01f3880df721147b1c71
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14555
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-10-17 12:38:20 +00:00
gongwei
6f445382a8 bdev_iscsi: add bdev iscsi config json
save bdev iscsi opts config

Signed-off-by: gongwei <gongwei833x@gmail.com>
Change-Id: I9601098b426b8c080ae374f2fa1c23eec14f140b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14898
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-10-17 12:37:36 +00:00
Pawel Piatek
f658f463a4 autobuild: tiny version
Allow to run AUTOBUILD test in two versions:
* full - original one
* tiny - minimalistic, currently only check format and
         check so deps

We want to reduce first phase of per-patch (called Pre-Tests)
by running only basic tests. To accomplish that we will run
autobuild test in tiny version. Full version will be executed
in second phase of per-patch (called Tests).

Signed-off-by: Pawel Piatek <pawelx.piatek@intel.com>
Change-Id: I87d503ec272080ab238cf5129c44f92d8bb690ed
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14481
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Michal Berger <michal.berger@intel.com>
2022-10-17 10:50:40 +00:00
Michal Berger
e802c16570 test/packaging: Introduce separate test suite for packaging
Move the RPM builds away from autopackage.sh into a dedicated test
suite - this will allow the actual packaging tests to be run in a
separate instance.

Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I23a824db7aa3a5e469bea9b2536a83ae0a5c8e04
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14937
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2022-10-17 10:48:31 +00:00
Michal Berger
8ecd4994fb rpmbuild: Disable tracing for the %build stage
rpmbuild has a built-in variable that enables tracing for each script
template. This generates unecessary noise throughout the build part so
disable it there.

Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I1a0010d518915d11af4a628b929edbc3b670d56e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14803
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2022-10-17 10:48:31 +00:00
Michal Berger
a7de03068d rpmbuild: Don't use default compiler flags set by rpmbuild
Latest rpmbuild has an enhanced set of macros which define different
compiler flags inside the build environment. Their defaults, however,
may impact the overall SPDK build (and its submodules) in a bit
unpredictable manner. With that in mind, make sure that defaults are
not use and only flags specified by the user are kept in use.

Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: Ie3a3123011a43be78f20f805ea6f8cad6571beec
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14802
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
2022-10-17 10:48:31 +00:00
Michal Berger
23019242d7 rpmbuild: Add wrapper function for copying .so files
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I0c8b5549080c1617a5eb0c1cc6145af6f1b0845d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14801
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
2022-10-17 10:48:31 +00:00
Michal Berger
cd9266403a rpmbuild: Include libvfio-user libraries
In case --with-shared build bundles with --with-vfio-user the vfio .so*
files must be also included for the applications to link properly. To
that end, create a separate package that will ship with these files on
board.

Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I2c7e933b531b67a00e5af1e3b4dfbfaf8759d7ad
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14800
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
2022-10-17 10:48:31 +00:00
Michal Berger
5e75ad20d0 rpmbuild: Add helper functions for copying files into the spec
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: Ic53bd1733e2462c8d5c5d0ff057405fbcb20c19b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14799
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
2022-10-17 10:48:31 +00:00
Michal Berger
3884539699 rpmbuild: Don't copy binaries from build/bin build/examples
make install already does that for us - from build/examples some
subset of binaries is selected but in the rpm scope it should be
enough.

Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I3045711fc3daead9c255fc586600abe5796bc783
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14798
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: <sebastian.brzezinka@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
2022-10-17 10:48:31 +00:00
Michal Berger
b1aaa1e817 rpmbuild: Pass $MAKEFLAGS to make {clean,install}
This can speed up these commands by passing, .e.g, -j N args as per
default defined in rpm.sh.

Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: Ia64d3c0a0eef18e449e3020f31eaae8dd4caf483
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14797
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: <sebastian.brzezinka@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
2022-10-17 10:48:31 +00:00
Michal Berger
3915135275 rpmbuild: Simplify cmdline checks
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I7b7e4118e3573d3d1ad83a50d4a980bd69c8c0f4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14480
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: <sebastian.brzezinka@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
2022-10-17 10:48:31 +00:00