Commit Graph

320 Commits

Author SHA1 Message Date
paul luse
19e2dc3853 configure: rename --with-reduce --with-vbdev-compress
This is in prep for adding a new compressDev accel_fw
module that will contain all of the DPDK compressDev specifics
on it, the vbdev will make calls to the accel_fw instead.

As the accel_fw has SW based compression, we want the configure
option to apply to building the vbdev module but not the accel_sw
software implementation or the upcoming compressdev module.

Renamed to "compress" as reduce is a term specific to the vbdev
implementation of the compression to be provided by the accel_fw
and thus the same reason why we leave the test flag called REDUCE
because it's controlling tests for the reduce library as well as
the vbdev module that is using reduce.  The flag does not apply
to the SW implementation of compression.

This does not affect upcoming accel_fw compressdev module, that
will have its own configure option.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: If8ed3e48e1e3dabcaad1cd161289e78122cd9d58
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15179
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2022-12-08 12:55:27 +00:00
paul luse
eb53c23236 add (c) and SPDX header to bash files as needed
per Intel policy to include file commit date using git cmd
below.  The policy does not apply to non-Intel (C) notices.

git log --follow -C90% --format=%ad --date default <file> | tail -1

and then pull just the year from the result.

Intel copyrights were not added to files where Intel either had
no contribution ot the contribution lacked substance (ie license
header updates, formatting changes, etc)

For intel copyrights added, --follow and -C95% were used.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I2ef86976095b88a9bf5b1003e59f3943cd6bbe4c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15209
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-11-29 08:27:51 +00:00
John Levon
0d0de8e7d9 lib/rpc: add RPC allow list
Add an optional allowlist for RPC methods: if the method is not listed,
it is not allowed to be called or visible. This can be used to restrict
accidental mis-configurations, and generally helps locking down the
configuration surface.

Signed-off-by: John Levon <john.levon@nutanix.com>
Change-Id: Ied78fc4b14b60cb94ed0852b92deb6df545cbec4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15275
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-11-15 08:31:02 +00:00
Jim Harris
c3e62d7729 test: add new cmdline.sh test script
This can be used to test various app command line
parameters - in this case, the new RPC allowlist
parameter.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iaa0e90940ac4fb0b044ab96bc2912afa9f48ddae
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15310
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-11-15 08:31:02 +00:00
Michal Berger
405d338f74 autotest: Clear traps right before entering autotest_cleanup
Currently, autotest_cleanup()'s failure causes the EXIT trap to
run it again. This step is not necessary, so clear the traps
prior running it.

Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I8f841186bf9e9c0e82c8afc409129acad393efe8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15154
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2022-10-27 09:24:38 +00:00
Michal Berger
008139c8de test/bdev: Add xnvme support to blockdev.sh setup
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I86ffc7d35243560bcfedc31bf21630b2381d8c18
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14995
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Kamil Godzwon <kamilx.godzwon@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-10-27 09:06:37 +00:00
Michal Berger
f72f154719 autotest: Move the llvm_nvme_fuzz to test/fuzz
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I2ccdd807a6585dd7b24c01f3880df721147b1c71
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14555
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-10-17 12:38:20 +00:00
Artur Paszkiewicz
030951b454 test/blockdev.sh: test raid5f
Change-Id: Id41448dc828d153ee328d9ce2d1dfff6e8fae5b7
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14718
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-09-29 22:52:45 +00:00
Sebastian Brzezinka
66e8a09181 test/nvmf: add vfio-user fuzzing target
Add test to per-patch/nightly CI pipeline job

Change-Id: I6e1f92158ccc549967234a9a4e78330f2ccb9c9e
Signed-off-by: Sebastian Brzezinka <sebastian.brzezinka@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13818
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-09-23 15:16:01 +00:00
Michal Berger
c738575823 autotest: Move nvmf's fuzzing outside the main suite
This test is static in nature and not dependent on actual nvmf setup
that comes from the CI - everything is driven via a static json config
that comes with the test itself.

Instead, keep fuzzing directly in autotest.sh, under a top dedicated
test flag SPDK_TEST_FUZZER - this should allow users to run only
fuzzing part without touching the majority of the nvmf suite.

Change-Id: I9c33c46e05f7c4b977b041673fe200d9d2989265
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14287
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
2022-09-20 00:46:11 +00:00
Michal Berger
11d13b85e6 test/nvmf: Test different abort qd sizes under SPDK and kernel targets
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I79ede7670477ed0d8eb38fa3eced4a9b70c64620
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8962
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-09-19 13:13:42 +00:00
Ben Walker
34c48f1b3b accel: Do not refer to the "framework" as "engine"
The word engine was both used (interchangeably with module) to refer to
the things that plug into the framework and to the framework itself.
This patch eliminates all use of the word engine that meant the
framework. It leaves uses of the word that meant "module".

Change-Id: I6b9b50e2f045ac39f2a74d0152ee8d6269be4bd1
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13918
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2022-09-06 07:16:17 +00:00
Michal Berger
53ec23cae6 test/xnvme: Add simple test covering xnvme
This test is meant to simply verify basic IO and rpc integrity with
regards to different io_mechanisms.

Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I936d57c6ac539e77761656a1ec2b8ad1013f918f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14119
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2022-08-29 11:40:08 +00:00
Artur Paszkiewicz
0291b2845a FTL: Add read path
Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Change-Id: Ib5bac109b59d5a21a7dad1f8e79b5da7633ffa9d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13334
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-08-19 17:37:14 +00:00
Jaroslaw Chachulski
cfbab2dcd7 test/sma: Add new SPDK_TEST_SMA test flag
Signed-off-by: Jaroslaw Chachulski <jaroslawx.chachulski@intel.com>
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: Ib3ee2c147f07345c9880f63d75e3a87144584653
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13901
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: <sebastian.brzezinka@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-08-12 09:00:04 +00:00
Wojciech Malikowski
81dca28884 ftl: remove deprecated ftl library
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Change-Id: I3ebb05be3f1b9864b238cb74f469b4fdf573cd0d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11120
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
2022-07-11 07:23:58 +00:00
yupeng
c7db171c01 blobstore: add test script for blobstore grow
It launches the bdev_svc app, creates an lvstore,
then grows it twice. In the first grow, the lvstore has
no logical volume. In the second grow, the lvstore has
a logical volume.

Signed-off-by: Peng Yu <yupeng0921@gmail.com>
Change-Id: I21a7d3f1d2e6a6fe72e42f89023a8588391d2aba
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12320
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-06-28 17:55:43 +00:00
Karol Latecki
a985a3ae01 test/common: don't specify output dir for core_pattern in command line
Remove output directory argument for core-collector.sh when
setting system core_pattern. Instead save the full output
directory path into a file to a known location and read it
directly in core-collector.sh.

The reason for this change is 128 bytes command line length
for core_pattern (see 'man core 5'). In case working with
long paths the core_pattern command line gets truncated and
core dumps are not generated.

Change-Id: Ia74c180e4923fb43d6ff66129ffe54c32827adca
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12755
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michal Berger <michallinuxstuff@gmail.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-06-08 17:29:08 +00:00
Rui Chang
1a14eb4e25 test/unit: fix "git clean" failure
In autorun.sh it executes autotest.sh as root. But in autotest.sh and
test/unit/unittest.sh, when run git clean -f'*.gcda', it may show errors like:

 08:03:54	-- unit/unittest.sh@277 -- # cd /home/jenkins/spdk_repo/spdk
 08:03:54	-- unit/unittest.sh@277 -- # git clean -f '*.gcda'
fatal: unsafe repository ('/home/jenkins/spdk_repo/spdk' is owned by someone else)

The reason is the spdk source folder owner may not be root but we run the git
clean command as root. Fix it by using the folder owner's id to do
the git clean operation.

Signed-off-by: Rui Chang <rui.chang@arm.com>
Change-Id: Ifbd3098d4c32ffef5dcca738625302f63572e0c6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12693
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-06-01 08:58:28 +00:00
Karol Latecki
fd9ad0254a autotest: add nvme_rpc_timeouts test to execution
Script was merged at some point, but was not executed
during tests.

Fixes #2443

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I29df83b034c98d10bc2900ab1a4162d62157cace
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12239
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Jaroslaw Chachulski <jaroslawx.chachulski@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jun Wen <junx.wen@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-04-20 08:20:24 +00:00
Krzysztof Karas
c8802892ab hotplug: rename and refactor hotplug.sh
Rename hotplug.sh to sw_hotplug.sh (software hotplug)
and refactor its code in preparation for new software
based hotplug test.

Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I751fc60ee013de341e064c83f0be3a9b404008e7
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8834
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
2022-04-05 07:42:43 +00:00
Michal Berger
cbda5664f5 test/nvmf: Remove support for soft-RoCE setups
Motivation: https://github.com/spdk/spdk/issues/2277

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I6a85816c65ebecf63c2f454e4b97484542faef9e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10929
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-03-22 16:01:54 +00:00
Michal Berger
e8a5e965de test/nvmf: Refactor hw detection routines
Merge detection functions into a single one where all supported
devices are picked up and exposed in the environment. Be verbose
about what's been found for given tests.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I780c5e480c301dbb92092074f927a731ed2680b1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7849
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
2022-03-01 16:01:37 +00:00
Tomasz Zawadzki
b8a51a937e test/common: move all LIB paths to autotest_common.sh
The common file is sourced in all test scripts, so it
is ideal place to put all library paths.

It will make changes to those paths simpler in next patch.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ic37148e6be669f296e8561d6554e735132941413
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6685
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2022-01-26 08:19:06 +00:00
Michal Berger
89df50bc2d scripts/core-collector: Don't limit core dump collection
We should gather the core regardless of the RLIMIT_CORE setting.
Also, don't limit number of parallel executions of the collector
in case more than 2 processes crashed (unlikely to happen anyway,
but just in case make sure we cover it). Lastly, wait a bit longer
for the potential cores.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Ic7de8f8000676122282469b729f45b235ab6f9ca
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10852
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-01-05 09:24:22 +00:00
Mao Jiang
a0d255ac02 test/vfio_user: Build vfio-user test directory
Add test script for launching VMs with vfio-user transport,
this test script not active until vfio-user capable qemu available,
new flag SPDK_TEST_VFIOUSER_QEMU will decide case whether to run.

Change-Id: I9b885cec797fe3bb28860f1ec436c31a6fa2f131
Signed-off-by: Mao Jiang <maox.jiang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10096
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2021-12-31 10:10:24 +00:00
Michal Berger
5247e10a06 scripts: Get soft core ulimit from the template
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I6794886832ec641ec18b72621f21264970d805d5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7118
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2021-12-16 08:46:14 +00:00
Michal Berger
d2fe6d7d96 test/nvme/zns: Add simple fio test for zoned nvme devices
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I2ecad040bcaa573ace5ea2adfb1c0735d8388ce2
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9182
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-12-06 08:34:46 +00:00
Michal Berger
0231fdc770 autotest: Skip use of any zoned nvme devices
Our tests, especially those which use nvme block devices for various
use-cases, won't be able to perform successful IO on such devices.
The idea is to skip them for now and introduce basic, dedicated,
tests for zoned nvmes in oncoming patches.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I67baad5c85c662921e3327f2101180283c89e96c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9181
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-12-06 08:34:46 +00:00
Michal Berger
ea71df4f48 scripts/vagrant: Drop OCSSD awareness from functional tests
This also translates into switching fully to upstream QEMU for the
vagrant setup.

This is done in order to move away from OCSSD and SPDK's qemu fork
and align with what upstream QEMU supports. Main changes touch the
way how nvme namespaces are configured. With >= 5.2.0 it's possible
now to configure multiple namespace under single nvme device. Each
namespace requires a separate disk image to work with. This:

-b foo.img,nvme,1...
-b foo.img
-b foo.img,,..

Will still configure nvme controller with a single namespace attached
to foo.img.

This:

-b foo.img,,foo-ns1.img:foo-ns2.img

Will configure nvme controller with three namespaces.

Configuring nvme controller with no namespaces is possible via:

-b none ...

Note that this still allows to define other options specific to nvme
controller, like CMB and PMR. E.g:

-b none,nvme,,true

This will create nvme controller with no namespaces but with CMB
enabled.

It's possible now to also request for given controller to be zoned.
Currently if requsted, all namespaces under the target controller
will be zoned with no limit set as to max open|active zones.

All nvme devices have block size fixed to 4KB to imititate behavior
of the SPDK's qemu fork.

Compatibility with spdk-5.0.0 fork is preserved in context of setting
up namespaces so this:

-b foo.img,nvme,2

is valid as long as the emulator is set to that of spdk-5.0.0's.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Ib5d53cb5c330c1f84b57e0bf877ea0e2d0312ddd
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8421
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-12-06 08:34:46 +00:00
Michal Berger
a3ac80c105 test/config_converter: Drop config_converter tests
It's been some time since SPDK switched from ini to json,
there's no much benefit from keeping these tests around.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I0cf7fcd6587abe872553211e41bee12e14bf234f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10065
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-12-02 08:09:45 +00:00
Tom Nabarro
921c92e2a5 test/config: verify extra key in config doesnt break parsing
Add autotest case to verify that additional top-level key in JSON config
file doesn't break parsing when loading and initializing subsystems from
said file. This is a regression test to help applications use the same
config file to communicate private and SPDK data.

Signed-off-by: Tom Nabarro <tom.nabarro@intel.com>
Change-Id: I3ef6fe20d21398d30fa8011e20de1524e60ee841
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10204
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-11-30 09:07:02 +00:00
Michal Berger
a57af488ff autotest: Add routines for cleaning up lingering SPDK processes
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I713b2908fb124c76380b66659a1f646548fe1b70
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10205
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-11-24 08:34:47 +00:00
Michal Berger
6c5d98817a autotest: Move revert_soft_roce() outside of autotest_cleanup()
This function is part of test/nvmf/common.sh which is not included
in the test/common/autotest_common.sh. This makes autotest_cleanup
unusable unless parent script sources test/nvmf/common.sh directly.

To make its scope more obvious, put the revert_soft_roce() inside
autotest.sh.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Id8a8cf448d73abb6fe924b452f984f6e5381266f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10206
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-11-17 16:58:39 +00:00
Michal Berger
1b5d4b5265 test/nvmf: Limit number of tests executed under uring job
The nvmf-tcp-uring-vg job is currently fully packed with tests what
causes random timeouts across the CI. Since most of these tests are
run as part of the nvmf-tcp-vg job and are not uring dependent we
can simply skip them to save some time (~ 5 minutes).

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I5bc85afee094697b74efbef4777cb4eb1be04583
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9937
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2021-11-15 12:00:30 +00:00
Maciej Wawryk
1aa72efad3 scripts: exit from tests when no qat card
Also moved qat_setup.sh to functional tests
fixes: GH#2203

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Iab69b192a29013a69f4cc4a251a34dcc6fbf6b7d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9952
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
2021-10-21 21:26:44 +00:00
Liu Xiaodong
b1906912e9 thread: reap unreg pollers in intr
When intr mode is enabled, it will be common that
poller is unregistered during interrupt processing.
Since poller unregister is a delayed operation,
mark it in spdk_thread object, and reap unregistered
pollers out of poller execution.

Fixes #2143

Change-Id: Ieb61fc7685f85af5c15e833dd1dd56f8c97a3b12
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5770
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-10-21 21:25:53 +00:00
Michal Berger
76f840c049 autotest: Check if nvme devices are in use before the wipe
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I4f838df0bfb91398eb5d179a982165adb8af3476
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9291
Reviewed-by: Konrad Sztyber <konrad.sztyber@gmail.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-10-20 21:18:52 +00:00
Michal Berger
8419c294b4 test/bdev: Make sure cleanup() is always called upon exiting
This is mostly relevant for making sure the nvme drive used for gpt
test is cleaned up. To that effect, replace dd with wipefs to make
sure all signatures are wiped and kernel updated with the changes.

Also, since gpt test is not supported on FreeBSD don't try to run
it from autotest.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Ic970026363a6269f85b63e713f4187268fd6f923
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9290
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@gmail.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-10-20 21:18:52 +00:00
Michal Berger
4f9ef50850 test/nvme: Simplify hotplug test
Instead of relaying on vhost VM image, which may be incompatible with
the host system, perform the hotplug test locally against detecte
nvme device.

Also, limit this test to Linux since it never was intended to run
on other platforms in the CI due to qemu dependency.

Fixes #2150

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Ie048343ceb52c1a45193a6c857bc3a09fa1ec3fd
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9461
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-10-07 09:36:00 +00:00
Karol Latecki
8106e20726 autotest.sh: remove udev.log if tests successful
Remove udev.log just before exiting from autotest.sh.
This file probably won't be useful in case all functional
tests passed and leaving it unnecesarily bloats the output
directory.

Change-Id: I1c019a09ed573a0423e085ce96919162efec80a6
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9413
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
2021-09-10 15:59:27 +00:00
Krishna Kanth Reddy
e93e308d3e test/nvme: Add test application to verify Boot Partition Read / Write support
Signed-off-by: Krishna Kanth Reddy <krish.reddy@samsung.com>
Change-Id: Id26304baef0dd320ecdd4e86cb069bfe8475f6c3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8632
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-09-07 07:34:09 +00:00
Shuhei Matsumoto
801bce0d33 test/thread: Add poller_perf to evaluate cost to run poller
Add a simple tool, poller_perf, to evaluate cost to run non-timed
poller or timed poller by any period, number of pollers.

struct spdk_thread and struct spdk_thread_stats can have the count
of poller runs but we do not have any particular use case for it.
So the poller_perf tool use its global counter for now.

By following similar tools, event_perf and reactor_perf, run
poller_perf once for non-timed poller and once for timed poller.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ic5b297bb23d0cd0dc1313e1638ac540ffd5fb8f8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8925
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-08-20 07:29:42 +00:00
Michal Berger
0b55bf715e test/nvme: Add basic test covering cmb_copy functionality
Change-Id: I083ce9560c37b0f0bd39c9dce3cf97e4038b5e31
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2006
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-07-20 07:59:43 +00:00
Michal Berger
c6db5f41e5 autotest: Don't run spdk_dd tests under crypto job
Currently, crypto job also executes tests from the SPDK_TEST_BLOCKDEV
suite, including spdk_dd tests. The dd tests under a bare-metal system
can take up to 4 minutes alone and that brings the crypto job to the
very edge of timing out.

Since these tests are extensively covered under separate vg jobs we
won't lose any coverage by excluding them from crypto's - the basic
set of SPDK_TEST_BLOCKDEV tests would still remain as part of this
job.

Fixes: #2017

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I877e63de1e43aeb06c134fdacd6f5efbbe2a6317
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8545
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
2021-07-07 07:26:13 +00:00
Darek Stojaczyk
9d17c1d0fc autotest: don't drop system cache
Without hugepage allocation at runtime this is
no longer required. Additionally CI VMs now
have a lifespan of just 1 test run, so there's
no need to clear the cache anymore.

Saves us a few seconds on every run.

Change-Id: I12cb4095c37bd5d22d3b3fcbce4b5072118d3000
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2375
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: <tomasz.rochumski@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-07-05 14:36:08 +00:00
Michal Berger
227428c3a4 autotest: Don't override default amount of hugepages on Linux to 8GB
This is relevant mostly for the VMs. Allocating that much memory puts
a lot of pressure on remaining pieces of the system as currently we
allocate total of 12GB memory per VM instance in the CI pool. Default of
4GB hp should be enough to cover majority of the tests hence if there's
any suite that requires more memory, setup.sh should be called there
directly with an optimal value.

This is done to make sure we can accomodate proper resources in the CI
by knowing what are the actual requirements rather than just blindly
allocating "as much as possible".

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Ie958c518c7ed702b068c7fc9042183fdf41aa816
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8436
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-06-24 17:33:10 +00:00
Michal Berger
a3a0651569 autotest: Run dd tests under SPDK_TEST_URING
As these tests were limited to SPDK_TEST_BLOCKBDEV the spdk_dd was
not being tested against liburing. Make sure it is the case now.
Also, fail the test if spdk_dd is not linked to liburing in case
where such tests were requested.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I5704577530048ed4775ea15a6cdacf23380b5072
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7653
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-05-20 15:15:33 +00:00
Krishna Kanth Reddy
861bf2d1c3 test/nvme: Add test application to verify Simple Copy Command support
Signed-off-by: Krishna Kanth Reddy <krish.reddy@samsung.com>
Change-Id: I7bf61d98c444b473ec73f6afaf31ac408af6e409
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7696
Community-CI: Broadcom CI
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-05-18 12:43:24 +00:00
Michal Berger
8651ef29ab autotest: Make sure entire EXIT trap is executed
process_core() indicates error where any core files are found. If
that happens to be the case when the trap is called, the remaining
pieces won't be executed due to errexit being enabled. With that
in mind, simply ignore process_core's exit status.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I83527468bd1b41bf92b9ee8189a4c425be914552
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6990
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-05-13 10:19:49 +00:00