Commit Graph

732 Commits

Author SHA1 Message Date
Karol Latecki
91407607e5 test/vm_setup.sh: add parted for Arch Linux
Parted is not installed in the system by default and
is needed by number of spdk tests.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I5d8628be2e328d528a8aa47a8084d33fbb63b187
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1482
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-04-08 06:44:03 +00:00
Tomasz Zawadzki
4faa7b5837 test: remove RUN_NIGHTLY_FAILING flag
Purpose of this flag is sketchy at best.
Failing tests should be either fixed or disabled/removed.
Running separate job in hopes of it fixing itself,
might not bring the expected outcome.

Case in point - test case in this patch
was added to this flag over a year ago.

This is last remaining test case under this flag.
So there is no need for it to exist.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I8481ac7102245889047dde083b34276d22271a03
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1495
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-04-08 06:43:31 +00:00
Shuhei Matsumoto
e7ead00b53 lib/thread: Wait until thread_is_exited() is true by thread_poll() after thread_exit()
Following patches will require thread termination so that thread_exit()
will move thread to exiting, thread_poll() will move thread from
exiting to exited, thread_is_exited() will detect the threadis exited,
and then call thread_destroy().

So change all places as a preparation.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I6b2e8aee5ed7cd160a88b4c9aaed7d90bd9dac07
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1640
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-04-08 06:43:05 +00:00
Shuhei Matsumoto
c9da0aecdc lib/thread: Ignore spdk_thread_exit() call after moving exited
Previously the caller had to check if thread is not exited when
it calls spdk_thread_exit().

Subsequent patches will change return type of spdk_thread_exit()
to void, and so include the check int spdk_thread_exit() in this
patch.

If spdk_thread_exit() is called when the thread is already exited,
collect INFOLOG and return normally.

This will make the next patch a little easier.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I8b94261575e770485b33c0b37e76e770b77b417c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1639
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-04-08 06:43:05 +00:00
Tomasz Zawadzki
0c7aeea925 test/mkfs: provide common function to create filesystem
There are tests which create a filesystem on a device.
Each one could use a timeout and retries for each mkfs.
Otherwise following error could appear due to not test
related system activities:
'mkfs.xfs: cannot open /dev/sdb1: Device or resource busy'

References #1141

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I7307b085ada42200995d8bd2fcfd0469ee60a732
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1178
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2020-04-07 08:37:49 +00:00
Vitaliy Mysak
d713b99c3a test/scripts: remove OCF from vm_setup script
Since we have OCF as a submodule, we do not need to
install it separately.
vm_setup.sh was used to install dependencies for testing SPDK,
so remove all references to OCF in the vm_setup.sh.

This is related to issue #1279

Change-Id: I616dd6e72101c78c6902603ef077eb7992809ab9
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1496
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-04-06 07:49:41 +00:00
Pawel Kaminski
7cf4602d11 test/vm_setup: Add module dependency for freebsd
Change-Id: Ied1da9dba02f9ff735ce603fee95aacd2981930d
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1140
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-04-02 14:51:24 +00:00
Maciej Wawryk
ae22c0019a test/vm_setup: Change cloning qemu submodule via git to https for avoiding proxy issues
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I4dab0b15234f0772356f79884867ac23fb319837
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1414
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-04-02 14:51:24 +00:00
Maciej Wawryk
9f6acc9a75 test/vm_setup: Update spdk git repository url
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I1f0f191ccbaa3e9464cd1b788599935d1bc994c2
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1413
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-04-02 14:51:24 +00:00
Maciej Wawryk
051d3fe016 test/vm_setup: Fix permision error in Centos7
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Ic704e9ae97bb68d96401818fb52940e86eaa36a3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1409
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-04-02 14:51:24 +00:00
Pawel Kaminski
add06ac392 test/vm_setup: Add packages needed for centos7.
Nbd package will enable tests that uses them.
Python2 is outdated. Python3 should be available.

Change-Id: I244c82b8ab2b8e4217dfb814f69882b2963fac70
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1079
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-04-02 14:51:24 +00:00
Pawel Kaminski
ebbcd7d9bc test/vm_setup: Add newer version of git for centos7
Git package from yum doesn't support -C parameter.

Change-Id: Ieb5f6e18dbab5d80897c1c5734af60535fb70eaf
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1064
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-04-02 14:51:24 +00:00
Maciej Wawryk
38fb0e01f7 test/vm_setup: Update QAT driver version
Update QAT link to latest version.
Previous version does not compile on kernels 5.5+ and gcc 9.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I115f6ead9ccc40a08774a1508b19a8ac7572845e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1476
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-03-31 07:59:16 +00:00
Karol Latecki
dbb84dd5d6 test/iscsi: run cleanup once before running the tests
Run iscsicleanup once before running the tests. This is
to ensure we delete any stale entries or files if they
were not removed properly in previous test runs.

Fixes #1311

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: Idb415adddfaa7cbb443377a292fae357c1707e79
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1539
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-03-31 07:58:11 +00:00
Tomasz Zawadzki
99cff8350b test: remove SPDK_TEST_EVENT flag
Test in event.sh take 5 seconds on CI at this time.
For the tests to run both SPDK_RUN_FUNCTIONAL_TEST
and SPDK_TEST_EVENT had to be enabled.

This served as very minor time saver, so removing
this flag shouldn't affter test times more that the
5 seconds.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ie3eaa587daf28b2efbc3cfdff5ad648b7529e7f1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1493
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-03-31 07:57:40 +00:00
Tomasz Zawadzki
158de0bf8e test: remove unused SPDK_BUILD_DOC flag
Building docs is tested in autobuild_test_suite
when SPDK_TEST_AUTOBUILD is enabled.
SPDK_BUILD_DOC can be safely removed.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ic871febdcaba708e2fbe6a55d31ec3ddb101a818
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1492
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-03-31 07:57:40 +00:00
Tomasz Zawadzki
a9a9610b4b test: remove unused SPDK_BUILD_SHARED_OBJECT flag
Building "--with-shared" is tested in autobuild_test_suite
when SPDK_TEST_AUTOBUILD is enabled.
SPDK_BUILD_SHARED_OBJECT can be safely removed.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I05b1d04c3e28ce72c0a4a6deb89d0b6d02e2bb67
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1491
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-03-31 07:57:40 +00:00
Seth Howell
f3bea41181 test/common: add shebang to autotest_common.sh.
Without the shebang, the builtin file command assumes autotest_common.sh
is a c file. This is obviously not the case, but it messes up file type
checking when we are confirming that all files are checked against scanbuild.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I45f32b23ef01b07ce9b72caed743e6670b99055e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1311
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2020-03-31 07:56:15 +00:00
Shuhei Matsumoto
2139be1598 lib/thread: spdk_thread_poll() measures run time per call by reading end TSC
Update spdk_thread_poll() to count SPDK thread stats correctly on multiple
SPDK threads per reactor configuration.

spdk_thread_poll() gets start time and reads TSC at end as end time,
and then gets delta between them as run time. Run time is added to idle
time or busy time according to the result of polling.

Reactor overhead is included into the next thread which calls
spdk_thread_poll() now.

spdk_thread_poll() saves the end time to the current thread to use it
as the start time of the next thread.

Unit test framework for this patch and the next patch need to access
thread->tsc_last. In the next patch, reactor will use the end time of
the current thread to the start time of the next thread in reactor_run()
to realize the idea.

Hence add an new API spdk_thread_get_last_tsc(). The corresponding
variable is named as tsc_last and it is good and is aligned with
DPDK (DPDK has used tsc_start and tsc_end as variable name). But
last_tsc will be better as API name because the last TSC value is
easier to understand.

Then add necessary unit test and update the unit test framework.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I5e465e9283c032acb427576d0c90f9e1414f2271
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1048
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2020-03-25 07:52:53 +00:00
Changpeng Liu
5f66e5b160 test/fuse: make the --with-fuse option be consistent with configure
Change-Id: I47dce4dfe01a7badb493978257577c1a8fc0be93
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1404
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-03-23 09:36:46 +00:00
Michal Berger
004b52fe6c test/bdev: Move part_dev_by_gpt() into blockdev.sh
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I9a888f80148d5fd2eec5feed299b0c81bb78b984
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1282
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-03-20 13:59:17 +00:00
Ziye Yang
465f4c1254 sock/uring: Add the async network I/O support for socket
This patch is used to add the async network I/O support in
sock layer. If code is configured with I/O uring, --with-uring,
we can use io uring in Linux (version >=5.4-rc3).

PS: We also make VPP's default priority > uring, because
for the iSCSI or sock test linked with VPP. It tests VPP with
a given address (which is not a special VPP can only open address),
so using uring can also listen those address succefully. And if we make
uring with priority > VPP, actually, VPP will not tested in those cases.

Additionally, the current CI pool is not ready for test, we need
wait for the CI system ready. And I test on my local platform, it works.

Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: Ie8ee3c8ddf8d2a7264f2b382376733e002816dcc
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/952
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-03-20 08:41:28 +00:00
Darek Stojaczyk
7098c65339 test/common: inline fio_config_add_job
The function is almost 20 lines long and it essentialy
appends two lines two a specified file. We don't need it.

Change-Id: I565de8e54aa71b04f15745456bec7b2b3b8fc178
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1288
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-03-19 08:50:59 +00:00
Karol Latecki
db43b387ba test: do not use iter_pci_class_code in tests
Do not use iter_pci_class_code function in tests to
iterate over NVMe drives. This function can return
drives which at the moment of execution can not be
whitelisted for use.

This can result in test errors (such as simply
bdev_nvme_attach_controller RPC command failing) or
even using and deleting data from NVMe drive which
was not meant to be used in tests.

Fixes #1235

Change-Id: I82b9935fc88605b636c2096be6c71d4880a567c8
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1309
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-03-18 08:03:21 +00:00
Seth Howell
4fe25e7233 vm_setup: add abigail for fedora machines.
The fedora machines running unit tests will require this package to run
the ABI compliance checking tests.

Change-Id: I9f9307c04e235cb9b6f78c4908609a529a86ccba
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1137
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-03-18 08:02:30 +00:00
Michal Berger
053c15b32b fio: Remove hardcoded path to fio repo
The hardcoded path is replaced with $CONFIG_FIO_SOURCE_DIR as defined
during the compile time.

Additionally, all checks which determine if fio is available are now
based on $CONFIG_FIO_PLUGIN=y knob instead of the presence of the fio
repo.

Change-Id: Ie469747f3863c9561f53d32e8c3f29778afaaf74
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1108
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2020-03-17 14:44:20 +00:00
Michal Berger
6aa9db94b3 test/common: Include build's environment in the test suite
Change-Id: I0f45c80991c22ca72fb29fdf9af6bc50fcd7ae99
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1107
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-03-17 14:44:20 +00:00
Tomasz Zawadzki
863d73d34b test: limit files scanned when searching for test completions
When gathering test completions original grep went over all
files in repository recursively. In some cases it took
way longer than required.

This patch limits searches only to rootdir and test directories.

References #1068

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ice0bba25f2fad62516226a7b045d12b6614bead5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1089
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-03-17 08:26:06 +00:00
Hailiang Wang
bb5567496f test/iscsi: add application for target fuzz testing.
Enables us to test randomized data against the iSCSI target interface.

Change-Id: I56bd5bcd936b92ba152d4d5678d7124b3165c03c
Signed-off-by: Hailiang Wang <hailiangx.e.wang@intel.com>
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/509
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-03-13 08:53:46 +00:00
Darek Stojaczyk
8b98cdb64a scripts/rpc.py: add daemon mode
Add rpc_cmd() bash command that sends rpc command to an
rpc.py instance permanently running in background.
This makes sending RPC commands even 17 times faster.

We make use of bash coprocesses - a builtin bash feature
that allow starting background processes with stdin and
stdout connected to pipes. rpc.py will block trying to
read stdin, effectively being always "ready" to read
an RPC command.

The background rpc.py is started with a new --server flag
that's described as:

> Start listening on stdin, parse each line as a regular
> rpc.py execution and create a separate connection for each command.
> Each command's output ends with either **STATUS=0 if the
> command succeeded or **STATUS=1 if it failed.
> --server is meant to be used in conjunction with bash
> coproc, where stdin and stdout are named pipes and can be
> used as a faster way to send RPC commands.

As a part of this patch I'm attaching a sample test
that runs the following rpc commands first with the regular
rpc.py, then the new rpc_cmd() function.

```
time {
        bdevs=$($rpc bdev_get_bdevs)
        [ "$(jq length <<< "$bdevs")" == "0" ]

        malloc=$($rpc bdev_malloc_create 8 512)
        bdevs=$($rpc bdev_get_bdevs)
        [ "$(jq length <<< "$bdevs")" == "1" ]

        $rpc bdev_passthru_create -b "$malloc" -p Passthru0
        bdevs=$($rpc bdev_get_bdevs)
        [ "$(jq length <<< "$bdevs")" == "2" ]

        $rpc bdev_passthru_delete Passthru0
        $rpc bdev_malloc_delete $malloc
        bdevs=$($rpc bdev_get_bdevs)
        [ "$(jq length <<< "$bdevs")" == "0" ]
}
```

Regular rpc.py:
```
real    0m1.477s
user    0m1.289s
sys     0m0.139s
```

rpc_cmd():
```
real    0m0.085s
user    0m0.025s
sys     0m0.006s
```

autotest_common.sh will now spawn an rpc.py daemon if
it's not running yet, and it will offer rpc_cmd() function
to quickly send RPC commands. If the command is invalid or
SPDK returns with error, the bash function will return
a non-zero code and may trigger ERR trap just like a regular
rpc.py instance.

Pipes have major advantage over e.g. unix domain sockets - the pipes
will be automatically closed once the owner process exits.
This means we can create a named pipe in autotest_common.sh,
open it, then start rpc.py in background and never worry
about it again - it will be closed automatically once the
test exits. It doesn't even matter if the test is executed
manually in isolation, or as a part of the entire autotest.

(check_so_deps.sh needs to be modified not to wait for *all*
background processes to finish, but just the ones it started)

Change-Id: If0ded961b7fef3af3837b44532300dee8b5b4663
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/621
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-03-11 11:35:03 +00:00
Tomasz Zawadzki
79c28aa698 test/build: disable compiling unused components in tests
This patch decreases the compile time when particular test
category is not executed by tests.

Added skipping the 'examples' dir during verification
of scanbuild scanned files. Only when examples were not
compiled.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I41e7b9d18913fd02e2b6cbe44f933ab2e0371d28
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1172
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2020-03-11 11:34:55 +00:00
Karol Latecki
148719dad9 test/common: skip run_test with warning
Add function to print warning with optional message
in case we'd like to skip a test.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: Iacb1031aac0e17ce80327ca826c51fb2b99a6be0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1080
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-03-04 10:05:26 +00:00
Pawel Kaminski
7c1a9e156b test/vagrant: Add support for yum in vm_setup.sh scripts.
Install centos related packages using yum.

Change-Id: I718222b2fb6dd7b249404e13f91849c54de09013
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1031
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-03-04 10:05:17 +00:00
Tomasz Zawadzki
7265600d63 test/common: do not double exclude autotest_common.sh
No need to exclude the autotest_common.sh twice when
creating test list.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ice7be31a9d7d284dd411ebfd3854e075b37c0275
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1088
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-03-04 10:04:38 +00:00
Tomasz Zawadzki
eba5d05515 test/common: only scan .sh files for run_test
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I6be62480adb37dd7b53429b831c5dcf5abfa5818
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1087
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2020-03-04 10:04:38 +00:00
Artur Paszkiewicz
73763d40ec module/raid: raid5 module
Add raid5 module and unit tests. Use './configure' with option
'--with-raid5' to enable it.

Change-Id: I9f07da8c3567fa65499444899c899adaa2e29550
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/855
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2020-03-04 10:03:45 +00:00
Michal Berger
692fce3222 test/common: Introduce timeout in discover_bdevs()
In case something goes awfully wrong with the test which calls this
particular function, e.g:

  https://ci.spdk.io/results/autotest-per-patch/builds/4787

the CI's autotest runs may be blocked for max of 15 minutes before
failing the build. Instead, introduce internal timeout for the
routine itself (default of 30s) and fail the test sooner when it
expires.

Change-Id: I2c792c1781987e80e34c1fa19ec3eef43c25e38a
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1026
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-03-03 07:55:12 +00:00
Maciej Wawryk
dad9d17406 test/common: Add smartmontools dependencies for FUSE and NVMe-CUSE in vm_setup.sh
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Ia8f12aca0a70e93bddd0ad7aeb658f85044ffdfa
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/956
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2020-02-27 10:16:19 +00:00
Darek Stojaczyk
ebe4bfc2bb test/common: add one more fio leak to ASAN suppresions list
It comes up on failed fio runs and bloats the log.

Change-Id: I8dd0196bfc98b3ee888aca0def83f0961fbc1197
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1010
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-02-27 10:15:52 +00:00
Michal Berger
c843612041 test/common: Introduce new test flag for enabling debug app runs
SPDK_AUTOTEST_DEBUG_APPS - if set to 1, all applications will be
			   executed with --log-flags=all

This should increase, to some extent, coverage of each application.

Change-Id: I1fc82a837abb7e94bda55f47056a120bd9be40d7
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/691
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2020-02-27 10:15:40 +00:00
Michal Berger
ee7cdc2c76 test/fuzz: Rename FUZZ_APP to VHOST_FUZZ_UP
Since there's more than one fuzz app covering different areas, rename
FUZZ_UP to VHOST_FUZZ_UP to make the type of the app clear right off
the bat.

Change-Id: I7730f1e14895d9221ec76dea0f53b5054a97be1c
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/978
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2020-02-27 10:15:40 +00:00
Michal Berger
b4c5509c6e test/common: Source *_APP from a common place
Instead of declaring each app in the actual test, source their default
declarations from a single location. Additionally, use them as arrays
now instead for easier cmdline management.

Change-Id: I0ea39403f070dc57987065ebb800e91c35de643a
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/690
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2020-02-27 10:15:40 +00:00
Michal Berger
e338a8dd37 test/common: Don't attempt to install tsocks package on fedora 31
Currently, this particular package is not available in fedora 31's
repos, thus don't exit the setup if its installation fails.

Change-Id: I56852ed172c95b23c92e25b1cec391b4e909f2bd
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/936
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-02-27 10:15:11 +00:00
Michal Berger
e4c26938a4 test/common: Drop single quotes in --extra-cflags arg
Since '' are placed within "" they are treated as literal part of the
argument passed to configure. Due to that, configure fails to run cc
since all -W flags are treated as one.

Change-Id: I69e182f308628cc5b63bd363d3a47cf87336a59a
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1024
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-02-27 10:14:51 +00:00
Shuhei Matsumoto
515733ca4a lib/thread: Add unique ID for each created SPDK thread
Add an unique ID for each created SPDK thread. Use a single 64 bits
variable, g_thread_id, and guard its update by the global mutex
g_devlist_mutex. For our safety, further thread creation is not
allowed if g_thread_id rolls over, and request user to restart SPDK
application.

Besides, as a minor update, move the debug log down and add ID to it
in spdk_thread_create(), and ID is added to thread_get_stats RPC and
framework_get_reactors RPC.

The thread ID will be used to set the cpumask of the running thread
to the specified value in the subsequent patches.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ic09f11d4c7175c3b89acba6a42e76063acd0d1a0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/498
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-02-27 10:14:23 +00:00
Jim Harris
39dce00bb5 test: add FreeBSD-specific ps options to get process name
Fixes issue #1145.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ifee3d8e041ca2f7acfb3e00c001e77027bf5cdb8

Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/970
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-02-26 09:16:45 +00:00
Tomasz Zawadzki
d7aff25df0 ut/common: add poll_thread_times() to poll select number of iterations
Right now functions poll_thread() and poll_threads(), polled
always until there were no further messages to process.
This allows to put next operation to execute only after
previous one completed.

Function poll_thread_times() allows to poll certain thread
for exact number of executions. If 0 is passed to max_polls,
then this function executes single message and all pollers,
until no messages are left.

With this change it is possible to only run a thread
selected number of iterations. Allowing to add another operation
without first one finishing.

It will become useful in blobstore UT, where two operations
will be executed with slight delay between them.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I4a6537695389f30130ea2fd2fd43d7b2cb2dea39
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/772
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-02-21 09:35:27 +00:00
Darek Stojaczyk
94675ba17b test/autotest_common: cleanup discover_bdevs
$rpc_server is hardcoded inside the function and can't
be empty, so remove the check for that.

Change-Id: Ic819089eb113b14a7c2407cd5ce882bcdee044bb
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/522
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-02-21 09:34:14 +00:00
Maciej Wawryk
9ebbf3acac scripts/pkgdep.sh: Add flag for developer tools installation
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I8876fd7c60198c597d931c5f498644c415c2a868
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/596
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-02-18 14:42:16 +00:00
Seth Howell
19260848f6 nvme: publicly declare spdk_nvme_transport
This will be useful in the upcoming spdk_nvme_poll_group api.

Change-Id: Id83340a2ce9887817312f5aac38db4de8c588974
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/577
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-02-18 08:05:08 +00:00
Karol Latecki
71efe5db24 Fix Markdown MD026 linter warnings - trailing punctuation in header
MD026 Trailing punctuation in header

This rule is triggered on any header that has a
punctuation character as the last character in the line

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I3ab4894092ef6b5f920d89b74e43c2e46e9581c0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/657
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-02-17 10:07:21 +00:00
Karol Latecki
93be26a51d Fix Markdown MD022 linter warnings - headers blank lines
MD022 Headers should be surrounded by blank lines

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I768324b00fc684c254aff6a85b93d9aed7a0cee5
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/656
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-02-17 10:07:21 +00:00
Karol Latecki
a41c031609 Fix Markdown MD012 linter warnings - multiple blank lines
MD012 - Multiple consecutive blank lines

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I3f48cdc54b1587c9ef2185b88f608ba8420f738b
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/654
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-02-17 10:07:21 +00:00
Shuhei Matsumoto
70ec72871e lib/thread: Fail spdk_thread_exit() if thread is already exiting
This is a preparation to support voluntary thread termination by
calling spdk_thread_exit().

Change spdk_thread_exit() to return -EINVAL if the thread is already
marked as exited.  This will be helpful to detect wrong call sequence
of voluntary thread termination.

Besides, update reactor shutdown and unit test framework shutdown
to incorporate this change accordingly.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I2296c61e273bf4d9580656dcbc2da0e8a8f3bcf7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/671
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-02-13 09:51:15 +00:00
Michal Berger
a8d016910a test/common: Make sure that extdebug is enabled as soon as possible
if given script, which executes under the debug tracer, fails before
xtrace_restore() is called, the BASH_ARGC[] will miss all the arguments
which were passed down till that very point. Similar case can be
experienced after tracer is disabled as then all the arguments hold by
BASH_ARG{C,V}[] become unavailable (i.e. until tracer is enabled again).

Since there's no actual benefit from toggling the extdebug (in fact, it
could break DEBUG|RETURN traps if ever used), enable it once when
autotest_common.sh is sourced and keep it enabled throughout entire
execution of given script.

Change-Id: I01001ead1570967a2e550d993f85f12b9f62553e
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/477
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-02-11 08:38:08 +00:00
Pawel Kaminski
1fd96c85b4 test/vhost: Replace old style config with json in initiator test
Change-Id: I622448dcd1de960793894395d7da55ebd612dcc7
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/529
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-02-10 10:25:11 +00:00
Michal Berger
a11aab73e8 test/common: Check if igb_uio is loaded before removing it
igb_uio doesn't necessarily have to be loaded into the kernel as SDPK
still may be built with support for it but only particular set of
tests is actually using it. If such a condition is met then rmmod will
fail the entire test run since it always exits with != 0 if given
module is not found.

Avoid this by checking first if igb_uio is actually in use.

Change-Id: Ib97488797c657f810b588a0b427e578807ebe6c6
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/625
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
2020-02-10 10:23:39 +00:00
Karol Latecki
3d8a0b19b0 doc: Fix Markdown MD032 linter warnings
"MD032 Lists should be surrounded by blank lines"
Fix this markdown linter error by inserting newlines or
adjusting text to list points using spaces.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I09e1f021b8e95e0c6c58c393d7ecc11ce61c3132
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/434
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
2020-02-06 09:46:06 +00:00
Seth Howell
a9a472b99b scripts: update to point to review.spdk.io
Change-Id: I10c79cd421bd735ced2e17919cfa38abec74edf2
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/451
Reviewed-by: Jim Harris
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: <sys_sgci@intel.com>
2020-02-04 23:05:51 +00:00
Michal Berger
9ba80de0dd test/nvme: Don't wait for spdk_stub0 if stub dies prematurely
If stub terminates right after execution, e.g. due to lack of system
resources (requested number of cpu cores to run on, right amount of
memory, etc.) start_stub() would end up blocking forever since there
wouldn't be any entity around that would mknod spdk_stub0 for it to
break the loop.

Avoid the above scenario by checking if $stubpid is still visible
under procfs and return if it goes missing.

To make sure kill_stub() is still called to clean up after start_stub()
declare proper trap prior the call to start_stub().

Additionally, avoid potential stderr noise in case kill|wait are told
to act on a PID that's not visible in the ns anymore.

Change-Id: Ief41200c57957f84b4f96a54baabc8da1f27dd43
Signed-off-by: Michal Berger <michallinuxstuff@gmail.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482653
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-02-04 20:22:23 +00:00
Michal Berger
4b1dbc948b test/common: Don't take randomize_va_space default setting for granted
Currently, kill_stub() doesn't gracefully handle this particular sysctl
since it overwrites it with assumed default that doesn't necessarily
have to be a part of kernel's config on a given system.

Don't presume what the setting should be, instead, read and save the
current value and try to restore it whenever kill_stub() is called.

Change-Id: I1f1ee85c29d5e2ec2f442a54f700e3bc45ee2437
Signed-off-by: Michal Berger <michallinuxstuff@gmail.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482652
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-02-04 20:22:23 +00:00
Michal Berger
18b9303dfb test/common: Include BASH_ARGV[]s in the backtrace
Enable extdebug and try to include all the arguments passed down
the function stack in the backtrace.

Change-Id: I81381c936b0f895f1ca8e31d57ef8116d737c6cd
Signed-off-by: Michal Berger <michallinuxstuff@gmail.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482695
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-01-31 12:39:22 +00:00
Michal Berger
1baf379e69 test/common: Don't attempt to read backtrace from unavailable source
In case cwd is changed during the execution of given BASH_SOURCE, i.e.,
when the dirstack is mangled by calls to cd, the actual executable may
end up missing from the path when run directly from its directory in
the ./ fashion. Example:

  [root@fedora31 fuzz]# ./autofuzz.sh --module=vhost --transport=all

autofuzz.sh cds into the $rootdir hence the BASH_SOURCE[i] in form of
./autofuzz.sh won't be found there, thus during a failure, since run
under a debug tracer, nl will fail with -ENOENT while trying to read
it.

To mitigate, check if $src is available for reading, if not, log that
the backtrace is not available.

Change-Id: I68988350ba36cca8464bdfac437f662ed4c30f67
Signed-off-by: Michal Berger <michallinuxstuff@gmail.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482694
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2020-01-31 12:39:22 +00:00
Tomasz Kulasek
327668c8c9 test/common: make gdb_attach global
Change-Id: Ib73bd9513681360f22251b54a2d15ca9807b72c2
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478807
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-01-27 17:39:52 +00:00
Or Gerlitz
e61b0904a8 sock/posix: Add flush
Initiator drivers (e.g nvme/tcp) don't use poll groups but rather directly
poll the qpair. In this case we want to allow the polling function (e.g
_qpair_process_completions()) to flush async writes pending on the socket.

Signed-off-by: Or Gerlitz <ogerlitz@mellanox.com>
Change-Id: Ibd8c73691213d58e287b7110d0f5a381a89a64d0
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/475419
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-22 13:53:09 +00:00
Seth Howell
253fb41b65 autobuild: Confirm we are building all files w/ scanbuild
This gives us an assurance that we know scan-build is being run against
all relevant files in the repository.

Change-Id: Ic0b871e98a9ea7acd2d6b2a99ab81955af29fc66
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/479898
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2020-01-22 13:50:17 +00:00
Seth Howell
d4e52cacfd test/bdev: remove the gpt conf file / parameterize tests
Previously, we only ran bdevperf tests against GPT partitioned NVMe
drives. These tests are generally applicable and should be run against
all of the bdev types we support.

So long story short this configuration file isn't needed and we can just
get rid of it.

Change-Id: Ia6ded22ce16fc1f76b7d99643b9d37e3ecbd1c60
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478244
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-01-22 13:50:17 +00:00
Seth Howell
df254d5a25 autorun_post: add a file for tests we don't run.
Change-Id: I8beac9959d08026ee984833d0e90ec17381a66e5
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478240
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-01-22 13:50:17 +00:00
Seth Howell
ea781d6dbb test: report test domains for tests.
This will allow us to run test cases under multiple suites without
clogging up the logs too much, but it will also preserve information
about which test suites were run or (more importantly) not run.

Change-Id: I2434a54a0877ae36b9f84bfab8a62653ac1172f8
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/477367
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2020-01-22 13:50:17 +00:00
Maciej Wawryk
53147e0dc8 test: fix issue with changing device names in waitforblk function
During test before upgrading our vm's to Fedora31 there were issue with failing
nvmf test on function waitforblk. In some cases hardcoded nvme0n1 name was not
found in system. Here is fix that change searching by name to searching by nvme
serial.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Ic5ee70804652a057fa26b8cc004b5227262d2122
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478471
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Seth Howell <seth.howell@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-20 10:00:06 +00:00
Ziye Yang
0bfaaace8f sock: Add impl_name parameter in spdk_sock_listen/connect.
Purpose: With this patch,

(1)We can support using different sock implementations in
one application together.

(2)For one IP address managed by kernel, we can use different method
to listen/connect, e.g., posix, or uring. With this patch, we can
designate the specified sock implementation if impl_name is not NULL
and valid. Otherwise, spdk_sock_listen/connect will try to use the sock
implementations in the list by order if impl_name is NULL.

Without this patch, the app will always use the same type of sock implementation
if the order is fixed. For example, if we have posix and uring together,
the first one will always be uring.

Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: Ic49563f5025085471d356798e522ff7ab748f586
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478140
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-01-16 09:11:32 +00:00
Seth Howell
f6cf92a31f lib/nvme: make transport.c use fn tables.
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: Ida58785784b4ed50393e1d43a9cd902de74a2eaa
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478873
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-01-16 09:10:38 +00:00
Seth Howell
e4eef6975c lib/nvme: add function tables for all transports.
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I4e7af1c42a19346f4abcb17910a41f8104a2de1b
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478871
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-01-16 09:10:38 +00:00
Seth Howell
5cddf93b47 test/nvme: add common_stubs.h file.
A lot of the stubs between these transports will end up being common.

Change-Id: Ib9c8ff947b95f34633eb13953405d3153a7f4ac7
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/479602
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-01-16 09:10:38 +00:00
Maciej Wawryk
19e48a66a4 test: Shellcheck - correct rule: Iterating over ls
Correct shellcheck rule SC2045: Iterating over ls output is fragile. Use globs.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I436878774d6f86c23b1c5cf5220da297053f03e2
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/477397
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-01-15 14:30:06 +00:00
Seth Howell
a571eb03e9 test: move report_test_completion into run_test
This also requires us to change the create_test_list function to rely on
the run_test function for creating the canonical test list.

Change-Id: Ib35e7752935a3ac83de2702b6dfbd42539027f6a
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476962
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-14 07:49:08 +00:00
Seth Howell
2f476a638f autobuild.sh: simplify and unify test suite.
We have added a lot of flags to this script over time to try and shorten
parts of it that take a long time or modify the way we make and
whatnot.

In the test framework today we really have two settings, we either want
to run the entire autobuild package with all the bells and whistles, or
we just want to make the code and get on with the rest of the tests. I
believe this change can save between 1 and 3 minutes on each of the
functional test suites.

Change-Id: I7519e8320aa16b57f09f633f866dc36cb494aa80
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478483
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-14 07:49:08 +00:00
Seth Howell
cbc1e069ee configure: allow cuncurrent build of RBD and reduce
This was disabled in configure because at one time there was an ISA-L
dependency break between the RBD libraries and our own submodule.
It seems that the dependency break was fixed a while ago, but the check
to not allow one to build with the other was never removed.
Unfortunately, I don't have the details on what that fix was.

Either way, this compilation works now.

Change-Id: Ic249791549dae36c7279114d67e77be94e842ddf
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/479726
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-14 07:49:08 +00:00
Seth Howell
39fe5c849b autotest_common: fix xtrace_restore_enable
My previous patch enabling this to be called recursively didn't
take into account that a single call to xtrace_restore would
override multiple levels of xtrace_disable nesting. This change fixes
that.

fixes: 190b2245c5

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I2167ba460e68223c9426b3d71e9c17019f947924
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478959
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-01-08 15:46:18 +00:00
Shuhei Matsumoto
f74107fb85 test/common: Mock CPU core operations for unit tests
Mock DPDK lcore operations for unit tests. Remove duplicated stup
from unit tests for NVMe-oF FC transport. The next patch will use
them.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I4f6fbd8f45942ca13fcf10ba1740a9fe8e573063
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478153
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-01-03 08:33:25 +00:00
Seth Howell
cb90136c33 test: eliminate case and suite from run_test
We will be able to distinguish case and suite in the post process
scripts based on if they have any internal tests.

Change-Id: Iff2aa5caa251924d8a842085d87eb2a17aca45d4
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478522
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
2019-12-24 08:24:42 +00:00
Shuhei Matsumoto
46e52a0bcb test/common: Set invalid value to CPU core API stubs
This change verifies that all existing unit tests don't care about
CPU core configuration.  The subsequent patches will mock CPU
core operations.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I44cc56908cf8609870625ff19d6100e21036c00f
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478152
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-12-20 10:05:38 +00:00
GangCao
82f60376ae test: properly handle the killprocess case
To check whether the process already killed and if the
process is not there, return error.

This is to fix the below issue:
https://github.com/spdk/spdk/issues/1066

Change-Id: I5f36213237a4b0eed8edbc69e666037cf17488b2
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476127
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
2019-12-17 15:28:37 +00:00
Seth Howell
190b2245c5 test: make xtrace_disable multi-call safe.
Previously, if you made two sequential calls to xtrace_disable without
calling xtrace_enable, you would lose the original value of x. Now this
is not the case. It allows us to call xtrace_disable freely without
worrying about whether xtrace_disable is invoked later in a function we
call.

Change-Id: I9818ae97532e7a31db576bb9bd2e2e30d67ab410
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/477965
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
2019-12-17 14:54:40 +00:00
Ben Walker
b7ad942612 sock: Add an asynchronous writev
Add spdk_sock_writev_async for performing asynchronous writes to
sockets. The user of this call is responsible for allocating their own
spdk_sock_request structures to pass to this call.

spdk_sock_writev_async will not return EAGAIN and will instead leave the
requests queued until they are fully sent or aborted due to socket
error.

Change-Id: Idf3239e65d26a3024e578122c23e4fb8f95e241b
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/470523
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-12-16 12:34:02 +00:00
Seth Howell
1fafd71bc3 test: add timing calls to run_test
And clean out the corresponding calls to timing_enter and timing_exit
from the various test scripts.

Change-Id: I0759417b5a529e4c3649ce04cca1799c089da278
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476804
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
2019-12-10 17:12:03 +00:00
Seth Howell
fd17f1aabc test: make run_test more robust
make sure we have enough arguments.

Change-Id: I76ce35635ef14289061323ee401d93d8d081888c
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476801
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2019-12-10 17:12:03 +00:00
Seth Howell
b8042c3850 test: use human readable names in test logs.
Change-Id: I163a7e08bfeff3da2ee627d9fad9f2f1a249c593
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476820
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2019-12-10 17:12:03 +00:00
Seth Howell
3710048472 test: add a test_name param to run_test
This will allow us to use timing_enter and timing_exit directly
inside the run_test function. That function already lends itself well to
nesting the way we do our timing.

This patch series is aimed at combining the timing_*, run_test, and
report_test_completions calls all into a single place. This will greatly
reduce the number of lines of code in our bash scripts devoted to
tracking timing, formatting, and test completion. It will also enable us
to expand on the reporting of test completions. Further down the line,
this will also allow us to unify test case documentation.

Change-Id: I8e1f4bcea86b2c3b88cc6e42339c57dfce4d58f2
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476799
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2019-12-10 17:12:03 +00:00
Maciej Wawryk
2951c1010e Change fio version
Whole autotest fails on VM Fedora31
GH #1081
Fio version update to fio-3.15

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I3b91c426050eb30af6b58434b6219090c61a48ba
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476893
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-12-06 14:50:59 +00:00
Maciej Wawryk
029251878d test: Shellcheck - correct rule: Consider using { cmd1; cmd2; }
Correct shellcheck rule SC2129: Consider using { cmd1; cmd2; } >> file instead of individual redirects.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Ic692e9f78f2d3d8a3b27d702884cbe97c880727c
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/474959
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2019-11-22 13:51:15 +00:00
Ben Walker
1a903f9c52 opal: Don't use set +e to ignore errors.
For errors that we do want to ignore, use || true after the commands.
This allows crashes in the background target application to fail
the tests.

Change-Id: I1fcd711c17ad0a956b6778260b2db8c0b801584f
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/475156
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-11-21 10:05:45 +00:00
Tomasz Kulasek
8e70c6e320 test/asan: suppress memory leaks in external libfuse3
There are some leaks in libfuse3 that is external to SPDK.
With this patch, any leaks in libfuse3 will be suppressed.


==1944861==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 32 byte(s) in 2 object(s) allocated from:
    #0 0x7fc26a462048 in __interceptor_realloc (/lib64/libasan.so.5+0xf0048)
    #1 0x7fc269bfcfe2 in fuse_opt_add_arg (/lib64/libfuse3.so.3+0x19fe2)

Indirect leak of 10 byte(s) in 2 object(s) allocated from:
    #0 0x7fc26a3ade60 in strdup (/lib64/libasan.so.5+0x3be60)
    #1 0x7fc269bfcfc6 in fuse_opt_add_arg (/lib64/libfuse3.so.3+0x19fc6)

SUMMARY: AddressSanitizer: 42 byte(s) leaked in 4 allocation(s).


Appropriate patch solving this issue was sent to libfuse, but
not merged yet.


Change-Id: I66625e155b78082f2f2c9790bf3f3b48c3c04f33
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/475117
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2019-11-21 08:01:57 +00:00
Pawel Kaminski
074df1d896 test: Shellcheck - apply rule SC2155
Declare and assign separately to avoid masking return values.

Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Change-Id: Ib90598e4268911a3056e8a0baa7d541edaa29b91
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/472287
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-11-12 18:14:59 +00:00
Tomasz Zawadzki
9a25fc12bb test: add SPDK_BUILD_PACKAGE flag
Until now the autopackage.sh execution was tied to
RUN_NIGHTLY flag. This was useful to check for
errors on all jobs/systems during a nightly test.
By their nature nightly does not run on per-patch
basis.

This allows for issues to slip in and be noticed only
by the nightly job after merge occurred.

Adding separate SPDK_BUILD_PACKAGE flag, allows to
explicitly test packaging on single job on per-patch.
RUN_NIGHTLY was kept as trigger for autopackage.sh
to still test it on all systems by default.

Change-Id: I29925fd6256b218e24c24ebcc4974c65a9bd986c
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/473519
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2019-11-08 22:18:24 +00:00
Jim Harris
2be2b6eba5 test: add additional asan suppressions related to fio
We are already suppressing fio (not SPDK fio_plugin)
leaks in a couple of other places, which could likely
be causing the indirect leaks we are now going to
suppress here.

Fixes issue #1003.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ie5283280495e7155cda1a93d2bd3d48ffbb6cba7

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/473495
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: yidong0635 <dongx.yi@intel.com>
Reviewed-by: GangCao <gang.cao@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-11-07 22:53:21 +00:00
Pawel Kaminski
39e8a95c71 test: Shellcheck - apply rule SC2152
Can only return 0-255. Other data should be written to stdout.

Change-Id: Idb8b387f438121e6b6afe62840ddee752872d7d7
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/472605
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-11-06 14:48:14 +00:00
Tomasz Kulasek
efbcd259ac lib/nvme: nvme character device tests
This patch adds new script to verify CUSE functionality for NVMe
devices:

 1) Starts spdk_tgt application
 2) Attaches first found controller
 3) Enables NVMe cuse devices for a controller and namespaces
 4) Retrieves CUSE device names for controller and namespaces
 4) Tests operations on exposed namespace devices
 5) Tests operations on controller devices

NOTE: These tests requires at least one NVMe device with at least one
      namespace available.


Change-Id: I5f5a7c86f8aefa73f12f4727f7520f16a599985b
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/468828
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-10-25 09:07:51 +00:00
Tomasz Kulasek
a072421c90 test/vm_setup: add CUSE dependencies
Dependencies required to compile and run nvme-cli tests:
 1) fuse module must be installed,
 2) cuse module must be installed,
    a. Fedora requires to install "kernel-modules-extra" providing
       cuse module for a kernel,
 3) systemd-devel package to compile nvme-cli application

While namespace devices are provided as character devices via cuse
module, this patch also installs nvme-cli version accepting namespaces
as character devices.

Required changes are available for review here:

	https://review.gerrithub.io/c/spdk/nvme-cli/+/472250

After merging the changes will be available on spdk/nvme-cli
repository at 'nvme-cuse' branch.

NOTE: changes above are not required when controller with nsid is
      provided as an nvme-cli option,
      e.g.

          "nvme id-ns /dev/spdk/nvme0 -n 1"

      instead of:

          "nvme id-ns /dev/spdk/nvme0n1"

Change-Id: I413c33d9891c5a39a5388ea7407d798c138b0584
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/472024
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-10-24 23:43:59 +00:00
GangCao
3fbc84d306 fio/bdev: fix the AIO bdev intermittent IO failure
This is to solve below github issue:
https://github.com/spdk/spdk/issues/935

Change-Id: I9b90f58c2756a415916d54ea49e62dbb467db4a0
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/471428
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-10-24 17:32:33 +00:00
Chunyang Hui
2176f081dd test/opal: Enable OPAL test
Also add opal_revert_cleanup at the start of autotest.sh
because opal test might fail before revert and the drive
might be kept locked and will cause deny of service in
later test.

Change-Id: Icb9d571c59804daa559784cee457c7e804eff121
Signed-off-by: Chunyang Hui <chunyang.hui@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/471973
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-10-24 17:09:57 +00:00
Maciej Wawryk
7c4bb5e2c3 test: Shellcheck - correct rule: Argument mixes string and array
Correct shellcheck rule SC2145: Argument mixes string and
array. Use * or separate argument.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I45102ead2e97cc1d6b11c21f269e58a235055c35
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/470926
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-10-16 15:51:31 +00:00
Maciej Wawryk
f1131f1d19 test: Shellcheck - correct rule: loops over find output are fragile
Correct shellcheck rule SC2044: For loops over find output
are fragile. Use find -exec or a while read loop.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I3fd84ab60daab7c6971769ff4dee8a24d5d3e1bb
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/470746
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-10-16 15:51:31 +00:00
Ben Walker
a0889ece60 sock: Add a function to check if a socket is connected
This is useful for detecting sockets that have been disconnected
by the other end without reading data.

Change-Id: Ieb6529984d282d48373766d9f5555cf11720f19b
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/470513
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-10-15 16:35:09 +00:00
Xiaodong Liu
7fa15e28e4 configure: add --with-fuse option
blobfs_bdev module may contain functions related to
FUSE which will utilize libfuse3.
By default, it is diabled to compile blobfs_bdev module
with FUSE related functions. Running './configure' with
option '--with-fuse' can enable it.

Change-Id: I6552a6c04cc3412c739691630a7a481e0ae6b59c
Signed-off-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/470712
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-10-10 16:20:46 +00:00
Maciej Wawryk
1db9e184e7 test: Shellcheck - correct rule: surrounding quotes actually unquote this
Correct shellcheck rule SC2027: The surrounding quotes actually
unquote this. Remove or escape them.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Ib17ee98920790da6aa1c8e02abc58c9365bb0e91
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/470634
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-10-09 14:28:21 +00:00
Maciej Wawryk
a3f1ce1589 test: Shellcheck - correct rule: "Use '[:lower:]'"
Correct shellcheck rule SC2018: Use '[:lower:]' to support
accents and foreign alphabets.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: If11d20a36419366adafc19f173ebbec60b2bea77
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/470109
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-10-07 11:29:29 +00:00
Maciej Wawryk
d32c76f4bf test: Shellcheck - correct rule: "Use '[:upper:]'"
Correct shellcheck rule SC2019: Use '[:upper:]' to support
accents and foreign alphabets.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I9fbaa22ee39d52d5d3a9397ad007b7df295b49b3
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/469990
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-10-07 11:29:29 +00:00
Karol Latecki
f657669eaf test: don't create Python .pyc files
Python byte code files are created with root ownership
if autotest.sh is run with sudo. This can cause permission
problems when doing "rm" or "git clean".

Change-Id: I0a099dcd94732bb293aea1bf0278656db0afba5f
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/469864
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2019-10-01 14:49:02 +00:00
Ben Walker
ab854135ec test: waitforbdev now accepts alias names
Change-Id: I366657ce00e5f0e17c6e9a7510a7fc2314c2782a
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/469621
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-09-27 14:57:20 +00:00
Pawel Kaminski
5017a67fdc test: Shellcheck - apply rule SC2191
The = here is literal. To assign by index, use ( [index]=value )
with no spaces. To keep as literal, quote it.

Change-Id: I14cada0e8c132286959aa47a1d37ede5e025d85f
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/467863
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-09-26 16:13:18 +00:00
paul luse
793d34e5f5 test/scripts: fix minor grammar issues
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Iee7502d37e7359d22bb2d72411d6555d6607c3ff
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/468648
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-09-19 20:56:35 +00:00
Konrad Sztyber
29a9a3089a test/ftl: remove SPDK_TEST_BDEV_FTL
There were two variables controlling libftl's tests.  Only one of them
is actually needed (SPDK_TEST_FTL), so this patch removes the redundant
one.

Fixes #956

Change-Id: I125b9cb7d855351488b445da165be9e27f10f914
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/467932
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: yidong0635 <dongx.yi@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-09-19 04:15:42 +00:00
Karol Latecki
03578a3d25 scripts/vm_setup: add Arch Linux test dependencies install
Dependencies install with -i option works OK.

Additional dependencies with -t option:
- rocksdb - OK
- librxe - OK
- iscsi - OK
- ocf - OK
- flamegraph - OK
- tsocks - OK
- nvmecli - OK
- fio - compilation errors on version 3.3
	OK if updated to 3.15
- qemu - OK, but need to disable some error warnings:
	-Wno-error=stringop-truncation
	-Wno-error=deprecated-declarations
	-Wno-error=incompatible-pointer-types
	-Wno-error=format-truncation
- qat - not run
- vpp - not run
- libiscsi - compilation errors

Change-Id: Ifde26c1f1d35c8fe06e18ddba33fe2d054b6d2cf
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/466948
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Seth Howell <seth.howell@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-09-18 22:25:18 +00:00
Ben Walker
714a56466a test/nvmf: allow running as unprivileged user
Add a new test flag SPDK_RUN_NON_ROOT forcing some of the
SPDK applications to run as an unprivileged user. For now
we implement it in nvmf tests. If enabled, nvmf_tgt will
run as the user who invoked `sudo ./autotest.sh`.

Running SPDK as non-root has two major prerequisites right now:
 * there must be an IOMMU in the system in order to use PCI devices
 * DPDK version must be either < 18.05 or >= 19.08-rc1

Change-Id: If1ba9dfcc09fc6cac059867408a1b54eddecfb4b
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/454679
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-09-18 20:34:39 +00:00
Pawel Kaminski
38d4a2a2f2 test: Shellcheck - apply rule SC2209
Use var=$(command) to assign output (or quote to assign string).

Change-Id: I4067db46697c32f24e0b4df7d573929ee75757a7
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/467291
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-09-18 20:21:13 +00:00
Maciej Wawryk
2c49e910cb RPC: rename rpc get_bdevs to bdev_get_bdevs
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I899c05963f4dd5f29df0b10fc030c27029c2c246
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/468057
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-09-13 22:39:15 +00:00
Maciej Wawryk
54ad489a53 test/common: fix use braces
Fix SC1087: Use braces when expanding arrays,
e.g. ${array[idx]} (or ${var}[.. to quiet).

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I0ffa91da9847ee0805b664cba573346ab5b83f3a
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/466898
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-09-09 04:18:48 +00:00
Maciej Wawryk
0b3f378f7a RPC: rename rpc get_lvol_stores to bdev_lvol_get_lvstores
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I419488ba971d523fd5285a51d467fbac9dd218a2
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/466282
Reviewed-by: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-08-27 16:55:45 +00:00
Seth Howell
07fe6a43d6 mk: move the bdev modules under module directory.
This is more accurate to what they are, and will make defining library
dependencies much simpler. This change in directory does not affect the
final placement of naming of libraries at the end of time.

Change-Id: Ic48a9233dff564e39ce357a9ea0a111ea2b6414b
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/465454
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-08-22 16:29:49 +00:00
Karol Latecki
ee7024d609 test/vm_setup.sh: update QAT driver URL
Update to version currently used on test systems.

Change-Id: Idbc12f0dd089dbe4b0919a936eac8a36020c3994
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/465665
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-08-21 23:16:27 +00:00
Marcin Dziegielewski
28f4299fe7 test: move get_nvme_name_from_bdf function to autotest_common.sh
This function is useful for each test which use nvme device, so
it will be easier to keep it in autotest_common.sh file instead of
including also nvme.sh in each test.

Signed-off-by: Marcin Dziegielewski <marcin.dziegielewski@intel.com>
Change-Id: Ic2e186826211725b95f3ec3b455d23f43ea41b8f
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/464691
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-08-14 21:24:58 +00:00
Tomasz Zawadzki
baa49be957 test/bdev: fix fio test verification
When this test was ran, it reported for each job:
"fio: verification read phase will never start because write phase uses all of runtime"

There was only randwrite phase, without any read phase with verification.
Alternatievly we can verify a set number of blocks during write using
"verify_backlog" argument.

With this patch writes to all bdevs will be verfied in 1024 chunks.

Change-Id: Ie79d543badd10f3b3ca91b3b56d9d8cb0dd332b2
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/464646
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
2019-08-09 23:49:16 +00:00
Karol Latecki
0f0cd0a87b test: Use $((..)) instead of deprecated $[..]
Fix SC2007: Use $((..)) instead of deprecated $[..].
SC2007 removed from check_format.sh exclude list.

Change-Id: Ifd858857e461d785d6d6f101acca13c326ee637e
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/464172
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-08-08 21:27:40 +00:00
Karol Latecki
1ccc878e7f scripts: use -n instead of ! -z
Changing according to styling check done by ShellCheck.
Removing from check_format.sh exclusion list:
SC2236 - Use -n instead of ! -z
SC2070 - -n doesn't work with unquoted arguments. Quote or use [[ ]]

Change-Id: Ia9d645b9d0ce31b67c4de682395cf36f4ddc8d1f
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/463180
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-08-08 21:27:40 +00:00
Tomasz Kulasek
2f070beb79 net/vpp: set vpp to last official release v19.04.2
Before this patch scripts/docs based on tip of 19.04 VPP stable branch,
because VPP 19.04.2 was not yet released at the time of implementation merge.

Now that VPP 19.04.2 was released, we change to actual release tag.
In order to prevent using untested commit, if new patches were to be merged
on top of VPP 19.04 stable branch.

Change-Id: Id3584a13c48eba25b061ae7a6fc51e477a6425c5
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/463328
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-29 17:53:47 +00:00
Anil Veerabhadrappa
ed56a3d482 NVMe-oF Target: Add FC transport.
- New files and updates to existing SPDK files to add the NVMf-FC transport.
  - Depends on an existing low level driver library. This driver is not part of SPDK repository.
  - Makefile updates to build FC transport (using CONFIG_FC)
  - Update configure script for FC build.
  - New FC unit test for FC-LS commands.
  - Update unittest.sh to run FC unit test (when built).

Signed-off-by: John Barnard <john.barnard@broadcom.com>
Signed-off-by: Anil Veerabhadrappa <anil.veerabhadrappa@broadcom.com>
Change-Id: If31d4d25feab76c2dbe90a7faf71d465c2c3a354
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/450077
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-07-26 22:17:17 +00:00
Tomasz Kulasek
9307ff5a54 net/vpp: move to VPP 19.04
This patch updates net/vpp implementation from version VPP 19.01 to
VPP 19.04.

1. Some binary APIs are deprecated in 19.04 and message queue is used
   to handle control events:

   - vl_api_bind_sock_reply_t_handler by SESSION_CTRL_EVT_BOUND,
   - vl_api_unbind_sock_reply_t_handler by SESSION_CTRL_EVT_UNLISTEN_REPLY,
   - vl_api_accept_session_t_handler by SESSION_CTRL_EVT_ACCEPTED,
   - vl_api_connect_session_reply_t_handler by SESSION_CTRL_EVT_CONNECTED,
   - vl_api_disconnect_session_t_handler by SESSION_CTRL_EVT_DISCONNECTED,
   - vl_api_reset_session_t_handler by SESSION_CTRL_EVT_RESET

2. Fixes for Fedora 29/30:

   - added "-Wno-address-of-packed-member" (DPDK 19.02 fails to compile
     with gcc9.1),
   - force "-maes" compile flag for gcc9.1 to compile crypto_ia32 and
     crypto_ipsecmb plugins (gcc9.1 doesn't do that for -march=silvermont)
   - some minor fixes

3. Default path for VPP instalation is changed for test scripts from
   /usr/local/src/vpp to /usr/local/src/vpp-19.04 to avoid VPP version
   conflict.

Change-Id: I1d20ad7f138f5086ba7fab41d77d86f8139d038e
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459113
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-07-25 14:51:21 +00:00
Darek Stojaczyk
fa3e0a68c1 ut: move all spdk_env_* mocks to test_env.c
Cleanup.

Change-Id: If6c24ef6fcea5531007e6d4f10ee896939ffaa12
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/460906
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-07-18 04:21:08 +00:00
Wojciech Malikowski
1d717c7cd9 test/vmd: Basic VMD tests
Added basic tests for NVMe disks behind VMD.

Change-Id: I9613ca4aa813bab16f4085ac809dbdc8028f1cfc
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459639
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-18 03:52:37 +00:00
Tomasz Kulasek
3c006fe10b net/vpp: fix vpp 19.01 installation on fedora 29/30
Change-Id: I953add8d0c01317be041b83eddb4b62689badbf0
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/460940
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-07-12 07:48:09 +00:00
Ben Walker
48a6e736fd test: Install packaged QEMU in vm_setup.sh
We only need the fork for OCSSD tests. And eventually we won't
even need that.

Change-Id: I0f52c44f504435a3bab2f478664a88ba6acfe464
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459960
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-07-11 11:31:21 +00:00
Seth Howell
d19b101c41 test: add top level param for nvmf transport.
This is needed to create a job in jenkins that tests only the tcp
transport.

Change-Id: Iaa5a07617e9a241a9e4bd50105d5d5d1e849844e
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/460713
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2019-07-09 13:23:03 +00:00
Karol Latecki
772eb8ebbb scripts: replace egrep with grep -E
Calling egrep is non-standard and deprecated.
Use grep -E instead.

Change-Id: Ie5ce5faa641550c4fb08bb32796a46f2aeb76c61
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/460516
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-07-05 12:06:10 +00:00
Karol Latecki
cf090c6cb9 scripts: replace backticsk with dollar-parenthesis syntax
Scripts were using a mix of two approaches, lets unify that so
just dollar-parenthesis syntax is used.

Also update two scripts from spdk/test which we previously missed.

Change-Id: I429f9bc158076462b419fae597f716c329f9b7aa
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/460344
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-07-05 12:06:10 +00:00
Ziye Yang
b09bd95ad3 sock: update spdk_sock_group_add_sock
And also add spdk_sock_group_get_ctx function

Change-Id: I2a2a58b0588ff7d99d3538ea0a633a3b8c7a234b
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/454538
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
2019-07-04 08:21:05 +00:00
Tomasz Kulasek
29408ebaa9 test/vm_setup: install VPP 19.01.1
NOTE:

Due to some changes in VPP two cherry-picks from newer version of VPP
(at least 19.04):

	"sock api: allow to start client with no rx_thread"
		commit: 97dcf5bd26ca6de580943f5d39681f0144782c3d

	"dlmalloc: honor 8 byte alignment requests"
		commit: f5dc9fbf814865b31b52b20f5bf959e9ff818b25

These commits are already merged in VPP 19.04.

This patch includes also workaround for VPP 19.01.1 to prevent closing
sessions to already closed applications after timeout. It causes
intermittent segfaults.
This is temporal solution and should be solved with next releases of
VPP, but is required now to create more stable VPP environment.

Change-Id: If4b45b7159819cfd836dd7d50f333dbab2b38eab
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456462
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2019-06-27 08:23:08 +00:00
Tomasz Kulasek
14a62d67ac net/vpp: allow to compile with vpp
This patch prepares environment to allow compile applications and
libraries with VPP.

Change-Id: Icc067c17b57932062afa5e6a67791bd58de3a2cc
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456461
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2019-06-27 08:23:08 +00:00
Tomasz Kulasek
a44c743417 test/iscsi_tgt: make sure iscsi devices are ready
We have no guaranty that iscsiadm -m session shows all required iscsi
devices at some point of time after login. It may cause that not all
devices are used in further tests (e.g. in fio).

This patch allows to wait for a certain number of sessions after login.

Change-Id: I868cb8f1235dd04d88572bc8518e97de99a2c5e6
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458236
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-06-27 08:23:08 +00:00
Karol Latecki
c7abeb33b5 test/vm_setup.sh: add libudev-devel to install_qat
Missing dependency libudev-devel. If not installed then
./cofnigure fails with:

checking for libudev.h... no
configure: error:
    udev headers required

Change-Id: I53c8f522cbc8eb1e6b461fd78c0bf3305bfb09b9
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459323
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-06-26 08:05:49 +00:00
yidong0635
d2c169d51c vm_setup: change spdk source from gerrithub to github.
Our test destination code of spdk is rsynced from host.
former link address of spdk "https://review.gerrithub.io/spdk/spdk"
is not very fast and always failed .
So changed to "https://github.com/spdk/spdk.git" ,this spdk just for
excuting start testing command.

Change-Id: I26b5c1556d8068becd5c5a94067cc64d9d32b5e6
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459223
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-06-25 14:05:08 +00:00
Darek Stojaczyk
8258313465 test/common: refactor waitforlisten to try sending RPC commands
Instead of doing some OS-specific magic to detect if the
spdk app has already initialized, just try to send it an
RPC. This serves mostly as cleanup, but also simplifies
test output in cases where the spdk app could not be fully
initialized. Previous waitforlisten implementation would
return as soon as the rpc subsystem was initialized, but
the app could have still failed on e.g. bdev initialization
later on. Since waitforlisten() returned success, the
script could continue its execution and try to execute
some RPCs. The logs from the SPDK app and the bash script
could easily mix, rendering the entire output not so clear
to the user.

To fix it, just try to send some RPC commands inside
waitforlisten().

Change-Id: I33eaf362e3cc645f8ea3ee22fd48db1643442129
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457562
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-06-19 03:54:58 +00:00
yidong0635
48cec0b127 vm_setup: fix scan-build not found on ubuntu18
Report issue on ubuntu18 :
"spdk/autobuild.sh: line 55: hash: scan-build: not found"

Check that ubuntu18 should install this package:
apt-get install -y clang-tools, to fit version of ubuntu18
and later.

Move clang-analyzer which for fedora to vm_setup.sh as well.

Change-Id: Ia702c492f8b0f64705c7c15ee57a861ca14521f9
signed-off-by: yidong0635 <dongx.yi@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457554
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-06-18 12:55:25 +00:00
Tomasz Zawadzki
1891f2e5c5 test/asan: supress memory leaks in external libiscsi
There are some leaks in libiscsi that is external to SPDK.
With this patch, any leaks in libiscsi will be suppressed.

Please see examples of the leaks right now when running
bdevperf in initiator.sh test:

Direct leak of 176 byte(s) in 1 object(s) allocated from:
    #0 0x7fd64dddfe50 in calloc (/lib64/libasan.so.5+0xeee50)
    #1 0x7fd64c60edb4 in scsi_cdb_inquiry (/usr/lib64/iscsi/libiscsi.so.8+0x13db4)

Indirect leak of 256 byte(s) in 1 object(s) allocated from:
    #0 0x7fd64dddfe50 in calloc (/lib64/libasan.so.5+0xeee50)
    #1 0x7fd64c603a84  (/usr/lib64/iscsi/libiscsi.so.8+0x8a84)

Indirect leak of 52 byte(s) in 1 object(s) allocated from:
    #0 0x7fd64dddfe50 in calloc (/lib64/libasan.so.5+0xeee50)
    #1 0x7fd64c60d755 in scsi_malloc (/usr/lib64/iscsi/libiscsi.so.8+0x12755)
    #2 0x7fd64c6097f8  (/usr/lib64/iscsi/libiscsi.so.8+0xe7f8)

Change-Id: I80ef23b1464841f683344c7aab99f1658a46cd36
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456766
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-06-18 03:40:50 +00:00
Tomasz Zawadzki
1bed9c1f1a test/asan: preload ASAN for fio
ASAN needs to be LD_PRELOADed before SPDK fio_plugin
in order to analyze its code.
Just adding that will report any issues in fio binary as
well as the fio_plugin.
To prevent known fio leaks from affecting the results,
a suppression list for LeakSanitizer (used in conjunction with ASAN).

At this time the suppression list contains known leaks
for fio 3.3. The list might need adjustments as fio
version is updated.

Side note. Even though it is possible to specify directory
to ignore ("leak:/usr/src/fio/"). Which in theory should
suppress any leaks in fio. It has side effect of hiding
SPDK leaks as well, since the fio_plugins leaks are
seen as coming from /usr/src/fio/ioengines.c.

See below for examples of each suppressed error:

Direct leak of 42 byte(s) in 4 object(s) allocated from:
    #0 0x7f9d52f3e320 in strdup (/lib64/libasan.so.5+0x3b320)
    #1 0x41f267 in get_new_job /usr/src/fio/init.c:490

Direct leak of 914936 byte(s) in 10397 object(s) allocated from:
    #0 0x7f74422e8ea6 in __interceptor_calloc (/lib64/libasan.so.5+0x10dea6)
    #1 0x46402e in log_io_piece /usr/src/fio/iolog.c:214

Direct leak of 608 byte(s) in 19 object(s) allocated from:
    #0 0x7f74422e8ca8 in __interceptor_malloc (/lib64/libasan.so.5+0x10dca8)
    #1 0x44c4e1 in add_to_dump_list /usr/src/fio/parse.c:1039
    #2 0x44c4e1 in parse_option /usr/src/fio/parse.c:1098

Direct leak of 173 byte(s) in 20 object(s) allocated from:
    #0 0x7f744227153d in strdup (/lib64/libasan.so.5+0x9653d)
    #1 0x44b50d in __handle_option /usr/src/fio/parse.c:718

Indirect leak of 111925528 byte(s) in 1271881 object(s) allocated from:
    #0 0x7f74422e8ea6 in __interceptor_calloc (/lib64/libasan.so.5+0x10dea6)
    #1 0x46402e in log_io_piece /usr/src/fio/iolog.c:214

Indirect leak of 171 byte(s) in 19 object(s) allocated from:
    #0 0x7f744227153d in strdup (/lib64/libasan.so.5+0x9653d)
    #1 0x44c4ed in add_to_dump_list /usr/src/fio/parse.c:1040
    #2 0x44c4ed in parse_option /usr/src/fio/parse.c:1098

Indirect leak of 167 byte(s) in 19 object(s) allocated from:
    #0 0x7f744227153d in strdup (/lib64/libasan.so.5+0x9653d)
    #1 0x44c502 in add_to_dump_list /usr/src/fio/parse.c:1042
    #2 0x44c502 in parse_option /usr/src/fio/parse.c:1098

Change-Id: I9b5811993508421be50b12af160645c77ea93d7e
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456315
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-06-18 03:40:50 +00:00
Karol Latecki
fb9c4ee651 test: replace backticks with dollar-parenthesis syntax
Scripts were using a mix of two approaches, lets unify that so
just dollar-parenthesis syntax is used.

Change-Id: Id093b6c82d1f766ba6af13bed720977eceaa7ffc
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457744
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-06-17 07:19:03 +00:00
Darek Stojaczyk
fadcb08db3 autotest/common: define xtrace_restore as an alias
This doesn't make difference right now, but would be
useful if we printed $BASH_SOURCE or $LINENO in xtrace
(via PS4). Those two will now point to the original
line where xtrace_restore was called rather than
always the single line inside autotest_common.sh.

Change-Id: Idf3ac8d00ad9610960678351014334013149b88d
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456964
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-06-11 05:17:58 +00:00
Tomasz Zawadzki
9fda9814b7 test/asan: move LSAN_OPTIONS to autotest_common.sh
Further patches in series will modify the suppression file,
so this patch now creates such file in /var/tmp/.
Meanwhile adding the known false positive to the list.

Any known false positives or leaks in external libs/execs
should be added by adding further entries to the suppression
file.

Change-Id: I7b78d900a86c6eca0b41240fb34dc4f7ad597079
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456622
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-06-06 05:17:16 +00:00
Tomasz Zawadzki
3b9db6c403 test/fio: export single variable for running fio_plugins
This patch adds functions in autotest_common.sh,
fio_bdev() and fio_nvme() for their respective plugins.
It simplifies its usage throughout the scripts.

Additionally will help with expanding the fio cmd
line with nessecary changes only in single place.

This will be used in next patch in series to
LD_PRELOAD ASAN before fio_plugin.

Note: Did not implement changes for perf scripts,
since they are usualy run separately and didn't want
to affect those.
Similarly didn't change vhost initiator tests,
because the exported directories do not work for
inside the VM. Will need to think of a way to change it
there as well.

Change-Id: Idf750ee7aa9d5e30efc0ce430d15fa45ceccb203
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456314
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-06-06 05:17:16 +00:00
Shuhei Matsumoto
1554a344c9 env: Add free_space parameter to spdk_ring_enqueue
DPDK rte_ring_enqueue_bulk() has free_space parameter to return
the amount of space in the ring after enqueue operation has finished.
This parameter can be used to wait when the ring is almost full and
wake up when there is enough space available in the ring.

Hence we add free_space to spdk_ring_enqueue() and spdk_ring_enqueue()
passes it to rte_ring_enqueue_bulk() simply.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I9b9d6a5a097cf6dc4b97dfda7442f2c4b0aed4d3
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456734
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-06-05 05:23:58 +00:00
Jim Harris
02c1c5a69a test/nvmf: use --transport=rdma in nvmf.sh
We also no longer hardcode TEST_TRANSPORT=rdma.  Users
of the individual test scripts must now explicitly
pass --transport=rdma.  Support for tcp will be coming
in future patches.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I819d69cb0906eb27b692eb2755aca5085298d779

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456685
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-06-04 23:04:54 +00:00
Jim Harris
aa7a13981b test/nvmf: use TEST_TRANSPORT instead of 'rdma'
This prepares for using scripts for tcp testing as well.
Note that this patch just hardcoded TEST_TRANSPORT to
'rdma' for now.  An upcoming patch will require the
caller to pass --transport=rdma instead.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I085fdf51910aaf960959c71c73a187be41fd7ee8

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456667
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-06-04 23:04:54 +00:00
paul luse
975e461496 test/autotest_common: add new function waitforbdev()
Waits up to 2 seconds for a bdev to arrive, if not returns error.
In the next patch in this series this is used and seems like a good
general function.

Change-Id: I3402f34f3dad3d7373973dc785520a5c4e58cd14
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456091
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-06-04 05:20:51 +00:00
Tomasz Zawadzki
05dde5c296 test/autotest: expand PYTHONPATH to include SPDK RPC lib
When a new python script wanted to use RPC client or calls
from RPC lib from ./spdk/scripts/* it had to be created
within that directory.
By expanding PYTHONPATH in autotest_common, now any python
scripts can use that library.

First example of this is located in:
./spdk/test/bdev/bdevio/tests.py

Change-Id: Idf3e5891c3815a84c70525ab9338100acbfa4a43
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455219
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-05-30 21:58:23 +00:00
paul luse
8a43cd27e6 test/autotest: update how config options are set in autotest
NOTE: pulling the vbdevcompress change into its own patch...

While enabling reduce testing in autotest a few issues were found
in autotest in that some options were being set based on the existence
of a header without checking the test flag, if one existed. Then at
the end of adding options, specific test flags were checked for 0 and
the configure updated to --without.  So, to make this easier to read,
the options are organized in sections and where there is a header
dependency as well as a test flag, both are chcekced at the same time.

Change-Id: I8cdc6914906500bbac63528eb541e2b235feb797
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455137
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-05-30 17:56:54 +00:00
yidong0635
0a48fd0601 vm_setup:install right packages on ubuntu for SoftRoCE
Ubuntu18 integrates librxe to rdma-core, libibverbs-dev no longer ships infiniband/driver.h.
Don't compile librxe on ubuntu18, install package rdma-core instead. ubuntu16 keeps the old
method. Otherwise, there's no NIC can be found for SoftRoCE failed.

Change-Id: Ib639b96a4229c79f2b27fda7b981d7a805f808cb
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455336
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-05-29 21:31:36 +00:00
Jim Harris
6ee44c694e rpc: rename RPC get_rpc_methods to rpc_get_methods
Make the old name a deprecated alias.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ibbf50676e0d989b67121e465fc140f94faec46ed

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453033
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-05-27 12:52:53 +00:00
Seth Howell
f40d245180 autotest_common.sh: move exporting flag values to the top of the file.
Recently, we moved the check for running an installed version of DPDK
above the point where we export the SPDK_* test flags. This resulted in
SPDK_RUN_INSTALLED_DPDK being undefined when trying to check fi we
should run against an installed DPDK. This just caused autotest_common
to print an error message every time it was loaded. I don't think that
it actually caused any errant behavior but this gets rid of the error
message. It's also probably a good idea to keep these definitions as
close to the top of the file as possible.

Change-Id: I5aadbe5c925ecf1ac92926b75c8c043aab73b36b
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455456
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-05-24 23:43:37 +00:00
Jim Harris
6a8a1b6bef test: always parse common script args
Good suggestion from Darek - let's just always
parse common script args from autotest_common.sh.
These arguments follow common arg naming conventions
(i.e. --iso) so there's no harm just doing this for
any test that sources autotest_common.sh.  This has
the nice effect of not requiring scripts to
explicitly call this function.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Id89b68c22557a5a771be407873d0e57843f0d05a

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455552
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-05-24 05:36:46 +00:00
Jim Harris
390b364146 test: add parse_common_script_args function
iscsi test scripts can now take two arguments -
"iso" and then the sock type (posix or vpp).  They
need to be in that specific order too.  nvmf test
scripts also support "iso" and we want to add
the transport type (rdma or tcp) as well.  Even further
out, we may want to use a sock type for nvmf, i.e.
tcp transport with vpp.

We also have the iscsi_tgt fio_remove_nvme.sh test
that does both iscsi and nvmf.

So to make this all work a bit nicer, add a new
function called parse_common_script_args that
will take the command line arguments to a script
and set the appropriate variables, including defaults
when a specific parameter isn't specified.  We will
use getopt-like behavior for this also, instead of
enforcing a specific parameter order.  Then a script
could be called like this:

test/nvmf/target/shutdown.sh --iso --transport=tcp --sock=vpp

Individual test scripts then just need to do this
after sourcing autotest_common.sh:

parse_common_script_args $@

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ifb8d7666384991482a2d425e26ffa7525b9ac15a

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455283
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-05-22 23:43:51 +00:00
yidong0635
2dac00b9a4 vm_setup: fix pop-up dialogs in auto installing package
Installing this package "open-isns-utils" always with pop-up dialogs,
we don't want to these dialog, so add --no-install-suggests
--no-install-recommends to prevent it.

Use sudo apt-get remove --purge * to simulate.

Change-Id: I0b892b9e0c88c82ab9461a92e71dc0d9823ecaf9
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455333
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-05-22 15:37:39 +00:00
Ben Walker
e036215fe9 thread: Add a mechanism to exit a lightweight thread
Lightweight threads may now be exited by calling
spdk_thread_exit() within the thread. The framework
polling the thread can release the resources associated
with that lightweight thread by calling
spdk_thread_destroy().

Change-Id: I6586b9d22556b3874fb113ce5402c6b1f371786e
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455319
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-05-22 04:23:17 +00:00
Darek Stojaczyk
7c00bdab97 autotest: silence errors about missing timing.txt
Currently autotest prints the following at the start
of each run:

> grep: /[...]/output/timing.txt: No such file or directory

No harm is done and the file is created right after
the grep, but the message might be misleading. We get
rid of it now by ensuring the file always exists before
grepping it.

Change-Id: I271b68479a6fddcbe098c2657fd7c4dc39e6e50a
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/454708
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-05-16 20:53:49 +00:00
Karol Latecki
1839045ba2 test/common: add bc to vm_setup.sh
bc is currently used in iSCSI qos.sh test, but is not listed
as a pre-requisite in vm_setup.sh
Also it's a good alternative to using sed or awk for some
simple math operations.

Change-Id: I18742c0dedcdcc5eeebd310eb5760e226a163af7
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453998
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-05-15 18:46:53 +00:00
Ben Walker
49bff95f09 test: Add test_sock.c, a mocked up socket library
Instead of linking to the real sock library in the unit tests,
link to a mocked up version like we do with the env.

Change-Id: I9607f135c6af86e09fa52badf44897f90ce94521
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/454492
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-05-15 18:45:46 +00:00
Darek Stojaczyk
cff52180c4 autotest: don't source the configuration file in every test script
Most of our bash test scripts source autotest_common.sh
to be able to use some autotest-specific functions like
timing_enter(). The same test scripts allow specifying
custom command line parameters without actually realizing
that those parameters can be potentially picked up by
autotest_common.sh as well.

For example, if particular nvmf tests are run in "isolation"
mode by being executed with the first param set to "iso",
and there is a file named "iso" in the current dir, that
file will be sourced. This could be bad.

In this patch we stop sourcing or even processing $1 in
autotest_common.sh. Instead, the test configuration will
be sourced just once from autobuild.sh, autopackage.sh
and autotest.sh. If the user wants to run particular test
scripts manually, he should source an SPDK test configuration
by himself - manually as well. In most cases he won't even
have two, as only a few test scripts depend on SPDK_*
variables.

Note that we still have to setup the default values for
SPDK_* variables in autotest_common.sh because some of
our test scripts actually depend on them:

> if [ $SPDK_TEST_RBD -eq 1 ]; then ...

Because it lacks any type of quotes around SPDK_TEST_RBD,
it will print the following message when that variable
is unset:

> /bin/bash: line 0: [: -eq: unary operator expected

It doesn't trigger any error ($? == 0), but can be still
a bit misleading in the script output.

Change-Id: I350045d8582d66fe1ed7697d4bcbba324cb541ad
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453876
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-05-10 22:08:13 +00:00
Darek Stojaczyk
7612ac476e autotest/common: disable xtrace for the autotest_common.sh itself
Most of our test scripts source autotest_common.sh at the
very top of the file, and - since autotest_common.sh
enables xtraces inside - the first thing we'll see in test
output are likely xtraces from sourcing the configuration
file and setting up $config_params. Even a simplest test
script will produce a ton of output this way, making the
entire output unreadable. This is especially annoying when
the test script is run manually, so we get rid of it now.

All the SPDK autotest options and $config_params will be
printed elsewhere anyway.

Change-Id: I6d9c8e8ba261b632ffbb7d6d26d84eb7ccfb2ed8
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453875
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-05-10 22:08:13 +00:00
Darek Stojaczyk
85e70f4e21 autotest/common: set MAKEFLAGS at the top of the file
This way all the ./configure parameters are setup in one
place without any unrelated code in the middle.

Change-Id: I1cd8dc05e326b9ab3e8e51c193dd422fd8b5a00b
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453873
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-05-10 22:08:13 +00:00
Darek Stojaczyk
24a67e29de autotest/common: export DEFAULT_RPC_ADDR
We use that variable only inside autotest_common.sh so far,
but hopefully that's going to change now. Our tests still
hardcode /var/tmp/spdk.sock everywhere at the moment.

Change-Id: Iad862cb7674a8108d2a07f35b5298d21e009c5a1
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453872
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-05-10 22:08:13 +00:00
Darek Stojaczyk
caa76c39f9 autotest/common: reorder generic stuff to the top of the file
Put it all together at the top of the file so that
the file is less chaotic overall.

Change-Id: Ia92511e590692bbe31d0f626e9542374eaaab442
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453871
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-05-10 22:08:13 +00:00
Darek Stojaczyk
e20401c838 autotest/common: allow testing ASAN on BSD
SPDK_RUN_ASAN was only effective on Linux so far,
even though libasan is available on BSD as well.
Let's fix that.

Change-Id: I58a1796f736616882f797313e22055122b050b57
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453870
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-05-10 22:08:13 +00:00
Darek Stojaczyk
8d2247e2c4 autotest/common: produce xtrace output on xtrace_restore
This way we can consistently track when xtrace in our
scripts is enabled or disabled.

> [...]
> xtrace_disable
> PREV_BASH_OPTS=ehxBE
> set +x
> xtrace_enable
> [...]

Change-Id: I2e813dc2a237a4620ea72d26a22a3c8cbeb269f9
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453248
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-05-10 22:08:13 +00:00
Darek Stojaczyk
3b660ea857 autotest/common: factor out xtrace disable/restore
Factor out `set -x` and +x to separate functions. Changing
xtraces is not so trivial and we'll be improving it later
on. We can't factor out the code as is due to local variables,
so we already simplify it a bit in this patch.

Change-Id: Iecbf5cedf821b7b1b71da933ceb158761881a843
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453246
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-05-10 22:08:13 +00:00
Jim Harris
94e792934b test/nvme: remove pmap call
We added this a long time back when debugging some
multi-process failures.  This adds a lot of output
to the log which isn't helpful anymore.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I4d46b744f30b32fddedc3dae32f40c077717a1bb

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453909
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-05-10 19:43:31 +00:00
Darek Stojaczyk
d193d9873b autotest/common: fail if the requested test suite can't be run
Now that SPDK_TEST_* default to 0, we can decidedly say
that if the requested test suite can't be run on your
system due to unmet dependencies (e.g. missing system pkg),
you will get an error.

Previously we would silently disable that test - e.g. when
someone ran with SPDK_TEST_RBD=1 but had no librbd system pkg
on his system, the SPDK autotest would return success without
running any RBD tests. That was very counter-intuitive.

We still silently disable a lot of tests inside our particular
test scripts. Hopefully we'll stop doing that eventually.

Change-Id: Ib400ba772c095b4d65fa8893a0669f3d8c3984e6
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453712
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-05-09 04:32:23 +00:00
Darek Stojaczyk
005e18a881 autotest/common: silence hash output
We've got a bunch of `if hash` checks that generate
unnecessary output, e.g.:

$ if ! hash ceph; then [...]
/var/jenkins/workspace/Other_systems/centos7_autotest/
spdk/test/common/autotest_common.sh: line 28: hash: ceph: not found

We now silence that output in cases where we do expect
the binary to be unavailable and handle that case, but
we hold off in cases where `if hash` is just a sanity
check (e.g. `hash sgdisk` in part_dev_by_gpt() which
can make the entire function return immediately and do
nothing).

Change-Id: I7cbfada858aa86d57cbe182ea7f9b222debc8b88
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453207
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-05-08 10:58:51 +00:00
Jim Harris
ef6832ef56 test/nvme: kill stub with -9 when a failure occurs
This ensures the stub gets killed so that we can
make forward progress on collecting a backtrace.

This requires adding an optional parameter to the
kill_stub function to specify a signal.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ib46d07b6651c6f35382f38c97901def03f28289d

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452891
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-05-07 05:11:54 +00:00
Pawel Niedzwiecki
0560976d51 test/common: Disable autotest options by default.
By default most of the tests are enabled, if we want to
run only tests that interests us, we need to disable each flag
separatly.
Build SPDK without additional dependencies and exit autotest.sh
early if no test configuration file is provided.

Change-Id: Id4752689479d9325242364c39c89d1236bab0344
Signed-off-by: Pawel Niedzwiecki <pawelx.niedzwiecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/443762
Reviewed-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-05-06 17:43:08 +00:00
Ben Walker
df9129300c thread: Call spdk_thread_lib_init in unit tests
Change-Id: Ib05b9f2ea6582a96b0719a2834c2ad6896fb744e
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453011
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-05-06 17:34:31 +00:00
Jim Harris
b118729feb test: add helper function to wait until file is created
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ib706727c680471334b90960148ee1a637561cf64

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452805
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-05-02 23:31:50 +00:00
Jim Harris
af25204404 bdev/nvme: always enable FTL
FTL doesn't have any kind of special package requirements.
It is getting pretty good traction in the community, so
let's enable it by default.

Note that we will disable FTL on FreeBSD.  FTL uses
CIRCLEQ which is not available on FreeBSD.  Let's not
spend time trying to get FTL to work on FreeBSD until
there's a demand to do so.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I15525b6c4e6ee52f49adf74d55f9484fe08a6dcc

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452752
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-05-02 08:41:56 +00:00
Pawel Niedzwiecki
970a86fb06 scripts: vm_setup.sh add install vpp for ubuntu
Change-Id: I32ead1d453d261bcc5230fcead9b1fb6a0fd9187
Signed-off-by: Pawel Niedzwiecki <pawelx.niedzwiecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/451387
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
2019-05-01 17:44:29 +00:00
Ben Walker
5d0b5e2ce9 thread: Add cpumask parameter to spdk_thread_create
Allow users to optionally specify an affinity mask when
creating a thread. This isn't currently used, but we want
the API to be in its final form for the next release.

Change-Id: I7bd05e921ece6d8d5f61775bd14286f6a58f267f
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/451683
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-04-23 20:22:55 +00:00
Vitaliy Mysak
9e10f5fafa scripts: vm_setup.sh support INSTALL for Ubuntu
This commit adds functionality for installing most of the packages,
 which were previously installed for Fedora only,
 for Ubuntu as well

Some packages are not going to be installed by the script:
For ubuntu 16
  * libpmempool
  * open-isns-utils
  * perl-open
For ubuntu 18
  * perl-open

Remaining packages deserve their own patches since they are not so easy to
  install on Ubuntu

Change-Id: Ic5b744193ff4df46ab4e4fb3783e0515ccdbfecb
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/425388
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Paweł Niedźwiecki <pawelx.niedzwiecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-04-22 16:52:56 +00:00
Vitaliy Mysak
bef5509d4f scripts: vmsetup.sh fix pkgdep usage
vmsetup invokes pkgdep.sh with -i flags, but that flag is not supported.
This patch changes the call to just `pkgdep.sh`.

Change-Id: I1eb92b9ddfce1ea2d7ef450fd795f8f93fb198ec
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/450260
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-04-08 23:21:37 +00:00
Karol Latecki
ab1622aa35 autotest_common: delete ceph_raw.img after test
Recent change of ceph_raw.img size from 10G to 4G breaks
tests on patches which are not rebased or which use
previous SPKD releases.

Change-Id: I9dd79756e3cc7949593cd1c16d4735542d1fb32d
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449469
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-04-01 23:59:44 +00:00
Piotr Pelplinski
ae11723ae9 Fix autotest_common.sh to find only core files.
Currently the scripts deletes also files located
in ocf submoudle named core.py. This patch adds regex to avoid that.

Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Change-Id: I97a21c4860cb54f37f28641dc9cd47adcf2fd939

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/448519
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-03-21 17:43:11 +00:00
Darek Stojaczyk
40678c15ed env: add spdk_realloc()
It's a copy of spdk_dma_realloc() matching the new
spdk_malloc() naming convention.

Change-Id: I8e1b24aa0bae064392fe0f4ebc08c5723d9a5f1a
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/448169
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-03-20 11:13:36 +00:00
Darek Stojaczyk
27c42e313f nvme: don't rely on phys_addr retrieved from spdk_malloc()
The phys_addr param in spdk_*malloc() is about to be
deprecated, so use a separate spdk_vtophys() call to
retrieve physical addresses.

This patch also adds error checks against SPDK_VTOPHYS_ERROR.
The error handling paths are already there to account for
spdk_*malloc() failures themselves, so reuse them in case
of vtophys failures.

Change-Id: I377636e66b8c570d013c1bb2021f04bce4e6c0ce
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/416998
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-03-20 01:06:09 +00:00
yidong0635
d47468a151 test scripts: add rpm-build package in vm_setup.sh
This package is needed in vm_setup.sh for vpp,
without this will report "rpmbuild: command not found".

Signed-off-by: yidong0635 <dongx.yi@intel.com>
Change-Id: Ic00a55addc940e359aae17b45e58b9a736b08fd3
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/447282
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Piotr Pelpliński <piotr.pelplinski@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-03-18 12:55:02 +00:00
paul luse
5133af0921 test scripts: add -E to sudo git clone in vm_setup.sh
Running as a non-privileged user, sudo git clone won't work
without the -E because it won't get the proxy settings, for those
using a proxy.

Change-Id: I8b471abf0066312b0eb63449d00a5647737c0136
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/446162
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-03-04 19:12:10 +00:00
Jim Harris
577b667ac3 test/env: add UNIT_TEST_NO_PCI_ADDR
Similar to UNIT_TEST_NO_VTOPHYS, we need this since
a future patch will result in some of the pci_addr
functions getting linked in via a new dependency
between memory.c and init.c.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I61bd4e172ee70a5dcb99c8c6fc1fb19070a2a7cd

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/446460
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-03-04 14:00:16 +00:00
Vitaliy Mysak
34bdceabb8 ocf: update autotest to use OCF submodule
Make autotest.sh use OCF submodule as target

Change-Id: I1f6411c1b143d3c53ffa405a5bf01fdbcd4e9868
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.gerrithub.io/c/444844
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-02-27 17:26:51 +00:00
Vitaliy Mysak
80ff32cfcc scripts: vm_setup.sh fix OCF github repo path
Fix wrong url for git repo for OCF

This patch is connected to issue #670

Change-Id: I030889089a4b0433517dd909246a3bc16b67c71b
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.gerrithub.io/c/445249
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-02-21 18:02:11 +00:00
Jim Harris
f74d069ec0 env_dpdk: move vtophys.c contents to memory.c
CPU profiling on workloads with intensive vtophys
operations (i.e. very small CB-DMA transfers) exposed
overhead introduce by spdk_vtophys having to call
spdk_mem_map_translate in a different compilation
unit.  Let's just move the vtophys.c contents into
memory.c so that spdk_vtophys can inline
spdk_mem_map_translate and avoid this extra overhead.

This of course breaks the memory and vtophys unit
tests, so some additional changes are needed there
to keep everything linking correctly.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I295ed5f441d3eec7abdbc9d881c49d2174ec9f48

Reviewed-on: https://review.gerrithub.io/c/444975
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-02-20 16:41:46 +00:00
Wojciech Malikowski
0699216fac ftl: extended nightly tests
Change-Id: I8dc3e895f6b360f3ad81f157db502dd58629c5e6
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/442878
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-02-12 19:45:17 +00:00
Ben Walker
8abfb06e31 thread: Optionally allow the current time to be passed to
spdk_thread_poll()

This is an optimization if the calling function already knows the
current time.

Change-Id: I1645e08e7475ba6345a44e0f9d4b297a79f6c3c2
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/443634
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-02-11 22:58:45 +00:00
Darek Stojaczyk
0b4da45dbc vm_setup.sh: add iptables dependency
We started to use iptables in patch 21bd94275
(libsock: add functional tests) but never added
the package dependency.

Change-Id: I651f2545a11f546f8b47f9759fbaed3a141f0928
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/443597
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-02-11 19:23:55 +00:00
paul luse
d9d4e40dd2 bdev/compress: Add configure option and build dependencies
Includes the required DPDK dependencies for SPDK block Reduce aka
Compression.

Change-Id: Ic1ea3cbeb9373a7700f6f0c2a3194d65d6a34a41
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/429523
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-02-11 19:23:17 +00:00
Wojciech Malikowski
a4968634f1 test/common: spdk_mempool mock configurable element size
Change-Id: Ia93b4d7ea3ce77b8c45d34eedb5201ca5ea34688
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/443248
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-02-06 17:51:53 +00:00
Seth Howell
5232a73f38 autotest_common: add waitforblk_disconnect
This is the functional opposite of waitforblk.

Change-Id: I5e9f77c9ea05b83b8dbfbf49bb3baa2ae8cc1086
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/443316
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-02-06 17:33:18 +00:00
Wojciech Malikowski
eccf549159 test/common: Added spdk_ring_count mock
Change-Id: Iefb4c76f2db5416d185556fa8c84a36291e72d3e
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/443237
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-02-05 18:02:15 +00:00
Darek Stojaczyk
8604e568cb autotest: introduce SPDK_RUN_FUNCTIONAL_TEST
Introduced a new variable to run functional tests.
It's enabled by default, and can be manually disabled
on systems where e.g. only unit tests are run.

SPDK_RUN_FUNCTIONAL_TEST is a supplement to SPDK_UNITTEST.
The two are completely independent - both can be enabled,
disabled, or run in any combination.

The new variable is prefixed SPDK_RUN_ as it aligns nicely
with SPDK_RUN_CHECK_FORMAT, SPDK_RUN_VALGRIND, and
SPDK_RUN_ASAN, all of which control how much is tested.
SPDK_UNITTEST should eventually follow the same pattern
as well.

This gives us 2 layers of configuration:
SPDK_TEST_* <- what is tested
SPDK_RUN_* <- how it is tested

The following would run UT+ASAN for FTL and BlobFS, without
running their functional tests:

```
SPDK_RUN_FUNCTIONAL_TEST=0
SPDK_RUN_ASAN=1
SPDK_TEST_UNITTEST=1
SPDK_TEST_FTL=1
SPDK_TEST_BLOBFS=1
```

Change-Id: I9e592fa41aa2df8e246eca2bb9161b6da6832130
Signed-off-by: Seth Howell <seth.howell@intel.com>
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/442327
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-02-04 19:19:36 +00:00
Wojciech Malikowski
ca558b6163 bdev_ftl: Added FTL bdev functional tests
This patch introduces functional tests for FTL bdev.

The tests cover various I/O workflows and check data integrity. Several
scripts have been added to test the FTL library:
 * generate_config.sh - prepares configuration scripts for specified
 device
 * restore.sh - tests restoring device's state from the SSD
 * fio.sh - runs tests based on fio and fio_plugin

The tests are run from autotest.sh when the SPDK_TEST_BDEV_FTL flag is
set.

Change-Id: I561d99ed35fe91eadd3756789cc99afe2da8c1db
Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/431330
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2019-01-29 17:33:10 +00:00
Darek Stojaczyk
6903c407a5 test/common: suppress waitforlisten output on BSD
We currently use:

`./rpc.py 1>&2 2>/dev/null`

And apparently our stdout redirection gets treated as
a python argument, which is not what we intended. This
caused waitforlisten function to print some unnecessary
messages.

To fix, we could either wrap python with extra brackets
`$(./rpc.py) 1>&2 2>/dev/null` or use &>/dev/null. We
go with the latter.

Fixes #616

Change-Id: I49d2380392435447b7b93259055c1d02c4b9c157
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/442262
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2019-01-29 14:00:10 +00:00
Ziye Yang
552e21cce6 spdk: Add ISA-L support with related crc32 function
In SPDK, we will build isa-l with no shared option
and then integrate it into SPDK. And we do not need
to install isal in the system libaries.

Note: ocf build in autobuild.sh now needs to build
include/spdk/config.h before building the ocf library,
to ensure that header is available in a clean build
environment.

Change-Id: I3f0ce6932b386de17a77cf5bfdfd738b22417e2d
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Signed-off-by: paul luse <paul.e.luse@intel.com>
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Reviewed-on: https://review.gerrithub.io/c/441279
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Chunyang Hui <chunyang.hui@intel.com>
2019-01-29 08:31:00 +00:00
Vitaliy Mysak
ec2e259402 scripts: vm_setup.sh add OCF dependencies
Add auto installation procedure for OCF dependencies

Change-Id: I75fc1b26e0eb7132a310cff62cad61bb7c3b05ee
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.gerrithub.io/c/439359
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2019-01-29 06:10:36 +00:00
Vitaliy Mysak
2fde729fd7 OCF: tests: add integrity test
Add SPDK_TEST_CAS suite
Add basic integrity test with multithread-multidevice fio with cache
  modes WT and PT

Change-Id: If2916558894d9f7efd9628eb99f7338c90b312b1
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.gerrithub.io/c/439346
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-01-28 09:41:52 +00:00
yidong0635
310df31b88 test: add installing python3-pandas in vm_setup.sh
Add installing python3-pandas.
put it into vm_setup.sh, this package is needed by autorun_post.py

Change-Id: I76c98c54f16be69879ac05d5128a049fc693e455
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Reviewed-on: https://review.gerrithub.io/c/435968
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2019-01-18 15:55:32 +00:00
Wojciech Malikowski
4a4e928524 test/common: Changed spdk_ring mock to be thread safe
This change is required for unit tests that use spdk_ring mock
in multithreaded scenarios.

Change-Id: I8f5e83de69a685078cbb01e55daac650f6ae1db1
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/440721
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-01-17 20:58:57 +00:00
Ben Walker
9cba82b955 thread: Rename spdk_free_thread to spdk_thread_exit
This name more closely resembles pthread_exit, which is a
closer analogy to how the new threading library works.

Change-Id: I68b04509f3ff8e94b8688804a7e5661155a3ecd1
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/440597
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2019-01-17 11:24:38 +00:00
Ben Walker
b1c79d722b thread: Rename spdk_allocate_thread to spdk_thread_create
This mirrors pthread_create, which works more closely
to the new style where SPDK libraries can spawn their
own threads.

Change-Id: Ic524c4c35bcf7c1611e4f261ebb64b98ac5a5a1b
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/440596
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-01-17 11:24:38 +00:00
Ben Walker
1068e22da5 thread: spdk_free_thread now takes a thread parameter
Instead of implicitly grabbing the thread from the thread
local variable, make it explicit.

Change-Id: I733fad06181439e12b1e71a4829b84e7b64e2468
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/440595
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2019-01-17 11:24:38 +00:00
Ben Walker
6728c4f4c0 thread: Eliminate function pointers in spdk_allocate_thread
These are no longer used by anything.

Change-Id: I0db6bc88e4dc945ff4f64df2ac410e1d00a669c1
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/437601
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-01-17 11:24:38 +00:00
Ben Walker
5977aad8f7 thread: Eliminate use of pthread_self and thread_ids
SPDK threads are no longer mapped 1:1 to system threads. They are
instead identified by the memory address of the spdk_thread object.

Change-Id: I417f8376cebd2ee94f624f4436e6394b51486063
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/437999
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-01-15 16:53:12 +00:00
Ben Walker
9bef42f284 test: Use ut_multithread framework in bdev/pmem
Change-Id: Ib9de83db429e4ae6f8e0cc988ca161cebdb27c9c
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/437599
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-01-15 16:53:12 +00:00
Wojciech Malikowski
ef2e614cef ftl: Initial implementation
This patch adds core FTL (flash translation layer) submodules.

On regular SSDs the FTL is part of firmware, whereas Open Channel
enables moving it to the host and allows for having full controll
of data placement on the device.

Main functionalities added:
 * logical to physical address map
 * read / write buffer cache
 * wear-leveling
 * bad block management

Change-Id: I5c28aa277b212734bd4b1f71ae386b3d6f8c3715
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Signed-off-by: Jakub Radtke <jakub.radtke@intel.com>
Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/431322
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-01-11 09:15:39 +00:00
Darek Stojaczyk
8c4ed83b49 vtophys: add length parameter to the vtophys function
This follows the same trend as the mem_map APIs.

Currently, most of the spdk_vtophys() callers manually
detect physically noncontiguous buffers to split them
into multiple physically contiguous chunks. This patch
is a first step towards encapsulating most of that logic
in a single place - in spdk_vtophys() itself.

This patch doesn't change any functionality on its own,
it only extends the API.

Change-Id: I16faa9dea270c370f2a814cd399f59055b5ccc3d
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/438449
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: wuzhouhui <wuzhouhui@kingsoft.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-01-09 08:24:59 +00:00
Karol Latecki
2a9667066a test/vm_setup.sh: Add sshpass
sshpass used in test/nvme/hotplug.sh test but not present
in vm_setup.sh dependencies.

Change-Id: Ic9d7f137f7694e5318f6f412b1e7a73536a82f8c
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/438788
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2019-01-04 00:52:28 +00:00
yidong0635
6907c36f2c test/common: add stop.sh in ceph setup testting
Add stop.sh before start.sh,to avoid reporting resource busy
it could lead to start failed.

Change-Id: I1756adaad31937a7a4b95fdc488f2334360d4ac2
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Reviewed-on: https://review.gerrithub.io/437616
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-12-19 17:04:28 +00:00
Ben Walker
536beb1306 test: Use regular assert in allocate_threads
This allows us to call it during test set up instead of
only within a unit test.

Change-Id: I2daae8d630729b5e0712057766a3cba8812109e9
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/437479
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-12-18 23:31:08 +00:00
Ben Walker
7b94053827 thread: Rename spdk_thread_msg to spdk_msg_fn
This is the callback type used for message passing. The
old name is easy to confuse with the callback to pass
a message to a thread or the upcoming callback to
spawn a thread.

Change-Id: I5fd63b57c4be2a4262a197850e6de4901be03ee7
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/435941
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-12-18 23:31:08 +00:00
Ben Walker
605e530a4e thread: Use TLS to accelerate thread look up
Change-Id: I8136db265c9cb8d61de4845ce6eaff2351b5b597
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/435939
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-12-14 18:54:03 +00:00
Karol Latecki
0fc2933906 test/vm_setup.sh: update qemu to version 3.0.0
Use Qemu v.3.0.0 from spdk/qemu.

Change-Id: I44ecdac5f591a6096ec8450dd70044a7ba6628e0
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/436476
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-12-10 22:02:39 +00:00
Seth Howell
1500792f52 test: modify coredump maximums to 5G
Having unlimited size core dumps is not a good idea on a lot of
machines. Modify it to 5GB.

Change-Id: I1d52bfa9f2450e2d8f824c3b86aa2ad5fe4579c3
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/436412
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-12-10 10:03:04 +00:00
Darek Stojaczyk
76a7517caa pci: remove virtio-specific attach functions
As a part of cleanup they're replaced by a device-agnostic
attach API, which is easier for us to manage.

Change-Id: I2ec68f20ba209f02ee5c2de4b6fe5330a4bc0853
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/436480
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-12-07 17:24:19 +00:00
Darek Stojaczyk
41130653b7 pci: remove ioat-specific attach functions
As a part of cleanup they're replaced by a device-agnostic
attach API, which is easier for us to manage.

Change-Id: Ia92bd8f4525712bd27ade16ead67435c5e0fbe7a
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/436479
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-12-07 17:24:19 +00:00
Darek Stojaczyk
576f08f732 pci: remove nvme-specific attach functions
As a part of cleanup they're replaced by a device-agnostic
attach API, which is easier for us to manage.

Change-Id: I7558590e41e5c580a130a6aba7ae4f7dcff58da8
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/436478
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-12-07 17:24:19 +00:00
Pawel Wodkowski
db1236ef10 autotest_common: waitforlisten - add 20s timeout and FreeBSD support
In case the process we are waiting for fail to start
listening on RPC socket the test script will stuck till the build times
out. Fix this by adding 20s timeout and 0.5s sleep in the while
loop.

For systems where there is no ip command and netstat output is missing
'Status' column for Unix sockets (like FreeBSD) call the get_rpc_methods
RPC command to check if process is listening.


Change-Id: Ia8b06af7875b65a7fd8be65cf55e92881f6f95db
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/433102
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-12-05 00:35:35 +00:00
Pawel Wodkowski
f751ea1723 autotest_common.sh: enable trace flag only if it was anabled before
We are too verbose in many places. Some scripts don't want to be traced
all the way, instad they enable tracing only for some parts and
autotest_common.sh just do 'set -x' in many places.

This patch save 'x' flag on function enter and restore it at exit.

Change-Id: I39b3d3dd3f711e1131e476f9d322d9e1b097ad12
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/432604
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-12-05 00:35:35 +00:00
yidong0635
4b88ed5687 test: add a needed package in VM setup list.
filesystem need support btrfs.

Change-Id: Ie82d87dfd98ffcbeb674313e1ef4c5097763ad40
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Reviewed-on: https://review.gerrithub.io/435425
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2018-12-04 15:46:56 +00:00
Pawel Wodkowski
bf9dbae56b tests: wait for each start and stop nbd disk RPC
This workaround is needed to stop EBUSY errors from NBD ioctls(). This
can be fixed in NBD bdev code but till that point lets merge this
workaround.

Change-Id: Ic1546963538beda4d4409cef93062103e2a23e34
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/434040
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-11-21 07:28:31 +00:00
Seth Howell
be5d98000f test: adjust scripts for 01.org qat driver
Change-Id: I1cf9d27d05f09c199ef35066f2e62480025d3feb
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/433185
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-11-15 20:08:16 +00:00
Ben Walker
3c98150821 thread: Update unit test to use new threading logic
No longer override the send_msg implementation.

This updates ut_multithread.c, which has minor ripple
effects into bdev_ut.c. But that unit test is otherwise
unchanged.

Change-Id: I2fd30a1010bdff0a810d376d985ab1b8a2b22fb9
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/424262
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-11-15 18:17:34 +00:00
Jim Harris
8c50e8c106 test: handle align=0 in test_env spdk_malloc
DPDK treats align=0 as effectively align=8.  But the
test_env spdk_malloc uses posix_memalign and passes
the align as-is.  posix_memalign requires a non-zero
align parameter however.  So change the test_env
implementation to match DPDK.

This fixes libreduce unit tests, since it does some
allocations with default alignment.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: If2fbad0bcc3bc92b50b50f21d7e893e0ba05e325

Reviewed-on: https://review.gerrithub.io/433083
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-11-14 09:58:18 +00:00
Ed rodriguez
9ba446a25c test: Add support to configure ipsec git repo
Also make ipsec configuration and install optional
  in pkgdep.sh.

Change-Id: I0c973398cebce0f50c704b21f6397e8d996ba9f9
Signed-off-by: Ed rodriguez <edwinr@netapp.com>
Signed-off-by: John Meneghini <johnm@netapp.com>
Reviewed-on: https://review.gerrithub.io/432424
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-11-12 22:27:35 +00:00
Piotr Pelplinski
dadd2a6dc0 bdev: add unit tests for double buffering in bdev modules
Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Change-Id: Idcfb328a26fc85f3c144da9c27086a71c00d0d46
Reviewed-on: https://review.gerrithub.io/427830
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
2018-11-08 23:11:17 +00:00
Jim Harris
970228038f reduce: add configure option
--with-reduce required to build reduce.  This depends on
libpmem being installed.

We still need to work out details in pkgdep.sh and
vm_setup.sh.  Some distributions like Ubuntu still
require configuring extra package repositories to
get libpmem packages.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I4e056ce1da9a1fecb4458f8f5e7ff5d61c422533

Reviewed-on: https://review.gerrithub.io/430646
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-11-08 00:01:47 +00:00
Tomasz Kulasek
21bd942751 libsock: add functional tests
Change-Id: I3c86c1b4a9c8ae3f18264f33b15d647e4081a12c
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-on: https://review.gerrithub.io/421911
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-11-07 17:08:27 +00:00
Pawel Wodkowski
ea0744f435 autotest: match exact address in waitforlisten function
Current behaviour will match anything that contain address. Change this
to match whole address only.

Change-Id: I6343fb91ed624cba41adbc33d711fb48c2f167a1
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/431530
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-11-02 17:55:25 +00:00
Seth Howell
38d2cb200d vm_setup.sh: small fixes
Refactor the printout of autorun-spdk.conf and fix the path for
pkgdep.sh (the previous path was based on an old assumption)

Change-Id: Id95e0f80f835dd1b055a94a6bfdbffcce029b8af
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/430843
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-10-31 21:55:49 +00:00
Seth Howell
33df76dc93 test/bdev: change crypto device conf based on environment
Also, remove bdev_svc memory limit to prevent it from hanging while
handling discover_bdevs calls.

Change-Id: I2868839f40931b964d49a90a86dfb391e2247e30
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/429535
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-10-24 23:29:28 +00:00
Seth Howell
2772d86c23 test: remove -g from discover_bdevs call.
Forcing bdev_svc to use a single hugetlbfs file causes the crypto bdev
to fail to properly allocate qpairs for qat devices. The qpairs come
back from the dpdk function queue_dma_zone_reserve misaligned.

Change-Id: Ib0c95856031ca89f6528ea6d39ef68e03e2f9009
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/430372
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-10-22 22:33:13 +00:00
Seth Howell
d63d4d5a37 scripts: add a qat card setup script
This script should be called when testing the crypto module. It will
configure any qat cards with class code 37c8 for use with the spdk
crypto bdev module.

Change-Id: I233069fbe9ba3c7a6463f52443234f9fe2c9a1f7
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/428549
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-10-22 22:33:13 +00:00
Darek Stojaczyk
d68ee5b431 autotest/freebsd: fix contigmem.ko path when using external dpdk
Change-Id: I9997c848ecb2514fbaf0ae9f4a0eb80d5e1522bc
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/428900
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-10-17 15:21:05 +00:00
Darek Stojaczyk
d94846c7f0 autotest: remove usages of deprecated --with-raid
The RAID module is no longer marked experimental and
is now built by default. Using '--with-raid' was giving
us an additional deprecation message:

> --with-raid option ignored and is now deprecated.
> RAID module is always enabled.

Change-Id: If9cc65a4900fc93d07fa88a3ebe3498001916285
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/429179
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-10-12 22:47:50 +00:00
Ben Walker
d705ab93e1 Check file permissions in the check_format script
Change-Id: I572b24b2f4f149333f2e4b04f6f4961de9c23373
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/427539
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-10-04 23:08:12 +00:00
Karol Latecki
de2cd83828 test/vm_setup.sh: add missing assume yes option
Dnf install for perl-generators had "-y" option missing

Change-Id: Ib12edcc11902f73824d7ed27d89d00b1631b51f2
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/426077
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-10-03 16:16:03 +00:00
Pawel Wodkowski
ae2b2f64ea log: add a way to turn on and off stack unwinding in logs
As a side effect, if SPDK_LOG_DISABLED is used in spdk_log() as log
level then no message will be printed.

Change-Id: I2d57b60a5a310a9ef2a1187a81088d0acf828742
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/425105
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-10-02 22:09:27 +00:00
Seth Howell
30974c72ed test/common: change branch of nvme-cli
Update the branch to point at our code rebased on the most recent
mainline release of nvme-cli. This branch includes fixes for
compatibility with gcc 8.

Change-Id: Ie1bdb046d1e24e832bda585dd9841619451ec5f5
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/426985
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2018-10-02 22:08:12 +00:00
Seth Howell
f1b3df73b6 test/common: Configure repo locations inline in vm_setup
This allows a userto run the script without specifying a configuration
file. Removing this dependency opens us up to things like calling
vm_setup directly from a vagrantfile without having to first copy the
spdk source over.

Change-Id: I72074a445f8befc714c03cab57a2da539350c092
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/426944
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-10-01 16:15:22 +00:00
Karol Latecki
f2d9e5144a scripts/vagrant: automate vhost test VM creation
Enable users to automatically create VM image dependency
needed to run vhost tests.

Current state of Vagrantfile_vhost_vm is only valid
for Ubuntu 16.04 and 18.04.

Change-Id: I9cdf46c1db7fcd55cfda0dd7db7ff5570610c5ee
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/422948
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-09-28 18:43:42 +00:00
Seth Howell
a0a92be2df test/common: parallelize vm_setup.sh
We can significantly speed up the vm_setup.sh process by installing
dependencies in parallel.

Change-Id: I424efdf6da26a7f1394fbdca1cb6129e7329fc16
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/426981
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-09-28 18:29:29 +00:00
Seth Howell
5ae7353072 test/common: remove directory switching from vm_setup.sh
By keeping the main shell in the same directory the whole time, we will
be able to parallelize a lot of the work of cloning and making
dependencies. This should significantly reduce make time to run this
script on a target machine.

Change-Id: I5efbd9c8f50b95411cbe7be69905b08df557344e
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/426980
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-09-28 18:29:29 +00:00
Seth Howell
0c04c729ba vm_setup.sh: move all installations under a single command
This significantly decreases the amount of time it takes to run this
script.

Change-Id: I5424cd7c9f3caf49a529916d528c0cc3a7af8838
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/426947
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-09-28 18:29:29 +00:00
Seth Howell
99850ca7d5 vm_setup: add nvme-cli repo
The custom nvme-cli code is used in both the NVMe and NVMe-oF tests.

Change-Id: I51aab2050e1921aac899970e7abd3f1cd64836ba
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/425642
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-09-20 21:55:18 +00:00
Pawel Wodkowski
4dc0ef6f45 scripts: fix executable attribute for includes
rpmbuild pointed out that this is wrong to have executable files without
shebang. Taking the opportunity and fix this for other files too.

Change-Id: Ib21f436672150edc0aff511bff2eb6839870cf79
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/425382
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-09-18 16:34:45 +00:00
Pawel Kaminski
ca12cb3097 scripts: add config converter
Add a new script to enable converting
old INI config files to the new JSON-RPC
config file format.
This prepares for deprecating the INI
config file file format in the future.

Change-Id: I0f2bfa9a585ce3537772b662d8e4028ab08a554d
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.gerrithub.io/420457
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2018-09-18 15:24:40 +00:00
Paul Luse
51606ed402 bdev: Add crypto virtual bdev module
Initial support for softare AESNI_MB DPDK driver only.

Have tested (both aesni and QAT seprately and concurrrently) on underlying NVMe devices
with bdevio and a bdevperf script that runs IOs from 512B to 128K each with Q depths from
1 to 512 in powers of 2 for 30 seconds each run.

QAT can be included in the code (but not makefile) and marked as experimental
until we are ready to test in CI.  It works well on 2 systems but is a big PITA to get
the hardware setup and configured for use with DPDK (IMHO).

Change-Id: If518c3df8e74e00efa18afdf194824c5e69778fc
Signed-off-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/403107
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-09-17 21:23:14 +00:00
Seth Howell
92a5fb593c vm_setup: add sshfs
sshfs is used to mount the shared filesystem used for communication
between the pool and agents.

Change-Id: I7c2a216ac57e394915f99bccf39d62d2ee17e5fb
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/425513
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-09-15 00:29:53 +00:00
Karol Latecki
af32aa1b4e test/nvmf: enable tracepoints for nvmf_tgt
Enable tracepoints and move shm file to
the output directory in case of crash.
Cleanup all shm files after successful run.

Change-Id: Ie46c86ddda71015038624f9b2b5fd4a3e1d7f61f
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/425078
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paweł Niedźwiecki <pawelx.niedzwiecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-09-12 16:58:58 +00:00
Karol Latecki
f0a2cc8247 test/common: check if pid is alive before kill
Need to check if process still exists before killing it.
Otherwise if process crashed and no longer exists kill will
return with "no such process" error and prevent other steps
in trap from executing.

Change-Id: I4402d06a4fafa09dc13fba4ef9ebeb2b2fc686bb
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/425353
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paweł Niedźwiecki <pawelx.niedzwiecki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-09-12 16:58:58 +00:00
Seth Howell
a3595cbec1 autobuild: add shared object build to tests
This was not being tested on every patch previously.

Change-Id: I0a1756f3709da6608f66a2160962ed9b0fb38c7b
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/424890
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-09-10 17:50:06 +00:00
Seth Howell
8b45e46698 test/common: update dpdk directory information
The default directory for dpdk on centOS7, fedora28, and ubuntu 16.04 is
/usr/share/dpdk/x86_64-default-linuxapp-gcc. update autotest_common.sh
to reflect this. Also, add a build pool flag to control whether or not
to ignore that direcory.

Change-Id: I618b902ed04dd4a9f63c5bd2014a9fbb7fcbf91b
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/424596
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-09-05 19:48:22 +00:00
Jim Harris
86d77e2eb6 bdev: account for missed qos timeslice timeouts
There could be cases (especially in virtualized and/or test
environments) where we could accumulate significant skew in
the timeslice frequency.  Rather than depend on the application
framework to try to guarantee the rate of timeslice poller
callbacks, keep track internally of the last time the poller
was invoked.  If/when we accumulate and detect skew equivalent
to one or more timeslices, increase the allowed IO and bandwidth
of the next timeslice to accomodate.

Since bdev poller now calls spdk_get_ticks() to do accounting,
this patch also fixes up the increment_time() unit test function
and the test env layer to properly increment the fake TSC.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iba301ddc0fb3d02042106a8bf6e4a6a9a84dc263

Reviewed-on: https://review.gerrithub.io/423580
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: GangCao <gang.cao@intel.com>
2018-09-02 23:59:27 +00:00
zkhatami88
eb6a2cb8f5 nvme:replacing spdk_dma_malloc with spdk_z/malloc
Change-Id: I755d36495e7d8e55c26c4103a9520102223798ef
Signed-off-by: zkhatami88 <z.khatami88@gmail.com>
Reviewed-on: https://review.gerrithub.io/409960
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-08-31 17:42:29 +00:00
Jim Harris
b5d1b4f445 test: don't print backtrace when errexit is off
Some test scripts may temporarily disable errexit -
for example, trying a command several times until
it succeeds.  In these cases we don't want to
print a backtrace every time that command fails.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I481f50b046eb13e23f7048672281ec7d62a573a2

Reviewed-on: https://review.gerrithub.io/423921
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-08-30 04:47:39 +00:00
Seth Howell
aca4ba4dd1 test/vm_setup: fix yum-deprectad for fc28 with vpp
Change-Id: I6608725c031daff6f6d59ced0c327f19b6eea979
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/423755
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-08-29 18:46:02 +00:00
Pawel Niedzwiecki
341fff6397 test/iscsi: Add installation of socat in vm_setup.sh configuration
script

socat is used in iscsi_tgt's libsock test

Change-Id: I027930a99e58ff9862a540d5eed3b1a50de28a93
Signed-off-by: Pawel Niedzwiecki <pawelx.niedzwiecki@intel.com>
Reviewed-on: https://review.gerrithub.io/422255
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-08-15 18:09:46 +00:00
Chen Wang
1f0bff73df test: refactor the run_test function to add detailed information
1.Refactor the run_test function which used to add detailed information
  during run test suites and test cases.
2.Refactor the lvol feature test scripts to make sure their log is the same.
3.Users can use "run_test suite command" to run test suites
  and use "run_test case command" to run test cases.
4.Update the vhost and lvol test as example.

Change-Id: I7b6387019a861bd1c4f89b9a7712e53150aea8fa
Signed-off-by: Chen Wang <chenx.wang@intel.com>
Reviewed-on: https://review.gerrithub.io/403610
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-08-15 17:24:01 +00:00
WangHaiLiang
8680bea106 test/common: remove "rbd pool init"
Fix the issue #384.
Avoid false information:unknown option 'pool init rbd'

Change-Id: I6aee52d1b3a1b4ca250963922d01d7f66151832f
Signed-off-by: WangHaiLiang <hailiangx.e.wang@intel.com>
Reviewed-on: https://review.gerrithub.io/421482
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Liang Yan <liang.z.yan@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-08-10 03:10:23 +00:00
Ben Walker
f56b230063 test: Simplify the mock library
Unify the regular mocks and the pointer mocks. Simplify
several of the #defines.

Change-Id: Ica8c69dbb70a685a55b5961b73fd7872f451c305
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/418884
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-08-09 17:18:23 +00:00
Karol Latecki
a1acca9274 test/common: fix typo in vm_setup.sh
sgdisk is a part of package gdisk, not a package itself

Change-Id: I214f9720aaeda88006c40e915960c2e1d5cd87b3
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/421496
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-08-08 15:54:02 +00:00
Karol Latecki
372b0d4931 test/common: add elfutils to vm_setup
Some functions in autotest rely on eu-readelf binary but only
elfutils-devel package was installed in vm_setup.sh script.

Change-Id: Id51ed672a0ef309121e1dddc2a1c8a360f3940c5
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/421189
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-08-06 16:29:19 +00:00
Karol Latecki
b606baabbc test/common: fix for eu-readelf path over 80 chars
Readelf psargs field holds file path up to 80 chars.
Above 80 chars path is truncated and --wide option does not
enable displaying longer path.

Added checking if the path was not truncated by testing if
file path from psargs exists. If it does not - check rest of
eu-readelf output for a matching path and use it instead.

Change-Id: Ia89b41115803e2463e2030e739121ce1fa8039e0
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/421100
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-08-06 16:29:19 +00:00
Pawel Niedzwiecki
a278b14296 test/qemu: Update qemu version to spdk-2.12
Change-Id: I2784166f6b92440ab4979d4ccfd588f91252bbaa
Signed-off-by: Pawel Niedzwiecki <pawelx.niedzwiecki@intel.com>
Reviewed-on: https://review.gerrithub.io/420878
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-07-31 19:59:10 +00:00
Jim Harris
ecb44872b6 test: add some basic raid tests
RAID is experimental for 18.07, pending iov support
which is needed for vhost.  But we need to at least
build RAID in our tests, and run some basic
bdev tests to ensure it doesn't quickly become stale.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I68bb45df6d00e31dc96ed8bc902b61a5ef699ba2

Reviewed-on: https://review.gerrithub.io/420678
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-07-30 14:59:45 +00:00
Seth Howell
8749a9a591 vm_setup.py: add sgdisk
Used in the ceph tests.

Change-Id: Iea7bda61e792542aae2dff09b0ac321bd4073cc1
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/420499
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-07-26 22:40:22 +00:00
Paul Luse
911c042df8 test: add mock and new function to test_env.c
Need to mock spdk_mempool_get() to test code path when it fails.
Also added spdk_mempool_get_bulk() to test env.

Change-Id: I7f64230c9841215a404149e9a48ad4bf8a63822c
Signed-off-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/420110
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-07-26 22:08:42 +00:00
Pawel Wodkowski
848daf274b log: add backtrace option
Add '--enable-log-bt=lvl' option to show simple backtrace in following
form:

thread.c: 346:spdk_io_device_register: *ERROR*: io_device 0xf2ef80
already registered
*ERROR*: === BACKTRACE START ===
*ERROR*:   1: spdk_io_device_register() at 0x6d64df
*ERROR*:   2:  spdk_copy_engine_initialize() at 0x71059c
*ERROR*:   3:   spdk_copy_engine_subsystem_initialize() at 0x572ed0
*ERROR*:   4:    spdk_subsystem_init_next() at 0x6ca756
*ERROR*:   5:     spdk_subsystem_verify() at 0x6caba7
*ERROR*:   6:      _spdk_event_queue_run_batch() at 0x6c1ffa
*ERROR*:   7:       _spdk_reactor_run() at 0x6c5349
*ERROR*:   8:        spdk_reactors_start() at 0x6c784f
*ERROR*:   9:         spdk_app_start() at 0x6bf18e
*ERROR*: === BACKTRACE END ===

This adds additional libunwind dependency so don't enable by default.

Change-Id: Ice93d7571a000d8a57d2fedda7670c9a0b6ff7b7
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/419726
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-07-25 15:48:25 +00:00
Shuhei Matsumoto
d891b398a3 test/nbdjson: Fix solid/intermittent CI failure
In part_dev_by_gpt(), stop_nbd_disk is missing and it looks that
this has caused solid/intermittent CI failure in the nbdjson test.

Change-Id: I46b14fa3b7c91547b88f49b28d0cba252b00e8a2
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/420123
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-by: Paweł Niedźwiecki <pawelx.niedzwiecki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-07-23 19:31:41 +00:00
Daniel Verkamp
fbb481c2c6 test: add SPDK_TEST_NVME_CLI autotest flag
As requested in GitHub issue #361, we should have a way to control
whether the SPDK-patched nvme-cli tests are executed.

Also add a report_test_completion call for the NVMe-oF nvme-cli test
so we can easily track whether it executed (the local PCIe nvme-cli
test already had a completion).

Change-Id: I70c26bcf2489ccfa563214a9723603aa705be7ab
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/418866
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: John Meneghini <johnm@netapp.com>
Reviewed-by: <ed.rodriguez@netapp.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2018-07-10 23:18:27 +00:00
Ed Rodriguez
2e0df7813e test: vm_setup.sh localization
Change-Id: Ief3a1fbde5d8eabefa87c39feefe3d496e3a5369
Signed-off-by: Ed Rodriguez <ed.rodriguez@netapp.com>
Signed-off-by: John Meneghini <johnm@netapp.com>
Reviewed-on: https://review.gerrithub.io/418236
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2018-07-10 22:47:16 +00:00
John Meneghini
806f43b26d test: improvements to vm_setup.sh and pkgdep.sh
Change-Id: Ie433edd7c72f84b62c17bacebd31d61ffa6f30ab
Signed-off-by: Ed Rodriguez <ed.rodriguez@netapp.com>
Signed-off-by: John Meneghini <johnm@netapp.com>
Reviewed-on: https://review.gerrithub.io/416052
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2018-07-10 22:47:16 +00:00
Paul Luse
58d8a4564b test: add ability to mock spdk_dma_malloc()
Used in crypto unit tests.

Change-Id: I7d7e31e9030c238cb539b9fcf0936e588c1a4211
Signed-off-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/418320
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-07-10 21:31:39 +00:00
Seth Howell
5b072d3739 test/config: add some comments to vm_setup
Just clarifying why and where some packages are used.

Change-Id: I36f17856a76ed1eb3d88054da2692beee9e569f2
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/418230
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-07-06 22:15:24 +00:00
Seth Howell
b3d61e5d01 test/config: add command line args to vm_setup.sh
Change-Id: Iae7a2206d3d64834da03226ebeba5e91c635703d
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/416885
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-07-02 23:09:23 +00:00
John Meneghini
f04277f0bf test: remove more sys_sgsw dependencies from test scripts
- To override the default /home/sys_sgsw dependency
   add DEPENDENCY_DIR to ~/autorun-spdk.conf

  E.g.: DEPENDENCY_DIR=/home/vagrant

 - To override the default HUGEPAGES use
   add HUGEMEM to ~/autorun-spdk.conf

  E.g.: HUGEMEM=1024

Change-Id: Ib8db9d7d053ae319fe4c725159742875468d47f0
Signed-off-by: John Meneghini <johnm@netapp.com>
Signed-off-by: Ed Rodriguez <ed.rodriguez@netapp.com>
Reviewed-on: https://review.gerrithub.io/415907
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-07-02 21:32:39 +00:00
Dariusz Stojaczyk
8681eecd9a env: introduce spdk_memzone_reserve_aligned
This is an spdk_memzone_reserve variant with additional
alignment parameter. Now that memzones must be used for
physically contiguous memory, it will become extremely useful.

Change-Id: Ie48d682217e0e2f5c859a1603bb8a81fd2a7d7df
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/416978
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-07-02 19:13:23 +00:00
Ben Walker
2fac05e919 test: Fix inaccurate mock of poller behavior
This exposes that one of the tests was not expecting
the correct behavior, so also fix that.

Change-Id: Idb73b3ea74950b2e6f959a40e5740375cb76b8c7
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/417364
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2018-07-02 19:12:45 +00:00
Ben Walker
f443bd40e4 test: Add spdk_mempool_put_bulk to test_env.c
Change-Id: Id5d22fb9c9e70c432b120676cbbf040b0c8e6c49
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/417363
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2018-07-02 19:12:45 +00:00
Ben Walker
ef8dfe1111 test: Mock spdk_ring operations in test_env.c
Change-Id: I1ef5cc1c58d536ccd6774c31cb633af494103e24
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/417361
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2018-07-02 19:12:45 +00:00
Pawel Kaminski
93fca0e79f test/json: Base bdevs test for bdev subsystem.
- load spdk_tgt with bdevs
   - check if json config is properly saved and loaded
   - check if all bdevs are properly loaded
   - check if configuration of spdk_tgt is properly set

Change-Id: I89226bc9c05880e73523fb189116e7433a513244
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.gerrithub.io/406326
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-06-29 16:27:57 +00:00
Seth Howell
a33a7b44a3 test/config: minor error fixes to vm_setup.sh
The name of the rpm source for open-iscsi changes between fedora
releases. Also, there was a typo on the tsocks check.

Change-Id: I0f9810c51c35c3efd5616fad303d08b02d9e2218
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/416882
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2018-06-28 17:38:48 +00:00
Jim Harris
9d04d0efd5 env_dpdk: pick base-virtaddr that ASAN won't override
ASAN has some memory address regions that it will not
allow applications to use for mappings (including mmap).
So when specifying --base-virtaddr to DPDK for
shared/mutli-process mappings, pick an address that
ASAN will allow the application to map.

Ref: https://github.com/google/sanitizers/wiki/AddressSanitizerAlgorithm

We will still disable ASLR while using the test stub for now.
Maybe we can eliminate this too in the future, but for now just
modify the autotest_common.sh comment since we've figured out
how to keep ASAN from messing with the mmap hint.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ie24fd35bd22aa3ceab6271e8936775b157e5c330
Reviewed-on: https://review.gerrithub.io/416420
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
2018-06-21 22:49:40 +00:00
Jim Harris
696d9eaed0 autotest: reduce HUGEMEM allocation to 2GB on FreeBSD
The FreeBSD contigmem driver has problems allocating
contiguous memory after a large number of unloads/reloads.
Since we run a very limited subset of SPDK tests on
FreeBSD, reduce the HUGEMEM allocation from 8GB to 2GB
to help alleviate the problem and reduce the frequency
of test failures due to contigmem panics.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I777153a6ad4f89311bb4e2f50bf69ee09e628eea

Reviewed-on: https://review.gerrithub.io/416214
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-06-20 22:48:30 +00:00
Seth Howell
714776c028 test/config: Call pkgdep.sh from vm_setup
vm_setup.sh now calls pkgdep.sh to ensure that the basic packages needed
to compile SPDK are installed. Pkgdep remains a separate script because
it is supposed to contain the minimal set of packages for building SPDK.
while vm_setup is used to configure a complete testing environment.

Change-Id: I4038f522c66ad5b2c55ed73b1170f796b2c94e27
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/415895
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-06-20 00:18:40 +00:00
Daniel Verkamp
42b6dd9399 pkgdep: add nasm in preparation for intel-ipsec-mb
A nasm or yasm compatible assembler will be required to build the
intel-ipsec-mb library needed for the DPDK crypto framework.

Change-Id: I3e1822069452208aa796caf84ddd9995fcac2cd6
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/415065
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2018-06-13 23:32:31 +00:00
Jim Harris
e30535fecf thread: add spdk_thread_get_count()
This will return the number of currently allocated threads.

Modify the bdev_io caching code to use this new API since these
caches are really per-thread, not per-core.  SPDK does not support
dynamic threading yet, but once it does, we will want callers to be
using functions from the thread API - not counting the number of cores
allocated to the application.

spdk_env_get_core_count may still be useful as a helper function, so
it is still kept and not deprecated.  For example, app.c uses it to
print the number of cores allocated to the application.  bdevperf should
eventually be modified to use spdk_thread_get_count, but holding off on
that for now until spdk_event_allocate() uses threads instead of a reactor
lcore to specify where the event should be executed.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I5a30e3e825e6821da87d3927a2443768dfd740f4

Reviewed-on: https://review.gerrithub.io/414709
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Madhu Pai <mpai@netapp.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-06-13 00:38:41 +00:00
Ben Walker
a83f91c29a thread: Replace #include of io_channel.h with thread.h
Change-Id: I6babd4cf990bf19b510db88bdfb0ca81e29d9252
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/414700
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Madhu Pai <mpai@netapp.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-06-12 15:24:07 +00:00
Jim Harris
f007d46c5a test: disable RBD and PMDK tests if not installed
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Id3152f911a9a1fb4b3ac9d8be56570a665c079f1
Reviewed-on: https://review.gerrithub.io/414281
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-06-11 23:25:55 +00:00
Daniel Verkamp
700477897a test/vm_setup.sh: update nvml-tools -> pmempool
This package has been renamed to reflect the new upstream project name.

Change-Id: I71f6bfaf36fb1fe7dc4cb050e3d7faad1097994e
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/411781
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-05-22 21:04:10 +00:00
Seth Howell
27d7bac9a0 test/common: Make common pmap call more generic
This is to support older versions of pmap (specifically procps v 3.2.8
which ships with centOS 6) which don't support the -p and -X options.

Change-Id: Ic8d4b94985c165e8bf224204af48b5d856aa2403
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/411755
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-05-21 17:09:23 +00:00
Seth Howell
86ee572b79 autobuild: On FreeBSD, install the contigmem driver
The contigmem driver is built with dpdk and frequently updated. We
should copy this driver each time we build dpdk in order to maintain
support for FreeBSD

kldload checks the sysctl variable kern.module_path when deciding which
modules to load. Since the values stored in this variable are not
determined by our program, I chose to copy the module to both common
directories for kernel modules in FreeBSD.

Change-Id: I2d439a9cbac5bebb79e6ee48bec47409bb215be9
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/409250
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-05-03 19:49:53 +00:00
Daniel Verkamp
d08917d89e ioat: remove ioat kperf kernel driver test harness
This driver was added to allow benchmarking of the SPDK user-mode I/OAT
driver vs. the Linux kernel I/OAT driver; however, this isn't a
particularly interesting test, since the kernel I/OAT driver is totally
inaccessible from user-mode code (it is only exposed to the in-kernel
dmaengine framework).

Maintaining an out-of-tree kernel driver for the sole purpose of
benchmarking is out of scope for the SPDK project, so remove the kperf
driver and test harness.

This can always be retrieved from git history if needed later.

Change-Id: I0ced6e8a88de2cf09a6c0970dfef0ae8f357f193
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/408900
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-04-30 17:11:53 +00:00
Tomasz Zawadzki
5bafc240e5 test/iscsi: utilize network namespaces in iSCSI tests
Network namespaces are used to assure that kernel
is not routing packets within host stack,
but they go through veth interfaces.

This patch serves as a base for future VPP test changes,
where namespaces are used as well.

Change-Id: Ic7b82b0a0837bca2e16774fde244348a691fe056
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/405641
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2018-04-26 17:10:36 +00:00
Seth Howell
9acb995ede test/common: Don't override autorun-spdk.conf
Change-Id: I1b75ffaccdf909ce1ce11f4464efdd539a7754a7
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/408410
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-04-24 21:53:51 +00:00
Seth Howell
fc0dc65adc test/common: Fix flamegraph typo in vm_setup
Change-Id: I27b9e118fe760bc5e7425fad3731f54b618fb347
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/408409
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-04-20 17:49:36 -04:00
Tomasz Zawadzki
0629b01d21 test/rbd: add configurable monitoring IP address for ceph
Ceph monitor IP address was always 127.0.0.1, with this change
it can be configured at setup time.

Since each test might prefer to specify different addresses
or ones that do not exist at start of autotest.sh. rbd_setup was
moved to begining of each test respectively.

Change-Id: I5626f7ea979d0db921208355ba23314cf48e971f
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/407910
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2018-04-17 13:55:44 -04:00
Jim Harris
18f3a22b20 test: disable ASLR for multi-process tests
Multi-process tests should work fine using the DPDK
-base-virtaddr parameter.  But we run tests with ASAN
enabled and ASAN instrumentation results in mmap
address hints getting ignored.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I089d31726c7b8f5ed0ccdc2deb19acc5431260f1

Reviewed-on: https://review.gerrithub.io/407843
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-04-17 13:26:24 -04:00
Jim Harris
dba0f6e0a8 test: run pmap on stub process after it starts
This will be a debugging aid to root cause some intermittent
test failures seen due to ASLR in the test pool.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I23a856ab53bdf378b5da51f1cb8d9f2cdf6efce1

Reviewed-on: https://review.gerrithub.io/407842
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-04-17 13:26:24 -04:00
Seth Howell
e3263286d3 test/nvmf: confirm kernel finds NVMe-oF namespaces
In Fedora 27 the nvme program returns before the kernel has loaded the
namespces associated with the subsystem. This patch ensures that those
namespaces can be enumerated by the kernel before we run any tests on
them.

Change-Id: I77505ed5cd5b1118a841650851fa5ecaf76f5619
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/407829
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-04-16 19:25:37 -04:00
Seth Howell
2393edd538 vm_setup: add new test flags to autorun-spdk.conf
Change-Id: I66cb037cf392230129a8fbcc5e455c1cc80c8903
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/406996
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-04-09 17:58:54 -04:00
Seth Howell
08939a7714 vm_setup.sh: add pre-existing directory checks.
This will allow us to run vm setup multiple times on a vm without it
failing due to pre-existing directories.

Change-Id: Ied17dd40daaa4b5f9a1e2448940d413c7db7cc96
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/405508
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-04-09 17:58:54 -04:00
Tomasz Zawadzki
ac277af97d test/config: add install VPP in vm_setup.sh
This patch adds compilation and installation of VPP.

Note that it removes one of VPP config files, that is
responsible for setting up hugepages. It is already done
with setup.sh script.
Parameters kernel.shmmax and vm.max_map_count were set
to low count and causing issues with hugepage total sizes
above 1GB.

Change-Id: Ic6c31f4192c654672e36c4131e34eb5b8aaac022
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/404144
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-04-09 17:45:31 -04:00
Jim Harris
218dcd840a bdev/pmem: change all NVML strings to PMDK
NVML (Non Volatile Memory Library) changed its name to
PMDK (Persistent Memory Development Kit), so make the
necessary changes to the SPDK repository.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Id07a87eafb4e9a3099603030c58c0e927bafa608

Reviewed-on: https://review.gerrithub.io/406256
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2018-04-07 00:07:57 -04:00
Jim Harris
6e1611da18 test: use ss -lx instead of netstat by default
netstat is deprecated on CentOS/RHEL in favor of
iproute utilities.  At least on one of the systems
in our test pool, netstat isn't installed - we could
install it but let's instead try to use ss if it's
available and still fall back to netstat when it's
not.

While here, remove waitforlisten_tcp instead of
changing it to use ss.  We are not using waitforlisten_tcp
anywhere and do not recommend enabling RPC over TCP except
for test purposes.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I4ee3597536797f501750915a4a219b55004bd454

Reviewed-on: https://review.gerrithub.io/406464
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2018-04-07 00:07:57 -04:00
Dariusz Stojaczyk
8e6b1fbd33 test/virtio: use "single file segments"
This enables virtio tests to run with 2MB hugepages

Change-Id: I0f9e2de0cb1f68fb967b929d8a27c6e410cb675a
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/404318
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-04-03 18:57:29 -04:00
Daniel Verkamp
20e69cf6b1 test/vm_setup.sh: add nvml-tools for pmempool
Change-Id: I964c45f652c6a24a05300d63f9c417e0f26e97e9
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/405919
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2018-04-03 14:14:29 -04:00
Seth Howell
397521bdb9 autotest_common.sh: handle ceph luminous in rbd_setup
unlike previous implementations of ceph, luminous (which ships with
fedora 27) does not create a default pool for us. In case we are on a
system using luminous, we need to create that pool ourselves.

Change-Id: I183dc4fff6b92f7b270bd5a5d05947f0d99e9d04
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/405703
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2018-04-02 16:58:18 -04:00
Seth Howell
899eb5f19a test: move ut_multithread.c to common/lib
Change-Id: Ied7da08503b52470dbb8bb77eee1a711f41e7f28
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/404977
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-03-29 00:31:24 -04:00
Seth Howell
c9fb57a298 test: move test_env.c under test/common/lib
This file is accessed by files in multiple directories under test.

Change-Id: I634481fb58eab5c097aaece5289f88e531954fcb
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/404976
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-03-29 00:31:24 -04:00
Seth Howell
b337d878ca test: move config directory under common directory
Change-Id: I9db13ab60348000f1bfeb39888fbaea6535d26ae
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/404953
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-03-23 19:45:09 -04:00
Liang Yan
2e0610a075 autotest: Add vtune test(porting from dts to autotest)
Add compiling SPDK with vtune test case.

Change-Id: I7ece8ef87e703eee5b1a8c326fd88224c9a02895
Signed-off-by: Liang Yan <liang.z.yan@intel.com>
Reviewed-on: https://review.gerrithub.io/392424
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-03-23 03:48:37 -04:00
Ziye Yang
15ce326251 test/iscsi: Add iSCSI initiator test
This patch leverages bdevperf, which configures
an iscsi initiator bdev to test iSCSI target.

Change-Id: I94c9299ef1f286a13678495d270326f4616c6241
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/404068
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-03-21 20:35:42 -04:00
Jim Harris
2af15344ab bdev: add iSCSI initiator bdev module
This uses libiscsi to implement an iSCSI initiator bdev
module for SPDK.  Still a lots of work to do on this - posting
it in case anyone is interested in working on this further.

A number of todo items are listed in a README in the lib/bdev/iscsi
directory.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Change-Id: I060e33de0cd6796246789bf0e1bb4f2df59d8f71
Reviewed-on: https://review.gerrithub.io/390313
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-03-21 20:35:42 -04:00
Pawel Niedzwiecki
b96cf2ef2a test/common: Separate failing nightly tests
Introduce RUN_NIGHTLY_FAILING flag for nightly tests
that are always or periodically failing.
A new cron job will be added to run test with this flag.

RUN_NIGHTLY_FAILING is set for:
test/vhost/readonly/readonly.sh
test/nvmf/multiconnection/multiconnection.sh
a fio job in test/lib/bdev/blockdev.sh

Change-Id: I32640c0d4b916156c2ee996a9847c7c9e64941f8
Signed-off-by: Pawel Niedzwiecki <pawelx.niedzwiecki@intel.com>
Reviewed-on: https://review.gerrithub.io/403985
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2018-03-21 12:52:10 -04:00
Pawel Niedzwiecki
ad22830896 test/common: exit with error in waitforlisten() if app crashed
In some cases, if vhost crashes on startup, test would continue
normally instead of failing

Change-Id: If2f82b5e3dad07403051353d6fcf5e83d5d002c0
Signed-off-by: Pawel Niedzwiecki <pawelx.niedzwiecki@intel.com>
Reviewed-on: https://review.gerrithub.io/403064
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-03-07 11:02:35 -05:00
Seth Howell
d1f9da828a autotest.sh: create cleanup function
This function consolidates cleanup behaviors that the pool should follow
in the event of a failure.

Change-Id: I8623f065146292b29a38eb199dcbeac5787723e7
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/401857
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-03-02 13:28:24 -05:00
Daniel Verkamp
d51ea8deb4 scripts/ceph: remove subshell and cd
The Ceph scripts can now be run from any directory; they don't depend on
their $PWD being the script location anymore.

Change-Id: Iff883a3e9b8ba2d0a6193e2f5c047eb9d7d423fd
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/401657
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-03-02 12:56:22 -05:00
Seth Howell
a562812dcf autotest: add test completion tracking
Change-Id: I1ca0578a010db2ff8535505bfd981cd1c368e403
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/392240
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-03-01 17:13:41 -05:00
Seth Howell
c9e5d2a6d0 autotest_common.sh: move to test/common
This is the first step in reorganizing the spdk test heirarchy.

Change-Id: I83467653af2da2a53251649092bed6902d6cb86e
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/401707
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-02-27 20:37:27 -05:00