Commit Graph

286 Commits

Author SHA1 Message Date
Tomasz Zawadzki
d7aff25df0 ut/common: add poll_thread_times() to poll select number of iterations
Right now functions poll_thread() and poll_threads(), polled
always until there were no further messages to process.
This allows to put next operation to execute only after
previous one completed.

Function poll_thread_times() allows to poll certain thread
for exact number of executions. If 0 is passed to max_polls,
then this function executes single message and all pollers,
until no messages are left.

With this change it is possible to only run a thread
selected number of iterations. Allowing to add another operation
without first one finishing.

It will become useful in blobstore UT, where two operations
will be executed with slight delay between them.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I4a6537695389f30130ea2fd2fd43d7b2cb2dea39
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/772
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-02-21 09:35:27 +00:00
Darek Stojaczyk
94675ba17b test/autotest_common: cleanup discover_bdevs
$rpc_server is hardcoded inside the function and can't
be empty, so remove the check for that.

Change-Id: Ic819089eb113b14a7c2407cd5ce882bcdee044bb
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/522
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-02-21 09:34:14 +00:00
Maciej Wawryk
9ebbf3acac scripts/pkgdep.sh: Add flag for developer tools installation
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I8876fd7c60198c597d931c5f498644c415c2a868
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/596
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-02-18 14:42:16 +00:00
Seth Howell
19260848f6 nvme: publicly declare spdk_nvme_transport
This will be useful in the upcoming spdk_nvme_poll_group api.

Change-Id: Id83340a2ce9887817312f5aac38db4de8c588974
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/577
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-02-18 08:05:08 +00:00
Karol Latecki
71efe5db24 Fix Markdown MD026 linter warnings - trailing punctuation in header
MD026 Trailing punctuation in header

This rule is triggered on any header that has a
punctuation character as the last character in the line

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I3ab4894092ef6b5f920d89b74e43c2e46e9581c0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/657
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-02-17 10:07:21 +00:00
Karol Latecki
93be26a51d Fix Markdown MD022 linter warnings - headers blank lines
MD022 Headers should be surrounded by blank lines

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I768324b00fc684c254aff6a85b93d9aed7a0cee5
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/656
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-02-17 10:07:21 +00:00
Karol Latecki
a41c031609 Fix Markdown MD012 linter warnings - multiple blank lines
MD012 - Multiple consecutive blank lines

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I3f48cdc54b1587c9ef2185b88f608ba8420f738b
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/654
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-02-17 10:07:21 +00:00
Shuhei Matsumoto
70ec72871e lib/thread: Fail spdk_thread_exit() if thread is already exiting
This is a preparation to support voluntary thread termination by
calling spdk_thread_exit().

Change spdk_thread_exit() to return -EINVAL if the thread is already
marked as exited.  This will be helpful to detect wrong call sequence
of voluntary thread termination.

Besides, update reactor shutdown and unit test framework shutdown
to incorporate this change accordingly.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I2296c61e273bf4d9580656dcbc2da0e8a8f3bcf7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/671
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-02-13 09:51:15 +00:00
Michal Berger
a8d016910a test/common: Make sure that extdebug is enabled as soon as possible
if given script, which executes under the debug tracer, fails before
xtrace_restore() is called, the BASH_ARGC[] will miss all the arguments
which were passed down till that very point. Similar case can be
experienced after tracer is disabled as then all the arguments hold by
BASH_ARG{C,V}[] become unavailable (i.e. until tracer is enabled again).

Since there's no actual benefit from toggling the extdebug (in fact, it
could break DEBUG|RETURN traps if ever used), enable it once when
autotest_common.sh is sourced and keep it enabled throughout entire
execution of given script.

Change-Id: I01001ead1570967a2e550d993f85f12b9f62553e
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/477
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-02-11 08:38:08 +00:00
Pawel Kaminski
1fd96c85b4 test/vhost: Replace old style config with json in initiator test
Change-Id: I622448dcd1de960793894395d7da55ebd612dcc7
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/529
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-02-10 10:25:11 +00:00
Michal Berger
a11aab73e8 test/common: Check if igb_uio is loaded before removing it
igb_uio doesn't necessarily have to be loaded into the kernel as SDPK
still may be built with support for it but only particular set of
tests is actually using it. If such a condition is met then rmmod will
fail the entire test run since it always exits with != 0 if given
module is not found.

Avoid this by checking first if igb_uio is actually in use.

Change-Id: Ib97488797c657f810b588a0b427e578807ebe6c6
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/625
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
2020-02-10 10:23:39 +00:00
Karol Latecki
3d8a0b19b0 doc: Fix Markdown MD032 linter warnings
"MD032 Lists should be surrounded by blank lines"
Fix this markdown linter error by inserting newlines or
adjusting text to list points using spaces.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I09e1f021b8e95e0c6c58c393d7ecc11ce61c3132
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/434
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
2020-02-06 09:46:06 +00:00
Seth Howell
a9a472b99b scripts: update to point to review.spdk.io
Change-Id: I10c79cd421bd735ced2e17919cfa38abec74edf2
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/451
Reviewed-by: Jim Harris
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: <sys_sgci@intel.com>
2020-02-04 23:05:51 +00:00
Michal Berger
9ba80de0dd test/nvme: Don't wait for spdk_stub0 if stub dies prematurely
If stub terminates right after execution, e.g. due to lack of system
resources (requested number of cpu cores to run on, right amount of
memory, etc.) start_stub() would end up blocking forever since there
wouldn't be any entity around that would mknod spdk_stub0 for it to
break the loop.

Avoid the above scenario by checking if $stubpid is still visible
under procfs and return if it goes missing.

To make sure kill_stub() is still called to clean up after start_stub()
declare proper trap prior the call to start_stub().

Additionally, avoid potential stderr noise in case kill|wait are told
to act on a PID that's not visible in the ns anymore.

Change-Id: Ief41200c57957f84b4f96a54baabc8da1f27dd43
Signed-off-by: Michal Berger <michallinuxstuff@gmail.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482653
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-02-04 20:22:23 +00:00
Michal Berger
4b1dbc948b test/common: Don't take randomize_va_space default setting for granted
Currently, kill_stub() doesn't gracefully handle this particular sysctl
since it overwrites it with assumed default that doesn't necessarily
have to be a part of kernel's config on a given system.

Don't presume what the setting should be, instead, read and save the
current value and try to restore it whenever kill_stub() is called.

Change-Id: I1f1ee85c29d5e2ec2f442a54f700e3bc45ee2437
Signed-off-by: Michal Berger <michallinuxstuff@gmail.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482652
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-02-04 20:22:23 +00:00
Michal Berger
18b9303dfb test/common: Include BASH_ARGV[]s in the backtrace
Enable extdebug and try to include all the arguments passed down
the function stack in the backtrace.

Change-Id: I81381c936b0f895f1ca8e31d57ef8116d737c6cd
Signed-off-by: Michal Berger <michallinuxstuff@gmail.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482695
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-01-31 12:39:22 +00:00
Michal Berger
1baf379e69 test/common: Don't attempt to read backtrace from unavailable source
In case cwd is changed during the execution of given BASH_SOURCE, i.e.,
when the dirstack is mangled by calls to cd, the actual executable may
end up missing from the path when run directly from its directory in
the ./ fashion. Example:

  [root@fedora31 fuzz]# ./autofuzz.sh --module=vhost --transport=all

autofuzz.sh cds into the $rootdir hence the BASH_SOURCE[i] in form of
./autofuzz.sh won't be found there, thus during a failure, since run
under a debug tracer, nl will fail with -ENOENT while trying to read
it.

To mitigate, check if $src is available for reading, if not, log that
the backtrace is not available.

Change-Id: I68988350ba36cca8464bdfac437f662ed4c30f67
Signed-off-by: Michal Berger <michallinuxstuff@gmail.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482694
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2020-01-31 12:39:22 +00:00
Tomasz Kulasek
327668c8c9 test/common: make gdb_attach global
Change-Id: Ib73bd9513681360f22251b54a2d15ca9807b72c2
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478807
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-01-27 17:39:52 +00:00
Or Gerlitz
e61b0904a8 sock/posix: Add flush
Initiator drivers (e.g nvme/tcp) don't use poll groups but rather directly
poll the qpair. In this case we want to allow the polling function (e.g
_qpair_process_completions()) to flush async writes pending on the socket.

Signed-off-by: Or Gerlitz <ogerlitz@mellanox.com>
Change-Id: Ibd8c73691213d58e287b7110d0f5a381a89a64d0
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/475419
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-22 13:53:09 +00:00
Seth Howell
253fb41b65 autobuild: Confirm we are building all files w/ scanbuild
This gives us an assurance that we know scan-build is being run against
all relevant files in the repository.

Change-Id: Ic0b871e98a9ea7acd2d6b2a99ab81955af29fc66
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/479898
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2020-01-22 13:50:17 +00:00
Seth Howell
d4e52cacfd test/bdev: remove the gpt conf file / parameterize tests
Previously, we only ran bdevperf tests against GPT partitioned NVMe
drives. These tests are generally applicable and should be run against
all of the bdev types we support.

So long story short this configuration file isn't needed and we can just
get rid of it.

Change-Id: Ia6ded22ce16fc1f76b7d99643b9d37e3ecbd1c60
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478244
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-01-22 13:50:17 +00:00
Seth Howell
df254d5a25 autorun_post: add a file for tests we don't run.
Change-Id: I8beac9959d08026ee984833d0e90ec17381a66e5
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478240
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-01-22 13:50:17 +00:00
Seth Howell
ea781d6dbb test: report test domains for tests.
This will allow us to run test cases under multiple suites without
clogging up the logs too much, but it will also preserve information
about which test suites were run or (more importantly) not run.

Change-Id: I2434a54a0877ae36b9f84bfab8a62653ac1172f8
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/477367
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2020-01-22 13:50:17 +00:00
Maciej Wawryk
53147e0dc8 test: fix issue with changing device names in waitforblk function
During test before upgrading our vm's to Fedora31 there were issue with failing
nvmf test on function waitforblk. In some cases hardcoded nvme0n1 name was not
found in system. Here is fix that change searching by name to searching by nvme
serial.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Ic5ee70804652a057fa26b8cc004b5227262d2122
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478471
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Seth Howell <seth.howell@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-20 10:00:06 +00:00
Ziye Yang
0bfaaace8f sock: Add impl_name parameter in spdk_sock_listen/connect.
Purpose: With this patch,

(1)We can support using different sock implementations in
one application together.

(2)For one IP address managed by kernel, we can use different method
to listen/connect, e.g., posix, or uring. With this patch, we can
designate the specified sock implementation if impl_name is not NULL
and valid. Otherwise, spdk_sock_listen/connect will try to use the sock
implementations in the list by order if impl_name is NULL.

Without this patch, the app will always use the same type of sock implementation
if the order is fixed. For example, if we have posix and uring together,
the first one will always be uring.

Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: Ic49563f5025085471d356798e522ff7ab748f586
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478140
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-01-16 09:11:32 +00:00
Seth Howell
f6cf92a31f lib/nvme: make transport.c use fn tables.
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: Ida58785784b4ed50393e1d43a9cd902de74a2eaa
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478873
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-01-16 09:10:38 +00:00
Seth Howell
e4eef6975c lib/nvme: add function tables for all transports.
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I4e7af1c42a19346f4abcb17910a41f8104a2de1b
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478871
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-01-16 09:10:38 +00:00
Seth Howell
5cddf93b47 test/nvme: add common_stubs.h file.
A lot of the stubs between these transports will end up being common.

Change-Id: Ib9c8ff947b95f34633eb13953405d3153a7f4ac7
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/479602
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-01-16 09:10:38 +00:00
Maciej Wawryk
19e48a66a4 test: Shellcheck - correct rule: Iterating over ls
Correct shellcheck rule SC2045: Iterating over ls output is fragile. Use globs.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I436878774d6f86c23b1c5cf5220da297053f03e2
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/477397
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-01-15 14:30:06 +00:00
Seth Howell
a571eb03e9 test: move report_test_completion into run_test
This also requires us to change the create_test_list function to rely on
the run_test function for creating the canonical test list.

Change-Id: Ib35e7752935a3ac83de2702b6dfbd42539027f6a
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476962
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-14 07:49:08 +00:00
Seth Howell
2f476a638f autobuild.sh: simplify and unify test suite.
We have added a lot of flags to this script over time to try and shorten
parts of it that take a long time or modify the way we make and
whatnot.

In the test framework today we really have two settings, we either want
to run the entire autobuild package with all the bells and whistles, or
we just want to make the code and get on with the rest of the tests. I
believe this change can save between 1 and 3 minutes on each of the
functional test suites.

Change-Id: I7519e8320aa16b57f09f633f866dc36cb494aa80
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478483
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-14 07:49:08 +00:00
Seth Howell
cbc1e069ee configure: allow cuncurrent build of RBD and reduce
This was disabled in configure because at one time there was an ISA-L
dependency break between the RBD libraries and our own submodule.
It seems that the dependency break was fixed a while ago, but the check
to not allow one to build with the other was never removed.
Unfortunately, I don't have the details on what that fix was.

Either way, this compilation works now.

Change-Id: Ic249791549dae36c7279114d67e77be94e842ddf
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/479726
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-14 07:49:08 +00:00
Seth Howell
39fe5c849b autotest_common: fix xtrace_restore_enable
My previous patch enabling this to be called recursively didn't
take into account that a single call to xtrace_restore would
override multiple levels of xtrace_disable nesting. This change fixes
that.

fixes: 190b2245c5

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I2167ba460e68223c9426b3d71e9c17019f947924
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478959
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-01-08 15:46:18 +00:00
Shuhei Matsumoto
f74107fb85 test/common: Mock CPU core operations for unit tests
Mock DPDK lcore operations for unit tests. Remove duplicated stup
from unit tests for NVMe-oF FC transport. The next patch will use
them.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I4f6fbd8f45942ca13fcf10ba1740a9fe8e573063
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478153
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-01-03 08:33:25 +00:00
Seth Howell
cb90136c33 test: eliminate case and suite from run_test
We will be able to distinguish case and suite in the post process
scripts based on if they have any internal tests.

Change-Id: Iff2aa5caa251924d8a842085d87eb2a17aca45d4
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478522
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
2019-12-24 08:24:42 +00:00
Shuhei Matsumoto
46e52a0bcb test/common: Set invalid value to CPU core API stubs
This change verifies that all existing unit tests don't care about
CPU core configuration.  The subsequent patches will mock CPU
core operations.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I44cc56908cf8609870625ff19d6100e21036c00f
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478152
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-12-20 10:05:38 +00:00
GangCao
82f60376ae test: properly handle the killprocess case
To check whether the process already killed and if the
process is not there, return error.

This is to fix the below issue:
https://github.com/spdk/spdk/issues/1066

Change-Id: I5f36213237a4b0eed8edbc69e666037cf17488b2
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476127
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
2019-12-17 15:28:37 +00:00
Seth Howell
190b2245c5 test: make xtrace_disable multi-call safe.
Previously, if you made two sequential calls to xtrace_disable without
calling xtrace_enable, you would lose the original value of x. Now this
is not the case. It allows us to call xtrace_disable freely without
worrying about whether xtrace_disable is invoked later in a function we
call.

Change-Id: I9818ae97532e7a31db576bb9bd2e2e30d67ab410
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/477965
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
2019-12-17 14:54:40 +00:00
Ben Walker
b7ad942612 sock: Add an asynchronous writev
Add spdk_sock_writev_async for performing asynchronous writes to
sockets. The user of this call is responsible for allocating their own
spdk_sock_request structures to pass to this call.

spdk_sock_writev_async will not return EAGAIN and will instead leave the
requests queued until they are fully sent or aborted due to socket
error.

Change-Id: Idf3239e65d26a3024e578122c23e4fb8f95e241b
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/470523
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-12-16 12:34:02 +00:00
Seth Howell
1fafd71bc3 test: add timing calls to run_test
And clean out the corresponding calls to timing_enter and timing_exit
from the various test scripts.

Change-Id: I0759417b5a529e4c3649ce04cca1799c089da278
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476804
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
2019-12-10 17:12:03 +00:00
Seth Howell
fd17f1aabc test: make run_test more robust
make sure we have enough arguments.

Change-Id: I76ce35635ef14289061323ee401d93d8d081888c
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476801
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2019-12-10 17:12:03 +00:00
Seth Howell
b8042c3850 test: use human readable names in test logs.
Change-Id: I163a7e08bfeff3da2ee627d9fad9f2f1a249c593
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476820
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2019-12-10 17:12:03 +00:00
Seth Howell
3710048472 test: add a test_name param to run_test
This will allow us to use timing_enter and timing_exit directly
inside the run_test function. That function already lends itself well to
nesting the way we do our timing.

This patch series is aimed at combining the timing_*, run_test, and
report_test_completions calls all into a single place. This will greatly
reduce the number of lines of code in our bash scripts devoted to
tracking timing, formatting, and test completion. It will also enable us
to expand on the reporting of test completions. Further down the line,
this will also allow us to unify test case documentation.

Change-Id: I8e1f4bcea86b2c3b88cc6e42339c57dfce4d58f2
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476799
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2019-12-10 17:12:03 +00:00
Maciej Wawryk
2951c1010e Change fio version
Whole autotest fails on VM Fedora31
GH #1081
Fio version update to fio-3.15

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I3b91c426050eb30af6b58434b6219090c61a48ba
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476893
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-12-06 14:50:59 +00:00
Maciej Wawryk
029251878d test: Shellcheck - correct rule: Consider using { cmd1; cmd2; }
Correct shellcheck rule SC2129: Consider using { cmd1; cmd2; } >> file instead of individual redirects.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Ic692e9f78f2d3d8a3b27d702884cbe97c880727c
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/474959
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2019-11-22 13:51:15 +00:00
Ben Walker
1a903f9c52 opal: Don't use set +e to ignore errors.
For errors that we do want to ignore, use || true after the commands.
This allows crashes in the background target application to fail
the tests.

Change-Id: I1fcd711c17ad0a956b6778260b2db8c0b801584f
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/475156
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-11-21 10:05:45 +00:00
Tomasz Kulasek
8e70c6e320 test/asan: suppress memory leaks in external libfuse3
There are some leaks in libfuse3 that is external to SPDK.
With this patch, any leaks in libfuse3 will be suppressed.


==1944861==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 32 byte(s) in 2 object(s) allocated from:
    #0 0x7fc26a462048 in __interceptor_realloc (/lib64/libasan.so.5+0xf0048)
    #1 0x7fc269bfcfe2 in fuse_opt_add_arg (/lib64/libfuse3.so.3+0x19fe2)

Indirect leak of 10 byte(s) in 2 object(s) allocated from:
    #0 0x7fc26a3ade60 in strdup (/lib64/libasan.so.5+0x3be60)
    #1 0x7fc269bfcfc6 in fuse_opt_add_arg (/lib64/libfuse3.so.3+0x19fc6)

SUMMARY: AddressSanitizer: 42 byte(s) leaked in 4 allocation(s).


Appropriate patch solving this issue was sent to libfuse, but
not merged yet.


Change-Id: I66625e155b78082f2f2c9790bf3f3b48c3c04f33
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/475117
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2019-11-21 08:01:57 +00:00
Pawel Kaminski
074df1d896 test: Shellcheck - apply rule SC2155
Declare and assign separately to avoid masking return values.

Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Change-Id: Ib90598e4268911a3056e8a0baa7d541edaa29b91
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/472287
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-11-12 18:14:59 +00:00
Tomasz Zawadzki
9a25fc12bb test: add SPDK_BUILD_PACKAGE flag
Until now the autopackage.sh execution was tied to
RUN_NIGHTLY flag. This was useful to check for
errors on all jobs/systems during a nightly test.
By their nature nightly does not run on per-patch
basis.

This allows for issues to slip in and be noticed only
by the nightly job after merge occurred.

Adding separate SPDK_BUILD_PACKAGE flag, allows to
explicitly test packaging on single job on per-patch.
RUN_NIGHTLY was kept as trigger for autopackage.sh
to still test it on all systems by default.

Change-Id: I29925fd6256b218e24c24ebcc4974c65a9bd986c
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/473519
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2019-11-08 22:18:24 +00:00
Jim Harris
2be2b6eba5 test: add additional asan suppressions related to fio
We are already suppressing fio (not SPDK fio_plugin)
leaks in a couple of other places, which could likely
be causing the indirect leaks we are now going to
suppress here.

Fixes issue #1003.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ie5283280495e7155cda1a93d2bd3d48ffbb6cba7

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/473495
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: yidong0635 <dongx.yi@intel.com>
Reviewed-by: GangCao <gang.cao@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-11-07 22:53:21 +00:00