This is needed to create a job in jenkins that tests only the tcp
transport.
Change-Id: Iaa5a07617e9a241a9e4bd50105d5d5d1e849844e
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/460713
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Scripts were using a mix of two approaches, lets unify that so
just dollar-parenthesis syntax is used.
Also update two scripts from spdk/test which we previously missed.
Change-Id: I429f9bc158076462b419fae597f716c329f9b7aa
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/460344
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
And also add spdk_sock_group_get_ctx function
Change-Id: I2a2a58b0588ff7d99d3538ea0a633a3b8c7a234b
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/454538
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
NOTE:
Due to some changes in VPP two cherry-picks from newer version of VPP
(at least 19.04):
"sock api: allow to start client with no rx_thread"
commit: 97dcf5bd26ca6de580943f5d39681f0144782c3d
"dlmalloc: honor 8 byte alignment requests"
commit: f5dc9fbf814865b31b52b20f5bf959e9ff818b25
These commits are already merged in VPP 19.04.
This patch includes also workaround for VPP 19.01.1 to prevent closing
sessions to already closed applications after timeout. It causes
intermittent segfaults.
This is temporal solution and should be solved with next releases of
VPP, but is required now to create more stable VPP environment.
Change-Id: If4b45b7159819cfd836dd7d50f333dbab2b38eab
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456462
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
This patch prepares environment to allow compile applications and
libraries with VPP.
Change-Id: Icc067c17b57932062afa5e6a67791bd58de3a2cc
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456461
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
We have no guaranty that iscsiadm -m session shows all required iscsi
devices at some point of time after login. It may cause that not all
devices are used in further tests (e.g. in fio).
This patch allows to wait for a certain number of sessions after login.
Change-Id: I868cb8f1235dd04d88572bc8518e97de99a2c5e6
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458236
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Missing dependency libudev-devel. If not installed then
./cofnigure fails with:
checking for libudev.h... no
configure: error:
udev headers required
Change-Id: I53c8f522cbc8eb1e6b461fd78c0bf3305bfb09b9
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459323
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Our test destination code of spdk is rsynced from host.
former link address of spdk "https://review.gerrithub.io/spdk/spdk"
is not very fast and always failed .
So changed to "https://github.com/spdk/spdk.git" ,this spdk just for
excuting start testing command.
Change-Id: I26b5c1556d8068becd5c5a94067cc64d9d32b5e6
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459223
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Instead of doing some OS-specific magic to detect if the
spdk app has already initialized, just try to send it an
RPC. This serves mostly as cleanup, but also simplifies
test output in cases where the spdk app could not be fully
initialized. Previous waitforlisten implementation would
return as soon as the rpc subsystem was initialized, but
the app could have still failed on e.g. bdev initialization
later on. Since waitforlisten() returned success, the
script could continue its execution and try to execute
some RPCs. The logs from the SPDK app and the bash script
could easily mix, rendering the entire output not so clear
to the user.
To fix it, just try to send some RPC commands inside
waitforlisten().
Change-Id: I33eaf362e3cc645f8ea3ee22fd48db1643442129
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457562
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Report issue on ubuntu18 :
"spdk/autobuild.sh: line 55: hash: scan-build: not found"
Check that ubuntu18 should install this package:
apt-get install -y clang-tools, to fit version of ubuntu18
and later.
Move clang-analyzer which for fedora to vm_setup.sh as well.
Change-Id: Ia702c492f8b0f64705c7c15ee57a861ca14521f9
signed-off-by: yidong0635 <dongx.yi@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457554
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
There are some leaks in libiscsi that is external to SPDK.
With this patch, any leaks in libiscsi will be suppressed.
Please see examples of the leaks right now when running
bdevperf in initiator.sh test:
Direct leak of 176 byte(s) in 1 object(s) allocated from:
#0 0x7fd64dddfe50 in calloc (/lib64/libasan.so.5+0xeee50)
#1 0x7fd64c60edb4 in scsi_cdb_inquiry (/usr/lib64/iscsi/libiscsi.so.8+0x13db4)
Indirect leak of 256 byte(s) in 1 object(s) allocated from:
#0 0x7fd64dddfe50 in calloc (/lib64/libasan.so.5+0xeee50)
#1 0x7fd64c603a84 (/usr/lib64/iscsi/libiscsi.so.8+0x8a84)
Indirect leak of 52 byte(s) in 1 object(s) allocated from:
#0 0x7fd64dddfe50 in calloc (/lib64/libasan.so.5+0xeee50)
#1 0x7fd64c60d755 in scsi_malloc (/usr/lib64/iscsi/libiscsi.so.8+0x12755)
#2 0x7fd64c6097f8 (/usr/lib64/iscsi/libiscsi.so.8+0xe7f8)
Change-Id: I80ef23b1464841f683344c7aab99f1658a46cd36
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456766
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
ASAN needs to be LD_PRELOADed before SPDK fio_plugin
in order to analyze its code.
Just adding that will report any issues in fio binary as
well as the fio_plugin.
To prevent known fio leaks from affecting the results,
a suppression list for LeakSanitizer (used in conjunction with ASAN).
At this time the suppression list contains known leaks
for fio 3.3. The list might need adjustments as fio
version is updated.
Side note. Even though it is possible to specify directory
to ignore ("leak:/usr/src/fio/"). Which in theory should
suppress any leaks in fio. It has side effect of hiding
SPDK leaks as well, since the fio_plugins leaks are
seen as coming from /usr/src/fio/ioengines.c.
See below for examples of each suppressed error:
Direct leak of 42 byte(s) in 4 object(s) allocated from:
#0 0x7f9d52f3e320 in strdup (/lib64/libasan.so.5+0x3b320)
#1 0x41f267 in get_new_job /usr/src/fio/init.c:490
Direct leak of 914936 byte(s) in 10397 object(s) allocated from:
#0 0x7f74422e8ea6 in __interceptor_calloc (/lib64/libasan.so.5+0x10dea6)
#1 0x46402e in log_io_piece /usr/src/fio/iolog.c:214
Direct leak of 608 byte(s) in 19 object(s) allocated from:
#0 0x7f74422e8ca8 in __interceptor_malloc (/lib64/libasan.so.5+0x10dca8)
#1 0x44c4e1 in add_to_dump_list /usr/src/fio/parse.c:1039
#2 0x44c4e1 in parse_option /usr/src/fio/parse.c:1098
Direct leak of 173 byte(s) in 20 object(s) allocated from:
#0 0x7f744227153d in strdup (/lib64/libasan.so.5+0x9653d)
#1 0x44b50d in __handle_option /usr/src/fio/parse.c:718
Indirect leak of 111925528 byte(s) in 1271881 object(s) allocated from:
#0 0x7f74422e8ea6 in __interceptor_calloc (/lib64/libasan.so.5+0x10dea6)
#1 0x46402e in log_io_piece /usr/src/fio/iolog.c:214
Indirect leak of 171 byte(s) in 19 object(s) allocated from:
#0 0x7f744227153d in strdup (/lib64/libasan.so.5+0x9653d)
#1 0x44c4ed in add_to_dump_list /usr/src/fio/parse.c:1040
#2 0x44c4ed in parse_option /usr/src/fio/parse.c:1098
Indirect leak of 167 byte(s) in 19 object(s) allocated from:
#0 0x7f744227153d in strdup (/lib64/libasan.so.5+0x9653d)
#1 0x44c502 in add_to_dump_list /usr/src/fio/parse.c:1042
#2 0x44c502 in parse_option /usr/src/fio/parse.c:1098
Change-Id: I9b5811993508421be50b12af160645c77ea93d7e
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456315
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Scripts were using a mix of two approaches, lets unify that so
just dollar-parenthesis syntax is used.
Change-Id: Id093b6c82d1f766ba6af13bed720977eceaa7ffc
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457744
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
This doesn't make difference right now, but would be
useful if we printed $BASH_SOURCE or $LINENO in xtrace
(via PS4). Those two will now point to the original
line where xtrace_restore was called rather than
always the single line inside autotest_common.sh.
Change-Id: Idf3ac8d00ad9610960678351014334013149b88d
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456964
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Further patches in series will modify the suppression file,
so this patch now creates such file in /var/tmp/.
Meanwhile adding the known false positive to the list.
Any known false positives or leaks in external libs/execs
should be added by adding further entries to the suppression
file.
Change-Id: I7b78d900a86c6eca0b41240fb34dc4f7ad597079
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456622
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This patch adds functions in autotest_common.sh,
fio_bdev() and fio_nvme() for their respective plugins.
It simplifies its usage throughout the scripts.
Additionally will help with expanding the fio cmd
line with nessecary changes only in single place.
This will be used in next patch in series to
LD_PRELOAD ASAN before fio_plugin.
Note: Did not implement changes for perf scripts,
since they are usualy run separately and didn't want
to affect those.
Similarly didn't change vhost initiator tests,
because the exported directories do not work for
inside the VM. Will need to think of a way to change it
there as well.
Change-Id: Idf750ee7aa9d5e30efc0ce430d15fa45ceccb203
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456314
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
DPDK rte_ring_enqueue_bulk() has free_space parameter to return
the amount of space in the ring after enqueue operation has finished.
This parameter can be used to wait when the ring is almost full and
wake up when there is enough space available in the ring.
Hence we add free_space to spdk_ring_enqueue() and spdk_ring_enqueue()
passes it to rte_ring_enqueue_bulk() simply.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I9b9d6a5a097cf6dc4b97dfda7442f2c4b0aed4d3
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456734
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
We also no longer hardcode TEST_TRANSPORT=rdma. Users
of the individual test scripts must now explicitly
pass --transport=rdma. Support for tcp will be coming
in future patches.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I819d69cb0906eb27b692eb2755aca5085298d779
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456685
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This prepares for using scripts for tcp testing as well.
Note that this patch just hardcoded TEST_TRANSPORT to
'rdma' for now. An upcoming patch will require the
caller to pass --transport=rdma instead.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I085fdf51910aaf960959c71c73a187be41fd7ee8
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456667
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Waits up to 2 seconds for a bdev to arrive, if not returns error.
In the next patch in this series this is used and seems like a good
general function.
Change-Id: I3402f34f3dad3d7373973dc785520a5c4e58cd14
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456091
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
When a new python script wanted to use RPC client or calls
from RPC lib from ./spdk/scripts/* it had to be created
within that directory.
By expanding PYTHONPATH in autotest_common, now any python
scripts can use that library.
First example of this is located in:
./spdk/test/bdev/bdevio/tests.py
Change-Id: Idf3e5891c3815a84c70525ab9338100acbfa4a43
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455219
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
NOTE: pulling the vbdevcompress change into its own patch...
While enabling reduce testing in autotest a few issues were found
in autotest in that some options were being set based on the existence
of a header without checking the test flag, if one existed. Then at
the end of adding options, specific test flags were checked for 0 and
the configure updated to --without. So, to make this easier to read,
the options are organized in sections and where there is a header
dependency as well as a test flag, both are chcekced at the same time.
Change-Id: I8cdc6914906500bbac63528eb541e2b235feb797
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455137
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Ubuntu18 integrates librxe to rdma-core, libibverbs-dev no longer ships infiniband/driver.h.
Don't compile librxe on ubuntu18, install package rdma-core instead. ubuntu16 keeps the old
method. Otherwise, there's no NIC can be found for SoftRoCE failed.
Change-Id: Ib639b96a4229c79f2b27fda7b981d7a805f808cb
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455336
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Make the old name a deprecated alias.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ibbf50676e0d989b67121e465fc140f94faec46ed
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453033
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Recently, we moved the check for running an installed version of DPDK
above the point where we export the SPDK_* test flags. This resulted in
SPDK_RUN_INSTALLED_DPDK being undefined when trying to check fi we
should run against an installed DPDK. This just caused autotest_common
to print an error message every time it was loaded. I don't think that
it actually caused any errant behavior but this gets rid of the error
message. It's also probably a good idea to keep these definitions as
close to the top of the file as possible.
Change-Id: I5aadbe5c925ecf1ac92926b75c8c043aab73b36b
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455456
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Good suggestion from Darek - let's just always
parse common script args from autotest_common.sh.
These arguments follow common arg naming conventions
(i.e. --iso) so there's no harm just doing this for
any test that sources autotest_common.sh. This has
the nice effect of not requiring scripts to
explicitly call this function.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Id89b68c22557a5a771be407873d0e57843f0d05a
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455552
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
iscsi test scripts can now take two arguments -
"iso" and then the sock type (posix or vpp). They
need to be in that specific order too. nvmf test
scripts also support "iso" and we want to add
the transport type (rdma or tcp) as well. Even further
out, we may want to use a sock type for nvmf, i.e.
tcp transport with vpp.
We also have the iscsi_tgt fio_remove_nvme.sh test
that does both iscsi and nvmf.
So to make this all work a bit nicer, add a new
function called parse_common_script_args that
will take the command line arguments to a script
and set the appropriate variables, including defaults
when a specific parameter isn't specified. We will
use getopt-like behavior for this also, instead of
enforcing a specific parameter order. Then a script
could be called like this:
test/nvmf/target/shutdown.sh --iso --transport=tcp --sock=vpp
Individual test scripts then just need to do this
after sourcing autotest_common.sh:
parse_common_script_args $@
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ifb8d7666384991482a2d425e26ffa7525b9ac15a
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455283
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Installing this package "open-isns-utils" always with pop-up dialogs,
we don't want to these dialog, so add --no-install-suggests
--no-install-recommends to prevent it.
Use sudo apt-get remove --purge * to simulate.
Change-Id: I0b892b9e0c88c82ab9461a92e71dc0d9823ecaf9
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455333
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Lightweight threads may now be exited by calling
spdk_thread_exit() within the thread. The framework
polling the thread can release the resources associated
with that lightweight thread by calling
spdk_thread_destroy().
Change-Id: I6586b9d22556b3874fb113ce5402c6b1f371786e
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455319
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Currently autotest prints the following at the start
of each run:
> grep: /[...]/output/timing.txt: No such file or directory
No harm is done and the file is created right after
the grep, but the message might be misleading. We get
rid of it now by ensuring the file always exists before
grepping it.
Change-Id: I271b68479a6fddcbe098c2657fd7c4dc39e6e50a
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/454708
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
bc is currently used in iSCSI qos.sh test, but is not listed
as a pre-requisite in vm_setup.sh
Also it's a good alternative to using sed or awk for some
simple math operations.
Change-Id: I18742c0dedcdcc5eeebd310eb5760e226a163af7
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453998
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Instead of linking to the real sock library in the unit tests,
link to a mocked up version like we do with the env.
Change-Id: I9607f135c6af86e09fa52badf44897f90ce94521
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/454492
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Most of our bash test scripts source autotest_common.sh
to be able to use some autotest-specific functions like
timing_enter(). The same test scripts allow specifying
custom command line parameters without actually realizing
that those parameters can be potentially picked up by
autotest_common.sh as well.
For example, if particular nvmf tests are run in "isolation"
mode by being executed with the first param set to "iso",
and there is a file named "iso" in the current dir, that
file will be sourced. This could be bad.
In this patch we stop sourcing or even processing $1 in
autotest_common.sh. Instead, the test configuration will
be sourced just once from autobuild.sh, autopackage.sh
and autotest.sh. If the user wants to run particular test
scripts manually, he should source an SPDK test configuration
by himself - manually as well. In most cases he won't even
have two, as only a few test scripts depend on SPDK_*
variables.
Note that we still have to setup the default values for
SPDK_* variables in autotest_common.sh because some of
our test scripts actually depend on them:
> if [ $SPDK_TEST_RBD -eq 1 ]; then ...
Because it lacks any type of quotes around SPDK_TEST_RBD,
it will print the following message when that variable
is unset:
> /bin/bash: line 0: [: -eq: unary operator expected
It doesn't trigger any error ($? == 0), but can be still
a bit misleading in the script output.
Change-Id: I350045d8582d66fe1ed7697d4bcbba324cb541ad
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453876
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Most of our test scripts source autotest_common.sh at the
very top of the file, and - since autotest_common.sh
enables xtraces inside - the first thing we'll see in test
output are likely xtraces from sourcing the configuration
file and setting up $config_params. Even a simplest test
script will produce a ton of output this way, making the
entire output unreadable. This is especially annoying when
the test script is run manually, so we get rid of it now.
All the SPDK autotest options and $config_params will be
printed elsewhere anyway.
Change-Id: I6d9c8e8ba261b632ffbb7d6d26d84eb7ccfb2ed8
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453875
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This way all the ./configure parameters are setup in one
place without any unrelated code in the middle.
Change-Id: I1cd8dc05e326b9ab3e8e51c193dd422fd8b5a00b
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453873
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
We use that variable only inside autotest_common.sh so far,
but hopefully that's going to change now. Our tests still
hardcode /var/tmp/spdk.sock everywhere at the moment.
Change-Id: Iad862cb7674a8108d2a07f35b5298d21e009c5a1
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453872
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Put it all together at the top of the file so that
the file is less chaotic overall.
Change-Id: Ia92511e590692bbe31d0f626e9542374eaaab442
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453871
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
SPDK_RUN_ASAN was only effective on Linux so far,
even though libasan is available on BSD as well.
Let's fix that.
Change-Id: I58a1796f736616882f797313e22055122b050b57
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453870
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This way we can consistently track when xtrace in our
scripts is enabled or disabled.
> [...]
> xtrace_disable
> PREV_BASH_OPTS=ehxBE
> set +x
> xtrace_enable
> [...]
Change-Id: I2e813dc2a237a4620ea72d26a22a3c8cbeb269f9
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453248
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Factor out `set -x` and +x to separate functions. Changing
xtraces is not so trivial and we'll be improving it later
on. We can't factor out the code as is due to local variables,
so we already simplify it a bit in this patch.
Change-Id: Iecbf5cedf821b7b1b71da933ceb158761881a843
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453246
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
We added this a long time back when debugging some
multi-process failures. This adds a lot of output
to the log which isn't helpful anymore.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I4d46b744f30b32fddedc3dae32f40c077717a1bb
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453909
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Now that SPDK_TEST_* default to 0, we can decidedly say
that if the requested test suite can't be run on your
system due to unmet dependencies (e.g. missing system pkg),
you will get an error.
Previously we would silently disable that test - e.g. when
someone ran with SPDK_TEST_RBD=1 but had no librbd system pkg
on his system, the SPDK autotest would return success without
running any RBD tests. That was very counter-intuitive.
We still silently disable a lot of tests inside our particular
test scripts. Hopefully we'll stop doing that eventually.
Change-Id: Ib400ba772c095b4d65fa8893a0669f3d8c3984e6
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453712
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
We've got a bunch of `if hash` checks that generate
unnecessary output, e.g.:
$ if ! hash ceph; then [...]
/var/jenkins/workspace/Other_systems/centos7_autotest/
spdk/test/common/autotest_common.sh: line 28: hash: ceph: not found
We now silence that output in cases where we do expect
the binary to be unavailable and handle that case, but
we hold off in cases where `if hash` is just a sanity
check (e.g. `hash sgdisk` in part_dev_by_gpt() which
can make the entire function return immediately and do
nothing).
Change-Id: I7cbfada858aa86d57cbe182ea7f9b222debc8b88
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453207
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
This ensures the stub gets killed so that we can
make forward progress on collecting a backtrace.
This requires adding an optional parameter to the
kill_stub function to specify a signal.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ib46d07b6651c6f35382f38c97901def03f28289d
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452891
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
By default most of the tests are enabled, if we want to
run only tests that interests us, we need to disable each flag
separatly.
Build SPDK without additional dependencies and exit autotest.sh
early if no test configuration file is provided.
Change-Id: Id4752689479d9325242364c39c89d1236bab0344
Signed-off-by: Pawel Niedzwiecki <pawelx.niedzwiecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/443762
Reviewed-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
FTL doesn't have any kind of special package requirements.
It is getting pretty good traction in the community, so
let's enable it by default.
Note that we will disable FTL on FreeBSD. FTL uses
CIRCLEQ which is not available on FreeBSD. Let's not
spend time trying to get FTL to work on FreeBSD until
there's a demand to do so.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I15525b6c4e6ee52f49adf74d55f9484fe08a6dcc
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452752
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Allow users to optionally specify an affinity mask when
creating a thread. This isn't currently used, but we want
the API to be in its final form for the next release.
Change-Id: I7bd05e921ece6d8d5f61775bd14286f6a58f267f
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/451683
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This commit adds functionality for installing most of the packages,
which were previously installed for Fedora only,
for Ubuntu as well
Some packages are not going to be installed by the script:
For ubuntu 16
* libpmempool
* open-isns-utils
* perl-open
For ubuntu 18
* perl-open
Remaining packages deserve their own patches since they are not so easy to
install on Ubuntu
Change-Id: Ic5b744193ff4df46ab4e4fb3783e0515ccdbfecb
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/425388
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Paweł Niedźwiecki <pawelx.niedzwiecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
vmsetup invokes pkgdep.sh with -i flags, but that flag is not supported.
This patch changes the call to just `pkgdep.sh`.
Change-Id: I1eb92b9ddfce1ea2d7ef450fd795f8f93fb198ec
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/450260
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Recent change of ceph_raw.img size from 10G to 4G breaks
tests on patches which are not rebased or which use
previous SPKD releases.
Change-Id: I9dd79756e3cc7949593cd1c16d4735542d1fb32d
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449469
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Currently the scripts deletes also files located
in ocf submoudle named core.py. This patch adds regex to avoid that.
Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Change-Id: I97a21c4860cb54f37f28641dc9cd47adcf2fd939
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/448519
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
The phys_addr param in spdk_*malloc() is about to be
deprecated, so use a separate spdk_vtophys() call to
retrieve physical addresses.
This patch also adds error checks against SPDK_VTOPHYS_ERROR.
The error handling paths are already there to account for
spdk_*malloc() failures themselves, so reuse them in case
of vtophys failures.
Change-Id: I377636e66b8c570d013c1bb2021f04bce4e6c0ce
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/416998
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
This package is needed in vm_setup.sh for vpp,
without this will report "rpmbuild: command not found".
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Change-Id: Ic00a55addc940e359aae17b45e58b9a736b08fd3
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/447282
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Piotr Pelpliński <piotr.pelplinski@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Running as a non-privileged user, sudo git clone won't work
without the -E because it won't get the proxy settings, for those
using a proxy.
Change-Id: I8b471abf0066312b0eb63449d00a5647737c0136
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/446162
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Similar to UNIT_TEST_NO_VTOPHYS, we need this since
a future patch will result in some of the pci_addr
functions getting linked in via a new dependency
between memory.c and init.c.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I61bd4e172ee70a5dcb99c8c6fc1fb19070a2a7cd
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/446460
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
CPU profiling on workloads with intensive vtophys
operations (i.e. very small CB-DMA transfers) exposed
overhead introduce by spdk_vtophys having to call
spdk_mem_map_translate in a different compilation
unit. Let's just move the vtophys.c contents into
memory.c so that spdk_vtophys can inline
spdk_mem_map_translate and avoid this extra overhead.
This of course breaks the memory and vtophys unit
tests, so some additional changes are needed there
to keep everything linking correctly.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I295ed5f441d3eec7abdbc9d881c49d2174ec9f48
Reviewed-on: https://review.gerrithub.io/c/444975
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
spdk_thread_poll()
This is an optimization if the calling function already knows the
current time.
Change-Id: I1645e08e7475ba6345a44e0f9d4b297a79f6c3c2
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/443634
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
We started to use iptables in patch 21bd94275
(libsock: add functional tests) but never added
the package dependency.
Change-Id: I651f2545a11f546f8b47f9759fbaed3a141f0928
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/443597
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Includes the required DPDK dependencies for SPDK block Reduce aka
Compression.
Change-Id: Ic1ea3cbeb9373a7700f6f0c2a3194d65d6a34a41
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/429523
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This is the functional opposite of waitforblk.
Change-Id: I5e9f77c9ea05b83b8dbfbf49bb3baa2ae8cc1086
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/443316
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Iefb4c76f2db5416d185556fa8c84a36291e72d3e
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/443237
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Introduced a new variable to run functional tests.
It's enabled by default, and can be manually disabled
on systems where e.g. only unit tests are run.
SPDK_RUN_FUNCTIONAL_TEST is a supplement to SPDK_UNITTEST.
The two are completely independent - both can be enabled,
disabled, or run in any combination.
The new variable is prefixed SPDK_RUN_ as it aligns nicely
with SPDK_RUN_CHECK_FORMAT, SPDK_RUN_VALGRIND, and
SPDK_RUN_ASAN, all of which control how much is tested.
SPDK_UNITTEST should eventually follow the same pattern
as well.
This gives us 2 layers of configuration:
SPDK_TEST_* <- what is tested
SPDK_RUN_* <- how it is tested
The following would run UT+ASAN for FTL and BlobFS, without
running their functional tests:
```
SPDK_RUN_FUNCTIONAL_TEST=0
SPDK_RUN_ASAN=1
SPDK_TEST_UNITTEST=1
SPDK_TEST_FTL=1
SPDK_TEST_BLOBFS=1
```
Change-Id: I9e592fa41aa2df8e246eca2bb9161b6da6832130
Signed-off-by: Seth Howell <seth.howell@intel.com>
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/442327
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This patch introduces functional tests for FTL bdev.
The tests cover various I/O workflows and check data integrity. Several
scripts have been added to test the FTL library:
* generate_config.sh - prepares configuration scripts for specified
device
* restore.sh - tests restoring device's state from the SSD
* fio.sh - runs tests based on fio and fio_plugin
The tests are run from autotest.sh when the SPDK_TEST_BDEV_FTL flag is
set.
Change-Id: I561d99ed35fe91eadd3756789cc99afe2da8c1db
Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/431330
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
We currently use:
`./rpc.py 1>&2 2>/dev/null`
And apparently our stdout redirection gets treated as
a python argument, which is not what we intended. This
caused waitforlisten function to print some unnecessary
messages.
To fix, we could either wrap python with extra brackets
`$(./rpc.py) 1>&2 2>/dev/null` or use &>/dev/null. We
go with the latter.
Fixes#616
Change-Id: I49d2380392435447b7b93259055c1d02c4b9c157
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/442262
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
In SPDK, we will build isa-l with no shared option
and then integrate it into SPDK. And we do not need
to install isal in the system libaries.
Note: ocf build in autobuild.sh now needs to build
include/spdk/config.h before building the ocf library,
to ensure that header is available in a clean build
environment.
Change-Id: I3f0ce6932b386de17a77cf5bfdfd738b22417e2d
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Signed-off-by: paul luse <paul.e.luse@intel.com>
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Reviewed-on: https://review.gerrithub.io/c/441279
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Chunyang Hui <chunyang.hui@intel.com>
Add SPDK_TEST_CAS suite
Add basic integrity test with multithread-multidevice fio with cache
modes WT and PT
Change-Id: If2916558894d9f7efd9628eb99f7338c90b312b1
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.gerrithub.io/c/439346
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Add installing python3-pandas.
put it into vm_setup.sh, this package is needed by autorun_post.py
Change-Id: I76c98c54f16be69879ac05d5128a049fc693e455
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Reviewed-on: https://review.gerrithub.io/c/435968
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
This change is required for unit tests that use spdk_ring mock
in multithreaded scenarios.
Change-Id: I8f5e83de69a685078cbb01e55daac650f6ae1db1
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/440721
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This name more closely resembles pthread_exit, which is a
closer analogy to how the new threading library works.
Change-Id: I68b04509f3ff8e94b8688804a7e5661155a3ecd1
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/440597
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
This mirrors pthread_create, which works more closely
to the new style where SPDK libraries can spawn their
own threads.
Change-Id: Ic524c4c35bcf7c1611e4f261ebb64b98ac5a5a1b
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/440596
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Instead of implicitly grabbing the thread from the thread
local variable, make it explicit.
Change-Id: I733fad06181439e12b1e71a4829b84e7b64e2468
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/440595
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
These are no longer used by anything.
Change-Id: I0db6bc88e4dc945ff4f64df2ac410e1d00a669c1
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/437601
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
SPDK threads are no longer mapped 1:1 to system threads. They are
instead identified by the memory address of the spdk_thread object.
Change-Id: I417f8376cebd2ee94f624f4436e6394b51486063
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/437999
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ib9de83db429e4ae6f8e0cc988ca161cebdb27c9c
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/437599
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This patch adds core FTL (flash translation layer) submodules.
On regular SSDs the FTL is part of firmware, whereas Open Channel
enables moving it to the host and allows for having full controll
of data placement on the device.
Main functionalities added:
* logical to physical address map
* read / write buffer cache
* wear-leveling
* bad block management
Change-Id: I5c28aa277b212734bd4b1f71ae386b3d6f8c3715
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Signed-off-by: Jakub Radtke <jakub.radtke@intel.com>
Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/431322
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This follows the same trend as the mem_map APIs.
Currently, most of the spdk_vtophys() callers manually
detect physically noncontiguous buffers to split them
into multiple physically contiguous chunks. This patch
is a first step towards encapsulating most of that logic
in a single place - in spdk_vtophys() itself.
This patch doesn't change any functionality on its own,
it only extends the API.
Change-Id: I16faa9dea270c370f2a814cd399f59055b5ccc3d
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/438449
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: wuzhouhui <wuzhouhui@kingsoft.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
sshpass used in test/nvme/hotplug.sh test but not present
in vm_setup.sh dependencies.
Change-Id: Ic9d7f137f7694e5318f6f412b1e7a73536a82f8c
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/438788
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Add stop.sh before start.sh,to avoid reporting resource busy
it could lead to start failed.
Change-Id: I1756adaad31937a7a4b95fdc488f2334360d4ac2
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Reviewed-on: https://review.gerrithub.io/437616
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
This allows us to call it during test set up instead of
only within a unit test.
Change-Id: I2daae8d630729b5e0712057766a3cba8812109e9
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/437479
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This is the callback type used for message passing. The
old name is easy to confuse with the callback to pass
a message to a thread or the upcoming callback to
spawn a thread.
Change-Id: I5fd63b57c4be2a4262a197850e6de4901be03ee7
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/435941
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Use Qemu v.3.0.0 from spdk/qemu.
Change-Id: I44ecdac5f591a6096ec8450dd70044a7ba6628e0
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/436476
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Having unlimited size core dumps is not a good idea on a lot of
machines. Modify it to 5GB.
Change-Id: I1d52bfa9f2450e2d8f824c3b86aa2ad5fe4579c3
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/436412
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
As a part of cleanup they're replaced by a device-agnostic
attach API, which is easier for us to manage.
Change-Id: I2ec68f20ba209f02ee5c2de4b6fe5330a4bc0853
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/436480
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
As a part of cleanup they're replaced by a device-agnostic
attach API, which is easier for us to manage.
Change-Id: Ia92bd8f4525712bd27ade16ead67435c5e0fbe7a
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/436479
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
As a part of cleanup they're replaced by a device-agnostic
attach API, which is easier for us to manage.
Change-Id: I7558590e41e5c580a130a6aba7ae4f7dcff58da8
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/436478
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
In case the process we are waiting for fail to start
listening on RPC socket the test script will stuck till the build times
out. Fix this by adding 20s timeout and 0.5s sleep in the while
loop.
For systems where there is no ip command and netstat output is missing
'Status' column for Unix sockets (like FreeBSD) call the get_rpc_methods
RPC command to check if process is listening.
Change-Id: Ia8b06af7875b65a7fd8be65cf55e92881f6f95db
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/433102
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
We are too verbose in many places. Some scripts don't want to be traced
all the way, instad they enable tracing only for some parts and
autotest_common.sh just do 'set -x' in many places.
This patch save 'x' flag on function enter and restore it at exit.
Change-Id: I39b3d3dd3f711e1131e476f9d322d9e1b097ad12
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/432604
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
This workaround is needed to stop EBUSY errors from NBD ioctls(). This
can be fixed in NBD bdev code but till that point lets merge this
workaround.
Change-Id: Ic1546963538beda4d4409cef93062103e2a23e34
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/434040
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
