Commit Graph

217 Commits

Author SHA1 Message Date
Jim Harris
6a8a1b6bef test: always parse common script args
Good suggestion from Darek - let's just always
parse common script args from autotest_common.sh.
These arguments follow common arg naming conventions
(i.e. --iso) so there's no harm just doing this for
any test that sources autotest_common.sh.  This has
the nice effect of not requiring scripts to
explicitly call this function.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Id89b68c22557a5a771be407873d0e57843f0d05a

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455552
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-05-24 05:36:46 +00:00
Jim Harris
390b364146 test: add parse_common_script_args function
iscsi test scripts can now take two arguments -
"iso" and then the sock type (posix or vpp).  They
need to be in that specific order too.  nvmf test
scripts also support "iso" and we want to add
the transport type (rdma or tcp) as well.  Even further
out, we may want to use a sock type for nvmf, i.e.
tcp transport with vpp.

We also have the iscsi_tgt fio_remove_nvme.sh test
that does both iscsi and nvmf.

So to make this all work a bit nicer, add a new
function called parse_common_script_args that
will take the command line arguments to a script
and set the appropriate variables, including defaults
when a specific parameter isn't specified.  We will
use getopt-like behavior for this also, instead of
enforcing a specific parameter order.  Then a script
could be called like this:

test/nvmf/target/shutdown.sh --iso --transport=tcp --sock=vpp

Individual test scripts then just need to do this
after sourcing autotest_common.sh:

parse_common_script_args $@

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ifb8d7666384991482a2d425e26ffa7525b9ac15a

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455283
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-05-22 23:43:51 +00:00
Jim Harris
d96abc9f0d test/nvmf: remove -p num_queues restrictions
There's a bug in Linux 5.1 (possibly 5.0 too) where
the kernel initiator driver crashes if it can't get
one queue per CPU.  This will get fixed eventually,
but for now we need to remove the cases where we
restrict the number of queues per controller so that
we can test on newer kernels.

Even on cases where we're testing the SPDK initiator,
there's no real need to restrict the number of queue
pairs.

The kernel will eventually get fixed, but we should
be testing with default behavior anyways (the kernel
wants lots of queues).  We'll also want to add some
regression tests to make sure the kernel doesn't
break again.  But that will all come later.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I9979e6d94456e075688b822b042936b63e518a4a

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/454819
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-05-22 14:50:05 +00:00
Darek Stojaczyk
d2e2758e0d test/qos: precalculate bandwidth limits
Don't do it at runtime, do it just once on script initialization.

Change-Id: Idb345fc0f72d3a41072d830b11a520584ec8b321
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/451886
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-05-22 10:21:49 +00:00
Darek Stojaczyk
ac8921f56c test/qos: use a single RPC for getting bandwidth and iops stats
Do a single RPC instead of two, cache the result,
parse it twice.

Change-Id: Ib0ae19da5cb6b6db7bf1a46c3960104d1975afdd
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/451885
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-05-22 10:21:49 +00:00
Jim Harris
a02b1481db test/nvmf: reduce duplication between target scripts
All of the checking for NVMF_FIRST_TARGET_IP, starting
the target, waiting on the target and modprobing
nvme-rdma is duplicated in every script.  So move
a lot of this either to nvmftestinit() or a new
nvmfappstart() function in common.sh.

Also just kill the nvmf target in nvmftestfini, rather
than the script to explicitly kill it.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I5864404610a4244473f460d48264de92687ed867

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/454678
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-05-17 12:48:19 +00:00
Jim Harris
5238a36bca test/iscsi: pipe scripts to rpc.py in trace_record.sh
This cuts off about 10 seconds from this test on my
system.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iec889b2c304c44c1e3f6b130900d1a9f4d0fc575

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453957
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-05-14 19:07:31 +00:00
Jim Harris
915270db68 bdev: make bdevs array for get_bdevs_iostat RPC
Fixes issue #775.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I5e07084599c2363b64619f38bd826fe100217020

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452477
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-05-08 22:43:00 +00:00
Seth Howell
03b9841dd9 test/iscsi: enable iso testing.
This paradigm has made it so much easier to test NVMe-oF locally. Extend
it to the iscsi tests as much as possible.

There are a few exceptions to the rule right now. Namely, the calsoft,
perf, pmem, and rpc tests which need to be slightly refactored before
they are ready for isolation running.

Change-Id: I3e92bd340c6a499dc77a56eb67bc7b074604746c
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453476
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-05-08 21:48:30 +00:00
Jim Harris
211cb4c250 test: don't call python explicitly
We use python3 now, not python.  This is important
because in next patch we will use a new 'aliases'
keyword argument in the argparse API that's only
available in Python 3.

While here, clean up some documentation that was
instructing users to call python explicitly instead
of just invoking scripts/rpc.py directly.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I0c11e131cd56f7c994aa8fe21b4b00ae33489fdd

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453460
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-05-08 14:22:19 +00:00
Seth Howell
a694d0bf31 test/iscsi: remove DPDK_DIR from iscsi ext4test.sh
This argument appears to be unused. I think it is a holdover from pre
dpdk submodule days.

Change-Id: I1f2084429dd9350730d7628af39db0b43dfdbd85
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453473
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-05-07 16:50:26 +00:00
Darek Stojaczyk
e600f09676 test/qos: disable iscsi qos tests
They have a lot of intermittent failures caused by the
test itself, not SPDK QoS. The actual bdev throughput
on our CI is roughly the same as the QoS limit in that
test, so the bdev can often fail the case where it's
expected to achieve more than the original QoS limit
after the limit is disabled.

Eventually this test will be replaced by a different
one, without any iscsi involved.

Change-Id: Ic71537b1c33376dda132e07210a34cae9bd073cc
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452424
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-04-29 17:32:26 +00:00
Seth Howell
0f94938a74 scripts/fio.py: add support for specifying num_jobs
This will allow us to test a greater number of connections without
having to add more subsystems when doing NVMe-oF testing.

Change-Id: I33203d6db79b30abb065f098c16840096478c5de
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/451677
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-04-24 09:22:16 +00:00
Karol Latecki
c1dd96ae63 test/iscsi: remove leftover code from fio.sh
Looks like a leftover from hotplug test when it was
done using a NVMe bdev instead of malloc.

Change-Id: Ia71d167b403d7f6d8ee5a621653f4062fce4ba6a
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/450035
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-04-04 03:58:56 +00:00
Karol Latecki
b6abc16b05 test: remove duplicate fio.py script file
scripts/fio.py and test/nvmf/fio/nvmf_fio.py are almost exact
same script. This commit unifies two files into one.

Change-Id: If753baaeb9f92dad2cda27bb4bed78ade4827d0b
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/448656
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-03-26 22:47:45 +00:00
GangCao
092360ce49 QoS: check whether to enable the QoS testing on iSCSI
Added a check whether to enable the QoS testing on iSCSI considering
VM or host has different performance data.

Change-Id: Iaf906ded70c3075fb3306e2301d638d613ec74dd
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/447133
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-03-26 03:01:35 +00:00
Ben Walker
a741e34180 bdev/nvme: Add configuration parameter to slow down polling
For NVMe devices, in conjunction with the new batching options,
it can be advantageous to artificially delay between polling
for completions. Add an option to slow this rate down.

Change-Id: I0fc92709ff45ead0beb388dda60694bf1ed8b258
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/447716
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-03-19 07:27:44 +00:00
Pawel Kaminski
51f411c213 test/iscsi_tgt: Use rpc instead of config file.
Remove unnecessary rm commands.

Change-Id: I2bf0da376c8ea492a74deb8c523773e30e199bdc
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/445304
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-02-22 18:28:25 +00:00
Jim Harris
5ab609b3bb test/qos: allow more variance due to iSCSI overhead
Running iSCSI in loopback runs into some cases where
a core is considerably impacted by the initiator
TCP traffic, even though the scheduler is trying
to schedule it on different cores.  Gang is working
through modifying the QoS tests to just use bdevperf
instead of iSCSI, but until that is ready, let's
allow more variance to reduce the fairly significant
rate of test failures we're seeing due to this test.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I99fe13621f5ca2f8e6402fc17fb859df815fe05e

Reviewed-on: https://review.gerrithub.io/c/444458
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-02-14 05:13:43 +00:00
Jim Harris
5f7eb6e38b test: remove references to stub application
We are doing away with using the stub application as
much as possible, so let's just remove references to it
where we can.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I52802474f1aaaf100c1b929f4e3ac555532a8410

Reviewed-on: https://review.gerrithub.io/c/444305
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-02-13 16:43:18 +00:00
James Bergsten
0c93fc7330 scripts:Fix Python errors in checking scripts
Python3 (3.6.7) gives W605 errors in regular expressions, causing the
SPDK style checks to fail on the Python scripts doing the style checking.

This fix allows these Python scripts to run without errors.

This is a known issue - see https://github.com/PyCQA/pycodestyle/issues/814

Change-Id: I71cdff5d6c89e19b200c989f3d9da35bb4f7189d
Signed-off-by: James Bergsten <jamesx.bergsten@intel.com>
Reviewed-on: https://review.gerrithub.io/c/443955
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-02-12 18:37:09 +00:00
Karol Latecki
5305a49593 test/iscsi: fix ext4test nightly
iSCSI target node was not deleted after error injection test.
Later in test this caused iscsiadm to connect to both EE target node
and to split nvme target node as they shared the same IP:PORT pair.

512MB malloc was too small to fit in spdk sources and compilation
products.

Fixes #642

Change-Id: I3d8ac6e1837cddad570f95263d3c6f529e64d5d8
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/442867
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Paweł Niedźwiecki <pawelx.niedzwiecki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-02-01 11:34:35 +00:00
Liu Xiaodong
a206234d09 lib/trace: add trace_record tool
trace_record is used to poll the spdk trace shm file
and store new entries from it to another specified trace file.
This could help retain the trace_entires from the overlay of
trace circular buffer

Note:
* trace_record reads the input tracefile into a process-local
memory and writes trace entries to the output file only at shutdown.
* trace_record can be shut down on SIGINT or SIGTERM signal.

A usage sample is:
./spdk_trace_record -s bdev_svc -p <spdk app pid> -f trace.tmp -q

Change-Id: If073a05022ec9c1b45923c38ba407a873be8741b
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/433385
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-01-30 06:36:25 +00:00
GangCao
05b43152b2 QoS/Bdev: add RPC support for Read/Write separate bandwidth limits
This patch adds the RPC support for the Read/Write separate
bandwidth limit controls. The basic usage as following:

usage:
rpc.py set_bdev_qos_limit [-h] [--rw_ios_per_sec RW_IOS_PER_SEC]
                               [--rw_mbytes_per_sec RW_MBYTES_PER_SEC]
                               [--r_mbytes_per_sec R_MBYTES_PER_SEC]
                               [--w_mbytes_per_sec W_MBYTES_PER_SEC]
                               name

positional arguments:
  name       Blockdev name to set QoS. Example: Malloc0

optional arguments:
  -h, --help show this help message and exit
  --rw_ios_per_sec RW_IOS_PER_SEC
             R/W IOs per second limit (>=10000, example: 20000).
             0 means unlimited.
  --rw_mbytes_per_sec RW_MBYTES_PER_SEC
             R/W megabytes per second limit (>=10, example: 100).
             0 means unlimited.
  --r_mbytes_per_sec R_MBYTES_PER_SEC
             Read megabytes per second limit (>=10, example: 100).
             0 means unlimited.
  --w_mbytes_per_sec W_MBYTES_PER_SEC
             Write megabytes per second limit (>=10, example: 100).
             0 means unlimited.

Change-Id: I822ec4814d21adff9826ce03a6af3783b1b98f44
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.gerrithub.io/c/417650
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-01-22 23:21:43 +00:00
Pawel Niedzwiecki
2d272954ac test/iscsi_tgt: Remove secondary process from iscsi_tgt tests.
Removing stub from iscsi_tgt tests reduces execution time
approximately 15-20s.

Change-Id: I4455b6d86ab8cd5bed67bd571dc748bc3c834313
Signed-off-by: Pawel Niedzwiecki <pawelx.niedzwiecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/436850
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-01-18 09:51:45 +00:00
Pawel Wodkowski
651d9158a0 test: consolidate tests for reading JSON config
This patch rework all JSON config file reading tests. This is needed as
current approach will require to modify each test to re-launch SPDK
applications for each test type. This is hard to do because tests are
not flexible and will require to be rewritten almost from scratch. Not
metioning time to relaunch each test separatly.

Instead, just replace all old tests with one test. Launch only one
target and only one initiator. Then configure it using RPC calls. Then
config is dumped and target and initiator application is relaunched to
reread the configuration from file. Then config is dumped again and
compared.

Change-Id: Id857089aed0e6f2197cbf3ae625a4e9f4b13e8bb
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/432318
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2019-01-16 14:17:10 +00:00
Tomasz Kulasek
bf6210b3c7 test/iscsi: fix support for python 3.6 in calsoft.py
Value of output field in subprocess.CalledProcessError exception
should be converted from 'bytes' to 'str' before write() operation.

Change-Id: I010c9ddd75302294853b2bab3db1bf013660a465
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-on: https://review.gerrithub.io/c/440081
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-01-14 05:45:35 +00:00
Shuhei Matsumoto
ed1e9613b8 test/iscsi_tgt: Move ip_migration test from nightly to per-patch
Github issue #544 was detected by ip_migration test. ip_migration
test consumes 26 seconds to completion, and not so long.

To avoid degradation, move ip_migration test from nightly to
per patch.

Change-Id: Ie11a9a1e9a82518133d619d48e1a1502074c99f6
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/c/438229
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2019-01-08 13:27:25 +00:00
Ziye Yang
9a0f019cda iscsi/test: remove the unnecessary -s option in ip_migration.sh
We already have the memory set for the stub process. If we set this
value, and the memory size set by the stub is changed. We will
cause the issue. So removing this option will be save.

Change-Id: I1968f5253796e766a8f1bb95f18554b0003a2969
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/c/438231
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2019-01-07 04:11:24 +00:00
Jim Harris
dbcbddbb22 test: mark iscsi_initiator.sh as executable
This fixes check_format.sh error.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I3748ce34b339a4b2e0a8358d38a68f0520177384

Reviewed-on: https://review.gerrithub.io/437875
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-12-19 19:16:08 +00:00
Pawel Niedzwiecki
ebac2feba5 test/iscsi_tgt: Add iscsi performance test.
Change-Id: I6d236c91e9624f61b235c10f076ed43341a34d74
Signed-off-by: Pawel Niedzwiecki <pawelx.niedzwiecki@intel.com>
Reviewed-on: https://review.gerrithub.io/404757
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-12-19 17:03:41 +00:00
Tomasz Kulasek
c68c2d28c1 test/iscsi_tgt: fixup bytes to str for python 3.5
Change-Id: Ia73e950c746fb23f1188dc09a4a34c8ba5f97985
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-on: https://review.gerrithub.io/436680
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-12-17 19:41:15 +00:00
Liang Yan
7b00091ac6 test: update qos.sh to add new test cases
Currently, SPDK can support bandwidth limiting. So add bandwidth
test cases into the qos.sh.

Change-Id: I85930506b296d6a94bd7dbb101a34ef804cd100e
Signed-off-by: Liang Yan <liang.z.yan@intel.com>
Reviewed-on: https://review.gerrithub.io/432141
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-12-07 17:15:41 +00:00
Pawel Wodkowski
d5256e152d test: fix iSCSI target RPC config test for Python3
Python3 Exception don't have 'message' attribute so need to add it in
RpcException class.

This attribute has beed dropped in pep-0352. We should switch to str(e)
instead in other places too.

Change-Id: I610696b9e374e11ffa95327af6a10c6874915022
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/435506
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-12-05 00:42:37 +00:00
Jim Harris
870b0e5b29 log: remove "trace" from log flag RPCs
We will keep the RPCs for now but mark them deprecated.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I0407dcb392ea0c9e89c0f26cd5670aed2dbfadef

Reviewed-on: https://review.gerrithub.io/435345
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-12-03 19:50:15 +00:00
Jim Harris
027a8e4b54 test/iscsi: remove reference to auth.conf in calsoft json
Also remove the discovery auth group from the json file,
and use RPC to set it instead.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I597b60a7c7e4de9d7e1719b0145ab2b11c6f26b0

Reviewed-on: https://review.gerrithub.io/434265
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-11-21 20:09:07 +00:00
Shuhei Matsumoto
cd209dbbaf test/iscsi_tgt: Port CHAP to JSON RPC in Calsoft test
Use JSON RPC instead of config file to configure CHAP secret.

Change-Id: Iecaa3eed00fd1b0c25d46693c4ebc04973970f6f
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/433205
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-11-19 18:17:22 +00:00
Shuhei Matsumoto
78a45b3f45 test/iscsi_tgt: Port running_config test to JSON config and re-enable it.
Change-Id: I7619ea8dc94e76934c8ebb37f4248376feaa23ef
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/433200
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-11-19 18:17:22 +00:00
Shuhei Matsumoto
a66c9ffd50 test/iscsi_tgt: Port config file from .INI to JSON in the FIO test
Port the config file used in the FIO test from .INI to JSON.

Besides, set_bdev_nvme_hotplug RPC cannot be used easily because
the iSCSI target is not the primary process in the FIO test.

The FIO test doesn't use the hotplug feature and hence remove
enabling hotplug simply.

Change-Id: Idf8809aa829b67a7028f6fa4778a7aae1b431bb8
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/433223
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-11-19 18:17:22 +00:00
Tomasz Kulasek
21bd942751 libsock: add functional tests
Change-Id: I3c86c1b4a9c8ae3f18264f33b15d647e4081a12c
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-on: https://review.gerrithub.io/421911
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-11-07 17:08:27 +00:00
Jim Harris
d40803609a test/iscsi: exclude *.o from ext4test.sh rsync
This will resolve out-of-space errors that have cropped
up as SPDK continues to grow.  There's no need to copy
*.o files to the mounted filesystem - we 'make clean'
right after the rsync anyways.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I6844183c527953fd4b3329f04171f05e503b04dc

Reviewed-on: https://review.gerrithub.io/429517
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-10-15 22:20:17 +00:00
Seth Howell
5fbbb6c056 test/nvmf: Use create_transport instead of set_tgt_opts
When running these tests, we want to set the transport opts, not
necessarily the target opts. This will also open us up to doing broader
tests in the future against different transports.

Change-Id: I73112f82c371592456c65355f53f7970878a59c3
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/429049
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-10-12 16:30:24 +00:00
GangCao
868c28cd13 QoS/Bdev: add the RPC support for the bandwidth rate limit
This patch added the support of RPC method to enable,
adjust, disable the bandwidth rate limit on the bdev.

And it can work together with the existing IOPS rate limit.

The RPC method has been consolidated to support both IOPS
and bandwidth rate limits as below:

usage:
rpc.py set_bdev_qos_limit [-h]
                          [--rw_ios_per_sec RW_IOS_PER_SEC]
                          [--rw_mbytes_per_sec RW_MBYTES_PER_SEC]
                          name

positional arguments:
  name       Blockdev name to set QoS. Example: Malloc0

optional arguments:
  -h, --help show this help message and exit
  --rw_ios_per_sec RW_IOS_PER_SEC
             R/W IOs per second limit (>=10000, example: 20000).
             0 means unlimited.
  --rw_mbytes_per_sec RW_MBYTES_PER_SEC
             R/W megabytes per second limit (>=10, example: 100).
             0 means unlimited.

Change-Id: I9c03cd635280add01801a81c6a6c02f0cf85bee1
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.gerrithub.io/416511
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-10-09 18:26:44 +00:00
Karol Latecki
85ba606943 test/iscsi: disable running_config test
running_config test step in test/iscsi_tgt/fio/fio.sh
breaks nightly test builds.

This is caused by "ReactorMask" parameter in generated
config file which was deprecated and seems to be no longer
supported.

Change-Id: Ibea72b7b883adaae939db1e5ea15724817d6dfd2
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/426199
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-09-20 21:52:18 +00:00
Ben Walker
ec1f425841 test: No longer use construct_nvmf_subsystem
Use nvmf_subsystem_create instead.

Change-Id: Idcfa6526ef47e2839f69acd5d7dc08e65e5150b5
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/425187
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-09-20 18:35:27 +00:00
Chen Wang
7e408509ba test/iscsi: add fio and lvol test cases with raid bdev
1. Use the Raid-0 bdev which based two malloc bdevs to instead of
   one malloc blockbdev.
2. Create the lvol store with Raid-0 bdev which based two malloc bdevs
   when the number of Malloc bdevs is equal to 1.

Change-Id: I8b584ad09011386b405e4756c04273834b24f029
Signed-off-by: Chen Wang <chenx.wang@intel.com>
Reviewed-on: https://review.gerrithub.io/423043
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-09-15 00:16:50 +00:00
Pawel Wodkowski
b96f97cf29 scripts: use python3 in all scripts
This it to prepare for RPM package. Also lower number of dependencies
needed by SPDK tools.

Update changelog to deprecate Python 2 and explicit interpeter invoking
in scripts.

Change-Id: I2497cca721cbcbadc1c99c675f8b8b7f682d5efa
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/425233
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-09-14 22:24:30 +00:00
Pawel Wodkowski
bd0abde125 test: sort objects in JSON cofniguration tests
Python 3 is changing order of objects in JSON object when doing load ->
dump. Using OrderedDict solves this issue.

Change-Id: I0aa6364de31e871aed36edd918d25c261c66b506
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/425494
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-09-14 22:24:30 +00:00
Jim Harris
55014ddf53 rpc: remove file parameters from save/load_config in rpc.py
User can pipe or redirect data as needed.  As features like
bdev crypto are added, storing config data in files needs to
be carefully scrutinized - don't make it easy by providing
ways to write data to files that may not be sufficiently
protected.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I24dd32f81b965e275e1bac875f6d008d1b68007a

Reviewed-on: https://review.gerrithub.io/425165
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-09-11 21:11:51 +00:00
Tomasz Zawadzki
897eed3179 test/ioat: remove unnecessary config entires
Since IOAT is disabled by default, that option can be removed from
any config file used in tests.

Change-Id: I681bd1eb6a8c81328ba580be75b418994ccd85e3
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/423357
Reviewed-by: GangCao <gang.cao@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-08-30 14:45:48 +00:00