Commit Graph

186 Commits

Author SHA1 Message Date
Karol Latecki
ed2a57a353 test/nvme_perf: add option for dpdk mem measurements
Allow to dump DPDK mem stats mid-test.

Change-Id: I5769bca9b3b10eb9a4114c29ae112d82066511aa
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3374
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-07-24 17:09:03 +00:00
Karol Latecki
f4e27ffa14 test/nvme_perf: add option to enable gtod_reduce
Optionally enable gtod_reduce for fio workload.
Results in greatly decreased latency metrics, but
improved IOPS/BW.

Change-Id: Ibe70d8e9a13a18a31e55b60b5cec7b353e2862c3
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3379
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2020-07-24 09:40:49 +00:00
Karol Latecki
0e897f08a6 test/nvme_perf: add option to set cpu frequency
If intel_pstate is disabled in boot options, allow to
change CPU frequency using "userspace" cpu governor.

Change-Id: I390492c7be28df408f3cbb774ba5360859649fe0
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3376
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-07-24 09:40:49 +00:00
Karol Latecki
88b991768d test/nvme_perf: add option for perftop measurements
Enable perf record during tests and create a report
file afterwards.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: If6951ad7425d8c1d4d2935c8af9f40245c1adbf7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3371
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
2020-07-24 09:40:49 +00:00
Karol Latecki
6330ee8fb5 test/nvme_perf: unify fio job section for kernel and spdk
For some reason the routine for generating fio
job/filename sections was different for kernel and
spdk fio modes. This made fio results hard to compare
because even configuration files themselves were not
comparable.
The function was modified to create the same fio
configuration files for kernel and spdk.

While at it, iodepth calculation has been slightly
modified as well. It is now being calculated for each
[ ] job section, so that each "filename" gets the
actual iodepth value specified in test run parameters.

While generating the config file the script will add
NUMA node information as a comment for "cpus_allowed"
and "filename" sections, so that it's clearly visible
how resources are aligned. In case of cross numa
configuration additional warnings will be printed to
screen as well.

For example, a configuration file for running test with
7 NVMe drives, 3 CPUs and effective (per filename) queue
depth of 32 would look like this:

run_perf.sh --cpu-allowed=0,1,2 --iodepth=32
  --driver=kernel-classic-polling [...other params...]
--------
[filename0]
iodepth=96
cpus_allowed=0 #CPU NUMA Node X
filename=/dev/nvme0n1 #NVMe NUMA Node X
filename=/dev/nvme1n1 #NVMe NUMA Node X
filename=/dev/nvme2n1 #NVMe NUMA Node X

[filename1]
iodepth=64
cpus_allowed=1 #CPU NUMA Node X
filename=/dev/nvme3n1 #NVMe NUMA Node X
filename=/dev/nvme4n1 #NVMe NUMA Node X

[filename2]
iodepth=64
cpus_allowed=2 #CPU NUMA Node X
filename=/dev/nvme5n1 #NVMe NUMA Node X
filename=/dev/nvme6n1 #NVMe NUMA Node X
--------

Change-Id: Ida2f781fbb93c4a8c62154e711151152843ab997
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3370
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-07-24 09:40:49 +00:00
Karol Latecki
b9150897d9 test/nvme_perf: move fio config generation into one place
Configuration file generation was split into multiple
places/functions in scripts. Move it into a single
function. "time_based" fio parameter has been moved to
fio config template file as it is not configurable.
There is still a lot of room for improvement here, because
create_fio_config() function uses mixed local variables
(as function arguments) and global variables.

Change-Id: Ica3717d0d28692d24c861c2d32e729761c5c8647
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3345
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-07-24 09:40:49 +00:00
Darek Stojaczyk
814072fa4e env_dpdk/pci: delay device initialization on hotplug
A workaround for kernel deadlocks surfaced in #1275.

DPDK basically offers two APIs for hotplugging all PCI devices:
rte_bus_scan() and rte_bus_probe(). Scan iterates through
/sys/bus/pci/devices/* and creates corresponding rte_pci_device-s,
then rte_bus_probe() tries to initialize each device with the
supporting driver.

Previously we did scan and probe together, one after another, now
we'll have an intermediate step. After scanning the bus, we'll
iterate through all rte_pci_device-s and temporarily blacklist any
newly detected devices. We'll use devargs->data field to a store
a timeout value (integer) after which the device can be un-blacklisted
and initialized. devargs->data is documented in DPDK as "Device
string storage" and it's a char*, but it's not referenced anywhere
in DPDK. rte_bus_probe() respects the blacklist and doesn't do
absolutely anything with blacklisted ones.

The timeout value is 2 seconds, which should be plenty enough
for an NVMe device to reset, leave the critical lock sections in
kernel, and let us initialize it safely.

Note that direct attach by BDF doesn't respect the blacklist,
so an NVMe attach RPC won't be delayed in any way, it will continue
to work as it always did. Only the automatic discovery & enumeration
is deferred.

Change-Id: I62b719271bd0755bc2882331ea33f69897b1e5e5
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1733
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-07-23 20:48:47 +00:00
Karol Latecki
ddba021eea test/nvme_perf: move getopts to main script
There's no reason to keep usage and getopts
outside of the main script.
Other small changes while at it:
- use small letters for all "dir" variables
  (consistent with most of other test scripts)
- try to move default & global variables into
  one place.
- remove long comment from top of the main script.
  It was basically the same as "help" for getopts.

Change-Id: Ic99f4bb1808b55dea80b534f7b4e0a65595ed053
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3335
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-07-23 07:28:53 +00:00
Karol Latecki
6cebf49243 test/nvme_perf: make fio log_avg_msec configurable
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I92c7af88713ff4aa9e4a7cdf347ca06a303e5a07
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3334
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-07-23 07:28:53 +00:00
Karol Latecki
30491c4675 test/nvme_perf: allow CPU list from file
Read CPU list to use in test directly from option
value of from a file if it's an existing path.
This will make running tests in CI a bit easier.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I174a38ab615c2f2cd2c3b4a95e0ebf886c478a51
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3333
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-07-23 07:28:53 +00:00
Karol Latecki
ae4c75b20d test/nvme_perf: add bdev io cache and pool size option
Allow to set custom size of bdev IO cache and pool
sizes.

Change-Id: I574d1e949f705d8f834539d5879f0a1e72865e00
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3297
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-07-23 07:28:53 +00:00
Karol Latecki
a7972f0a54 test/nvme_perf: use config file to select test disks
Use a simple config file to select NVMes for test
instead of selecting them all via gen_nvme.sh.
HW platforms may differ significantly in setup and
capabilities, so using all disks in sequential order
is not always a good way.
(e.g. using "first" four available NVMes for test
may yield not optimal results if it turns out that
NVMes are connected to oversubscibed PCIe switch)

Change-Id: I55dd85799a5859c4764e94e1d8058e01ad9b84f0
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3296
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-07-22 12:22:46 +00:00
Karol Latecki
20995d370b test/nvme_perf: remove --disk-no option from perf script
Script was written in such a way that tests would be
performed starting with all found NVMe disks in configuration.
Then the number of disks for the workload would be reduced in
loop until reaching 1. This is the scenario used in test
case 1, described in SPDK NVMe Performance Benchmark document.

--disk-no allowed to avoid running test in loop, and specify
just a single value for the number of disks to be tested.

Both loop logic and --disk-no are no longer required as
tests are scheduled and run via CI system which picks the
parameters for each test run.

Change-Id: Iad0f4d0d259ac64680fd5bb69d7cbb7e0afff79c
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3295
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-07-22 12:22:46 +00:00
Karol Latecki
0655f92a0f test/nvme_perf: improve fio results parsing
- Change divisor in nsec to usec calculation to "1000".
- Ignore MIX param when rw mode is set to 100% read or
  100% write.
- Use MIX param as number in 0.0-1.0 range for calculating
  summary read+write latency.
- Use jq "//" alternative operator when querying for fields
  which might not exist in JSON object. Assume "0" for
  non-existent fields, otherwise jq reports "null", which
  in turn results in script errors.

Change-Id: Ibc8f91f058d9ae31ee1b60f4b253c42b743ae22f
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3294
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-07-22 12:22:46 +00:00
Karol Latecki
da5c4e093d test/nvme_perf: reorganize fio config generation
Previous changes moved some of the fio CLI parameters into
a bash heredoc to save into a text file, but it was put in
wrong place in the script.

Move global fio parameters a couple lines up so that the
resulting fio config file has the correct form of:
[global]
  global_params
[testjob]
  testjob_params

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I3a958fe5ce880ab63a079c345e6b24fd1d3bae32
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3293
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-07-22 12:22:46 +00:00
Karol Latecki
3ef00ffb99 test/nvme_perf: fix nvme perf app directory
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I7f2b5bc3f25b444e1be8050a5ec74e884ff13068
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3292
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-07-22 12:22:46 +00:00
Maciej Szwed
e63eb0375d hw_hotplug: Fix HW hotplug test
Add missing exports

Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I35fe3d12ff278aaff62a19e4ce1ac95a96cc4ffa
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3377
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-07-21 09:23:09 +00:00
Changpeng Liu
919fdece12 test/nvme: enable OPAL test
Due to the firmware response time for REVERT action, currently
we only enable this test with P4510 drives.

Change-Id: I4b02008ec0b7d2caeba20c566b1cf1b69404ae12
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2816
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-07-16 07:30:36 +00:00
Tomasz Zawadzki
14ce25ab00 test/cuse: wait for device after reset
This patch adds waitforblk() after rebinding the driver.
If timing was unfortunate, it was possible to issue
id-ctrlr command to a device that was not yet ready.
Meaning the support for fw command was not read properly.

No error was reported due to "set +e", so this patch moves
it till after id-ctrlr.

Example of the error:
# get_nvme_ctrlr_from_bdf 0000:82:00.0
# readlink -f /sys/class/nvme/nvme9
# grep 0000:82:00.0/nvme/nvme
# bdf_sysfs_path=/sys/devices/pci0000:80/0000:80:03.0/0000:82:00.0/nvme/nvme9
# [[ -z /sys/devices/pci0000:80/0000:80:03.0/0000:82:00.0/nvme/nvme9 ]]
# basename /sys/devices/pci0000:80/0000:80:03.0/0000:82:00.0/nvme/nvme9
# printf '%s\n' nvme9
# nvme_name=nvme9
# [[ -z nvme9 ]]
# set +e
# ctrlr=/dev/nvme9
# ns=/dev/nvme9n1
# /usr/local/src/nvme-cli/nvme id-ctrl /dev/nvme9
# grep oacs
# cut -d: -f2
/dev/nvme9: Resource temporarily unavailable

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I5728a5062cd553eb39d18d9869fdf56319a27777
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2950
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2020-07-08 07:55:33 +00:00
Tomasz Zawadzki
f9122e9054 test/common: move nvme-cli compilation to common script
NVMe tests compiled the nvme-cli, but turns out that
NVMe-oF did not. This patch moves compilation to
common script. Meanwhile next one will use it in other tests.

Additionally return to the original directories with
pushd & popd.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ica60775a9b8e4347bed28d4c9dfae1acab2cae9f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2969
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-07-06 23:22:59 +00:00
Tomasz Zawadzki
9803ad73ea test/nvme-cli: clean all previous artifacts from builds
There are a lot of files in the nvme-cli repo on system that
either build up (*.core) or produce misleading logs (*.gcda).

This patch cleans the nvme-cli repository before compiling.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I8f169b9eaef2dc91b14b324a63b338611f82a6a5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2949
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2020-07-06 23:22:59 +00:00
Tomasz Zawadzki
af761ab167 test/nvme-cli: exit early when not running on Linux
Tests do not work on OS other than Linux, instead of
going through whole script - exit early with error.

Took this chance to move staring the stub, till after
nvme-cli compilation succeeds.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I5659888527bc2c25ab10bd871f7820b13f02e3fc
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2947
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2020-07-06 23:22:59 +00:00
Seth Howell
40d20acb72 autotest: always test against SPDK shared objects
We aren't going to run into any runtime issues when
linking against static libs, but we will when linking
against shared objects, so always test against shared
objects in autotest.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I2d3d6e90f5fa4c38b3f44cf35e24b4f7e43b60ba
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2980
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-07-01 07:50:27 +00:00
Seth Howell
3852afd6d8 test/nvme: in hotplug test copy lib files.
This gets us ready for the case where we want to run with
shared objects.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I84d22cf9def30a27aeb11fcf18eb80080a002ef9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3021
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-07-01 07:50:27 +00:00
Karol Latecki
8e8864f27e test/cuse: use 512B block size on namespace revert
Use 512B block size when running nvme_namespace_revert
routine. Need to use this block size because of
vhost_boot.sh test case.

Fixes #1452

Change-Id: I5f41cf2bf9d63b811849a99de6cba9b7b5a1d97d
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3003
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-06-23 08:04:05 +00:00
Seth Howell
0ebb046038 test/nvme.sh: rewrite nvme_fio test to find namespaces using identify
The old version of this test was using a function
that only detects NVMe drives bound to the kernel
driver so it was not finding any work to do.

Instead use identify to find NVMe namespaces.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I937fc364254a926b8d251ec0cb38ea026593d977
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2788
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-06-18 07:27:51 +00:00
Tomasz Zawadzki
f2f1060ed2 test/cuse: run namespace managment test if device supports it
In case when no device supporting Namespace Managment was found,
the test case exits without producing an error.
For example QEMU NVMe emulated device does not support it, thus
only physical devices have to be used right now.

This goes against design of "run_test" function, that when executed
should always run and never silently skip.

This patch adds check whether there is a device support and triggers
the test case only then.

Note that original code for selecting the device is still in original
script. This is to enable running nvme_ns_manage_cuse.sh on its own.

Right now CUSE tests are only enabled on VMs so nvme_ns_manage_cuse,
was added to exception list.

This will soon be reverted as the tests are moved to job that
runs on physical machines.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ie784696c2628b5fe45ce751b3978abcc03118cd9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2828
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2020-06-17 07:20:55 +00:00
Tomasz Zawadzki
2c47df9419 test/cuse: cleanup and better describe ns_manage test script
Added more comments for clarity.
Moved namespace size calculation to the clean_up() function.

Changed size calculation from nvmcap (id-ns) to tnvmcap (id-ctrlr).
There is noneed to cache the size before deleting namespace,
if we assume that all devices had single namespace encompassing the
whole device.

Changed blocksize from 512 to 4k. All devices in CI should support that,
and usually thats the default.
This could be further improved by reading LBAF and checking the Relative
Performance (RP), but that would convolute the test scripts. This
is good enough.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I8a38d00ae1442edf5480e4e72a8a8aa90380cd79
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2827
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2020-06-17 07:20:55 +00:00
Tomasz Zawadzki
933ba7f710 test/cuse: move nvme reset to reset_nvme_if_aer_unsupported() function
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I99925974b6004484a0d308ccd56120360b89b8ec
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2826
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2020-06-17 07:20:55 +00:00
Tomasz Zawadzki
4883aeb218 test/cuse: remove superfluous setup.sh
It is already assumed that enviroment was prepared,
setup.sh ran binding the drives.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I864590b57db880b33c7b32deba8d38e4d5637b5b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2825
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-06-17 07:20:55 +00:00
Tomasz Zawadzki
256f554d9e test/common: add get_nvme_ctrlr_from_bdf()
This is used couple times in cuse test, so move it to common script.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I1e9f973e7fb53b0024fb320d37632e1e011ba3e9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2824
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2020-06-17 07:20:55 +00:00
Tomasz Zawadzki
4e0eb251f7 test/cuse: move all nvme-cuse tests under single script
This will help with managing the test execution and preparation.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I0bd3612381d031a4015335d162f1296fd6dd213d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2823
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-06-17 07:20:55 +00:00
Tomasz Zawadzki
24260d2a0d test/cuse: fix conflicting name for cuse tests
"nvme_cuse" name for run_test was already used by one
of the tests in nvme.sh.

This removes the naming conflict and will prepare for further
restructure of nvme-cuse tests, where "nvme_cuse" name will
encompass all cuse tests.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: If141fe0ef492e0cde8e68490550de5d48fecb5e7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2822
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-06-17 07:20:55 +00:00
Changpeng Liu
d5ebb7e118 test/nvme_reserve: set host identifier at runtime
This will help us to change the host identifier if required, moreover,
some drives sometimes report set host identifier failed when startup,
so set host identifier at runtime making more sense.

Change-Id: I939c059d890b06aa2d51e82e0dede9630a6a1b51
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2860
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: wanghailiang <hailiangx.e.wang@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-06-15 15:27:50 +00:00
Changpeng Liu
6b757ecb0f test/nvme_reserve: rename the global variables with "g_" prefix
Change-Id: I6e031f78f9aa773ff105e4124240345b1669f684
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2859
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-06-15 15:27:50 +00:00
Ben Walker
85bdd43b40 build: Example applications build to build/example
This mirrors build/bin and build/lib

Change-Id: Ide1d17dfc8b425adfbe67f381c93c236653bd301
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2380
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2020-06-15 15:27:16 +00:00
Ben Walker
c3b5ae0c60 build: Place fio plugins into build/fio
This makes them easier to find. They are now named
spdk_nvme and spdk_bdev.

Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: I1f6736d8f7f5b9669583731fb0b710ba54f0d50a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2679
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2020-06-15 15:27:16 +00:00
Ben Walker
6b9b448e53 build: Output executables from the app directory to build/bin
Automatically place binaries produced from the app directory
into build/bin. This matches with the output in build/lib
that already exists.

Change-Id: I13cd2da71d2f88592e22308fe8a907bf458458b5
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2379
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2020-06-15 15:27:16 +00:00
Changpeng Liu
d3e9384c5a test/nvme_dp: return failure correctly when test it in CI
We will skip the test case if the drive didn't report such
capability, this isn't treated as an error case.

For vagrant environment, the namespace size is set to 1GiB or
2GiB, so here we also change the test LBA start from 0.

Also use printf instead of fprintf for normal exit.

Change-Id: Ib9f845ae548452921564aafebd2b23da6310b071
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2768
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-06-12 15:40:22 +00:00
Darek Stojaczyk
da2aa2aca0 test/nvme/hotplug: create QEMU host drives at startup
We used to create them at runtime with QEMU monitor, but
they can be created just once at startup. While here, start
using loops in bash to create/modify 4 equal drives.

Change-Id: Ie73ba3496a1cfb5d5a173118d331d90eee713270
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2144
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-06-08 09:33:35 +00:00
Darek Stojaczyk
85d0f9a83a test/nvme/hotplug: don't copy unnecessary files to the VM
This cuts down even 10 seconds. We only need a few binaries and spdk
./scripts directory on the VM, not all SPDK sources.

Change-Id: If63bdaec9ee4cf3f42ab7fce1bf107847580dbf7
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2145
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-06-08 09:33:35 +00:00
Darek Stojaczyk
4c6a9cdfa4 test/common: remove discover_bdevs()
There were many cases in the past where discover_bdevs()
is the first app to parse given (invalid) config and silently
fail without even printing the exact error. That's because
the spdk app that's started inside that function has &>/dev/null.

discover_bdevs() is currently used only in nvme perf scripts,
so move it there. It should be probably simplified there,
but the priority for now - and the purpose of this patch - is to
remove it from autotest_common.sh so that no one is tempted to use it.

Change-Id: I1d5840dde9272cdaa1ff282022c6ec29a297df50
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1554
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2020-06-05 09:03:50 +00:00
Tomasz Kulasek
f6fd2f521f test/nvme: add reset to ns manage test if AER ns change not supported
If Namespace Change AER is not supported by controller we
need to reset controller after changing namespace configuration.

Change-Id: I640e925fa02881c7b8541f2c0353e78c51074cdc
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1912
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-06-05 09:01:44 +00:00
Tomasz Kulasek
1e529db263 test/nvme: add nvme ns manage nvme-cli test
This test creates/deletes namespaces with Namespace Manage commands
and checks if appropriate NVMe CUSE device nodes are created/removed.

Change-Id: Iecbb4949c3843e41d4dbf365b0a57de3362c8947
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1131
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-06-05 09:01:44 +00:00
Tomasz Kulasek
2b9a331d0b test/nvme: dynamic nvme cuse devices update test
This test changes the range of active namespaces and checks
if apropriate CUSE devices for namespaces was stopped/started
properly on CUSE update.

Change-Id: I9683a0ab6ff28328d4788c49a834cef31c0582cc
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/780
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-06-05 09:01:44 +00:00
Darek Stojaczyk
482ca8eb60 test/nvme/hotplug: always print QEMU monitor output
Always print QEMU's output to the screen - there's not a lot
of it and something may come useful. Let's know what's happening.
The text-based QEMU monitor always runs in an interactive mode,
so post-process its output a bit:

$ monitor_cmd info status
QEMU 5.0.0 monitor - type 'help' for more information
(qemu) info status
VM status: running
(qemu)

Clean it up by removing the first line, then filter out any
lines prefixed with "(qemu) ".

Change-Id: I20b56bbdfc868bc11468dea5ef246cf1db1b4ab8
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2143
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-06-05 08:17:22 +00:00
Seth Howell
7ef497012a test/nvme: move nvme_fio test before kill_stub call
This test was after the part in the nvme.sh script where
we kill the stub. So the call to get_nvme_bdfs was invariably
returning nothing.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: Ice98cca8878ccff06057da5f96607eccffc5da39
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2746
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-06-03 07:39:08 +00:00
Changpeng Liu
da0ddc351d test/nvme_reserve: return failure if tests had errors
This will help us to capture the error case in CI.

Change-Id: I4ff348e492d6acd18ff4552b9a233965db5a1019
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2734
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-06-03 07:38:08 +00:00
Changpeng Liu
d9c6b6bb64 test/nvme_reserve: move examples/nvme/reserve to test/nvme/reserve
This only covered one test case for NVMe reservation feature, so remove
it to test directory.

Change-Id: I7920679ce14dd4d6a9164cb7663016bc7a3a4929
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2732
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-06-03 07:38:08 +00:00
Shuhei Matsumoto
948bf82228 test/nvme: Increment current_queue_depth only if I/O submission succeeded
current_queue_depth should have been incremented only if I/O submission
succeeded.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ib26b42b0ab195eba7536b2403a31d42305d77433
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2585
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-05-29 14:58:30 +00:00