Commit Graph

157 Commits

Author SHA1 Message Date
Karol Latecki
02cb1fe86a scripts/nvmf_perf: allow ADQ for SPDK mode only
Need more investigating how to properly enable ADQ
for Kernel mode. Do not run Kernel + ADQ configuration
for now.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I700ef417b9b398c067a586a1a5ad9947a92057ad
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6566
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-04-21 23:59:40 +00:00
Karol Latecki
b06212cfb5 scripts/nvmf_perf: re-write nvmf perf readme file
Readme was frequently missed when adding new stuff or
updating the performance script, and it is out of date.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I1710473be576ffbcc2fa8e3701b196bd46cf6654
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6538
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-04-21 23:59:40 +00:00
Karol Latecki
c0fc19f36d scripts/nvmf_perf: configure adq traffic classes
Create a default traffic class (tc0) with minimum
needed number of assigned threads and a priority class
(tc1) with number of assigned threads equal to
number of application threads.
Finally run set_xps_rxqs to configure symmertic queues.

set_xps_rxqs script used from Intel ICE driver
package available at:
https://downloadcenter.intel.com/download/29746/
Intel-Network-Adapter-Driver-for-E810-Series-Devices-under-Linux-

Change-Id: Ie0f2db266621a9dabb1621344bfdc5fa64fee03c
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6537
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
2021-04-21 23:59:40 +00:00
Karol Latecki
42d2e588d6 scripts/nvmf_perf: Keep track of num_cores for Initiator class
There are multiple cases for counting number of CPU threads
on Initiator side. Keep track of num_cores in all of them.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I4b8d1d79a2aa17706d206b149c1c8e8816ae2ef5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6536
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
2021-04-21 23:59:40 +00:00
Karol Latecki
31c50e110c scripts/nvmf_perf: disable sysctl busy_read timeout
Set busy_poll to 0 for non-ADQ. Having it enabled with
this value results is huge latency increase.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: Ic68ed2b7dc19b8d4b274ce8622194489693863cb
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7480
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-04-21 23:59:40 +00:00
Karol Latecki
4b3a60daee scripts/nvmf_perf: rename SPDK Target num_cores to core_mask
num_cores name was misleading, as the parameter actually
contained core mask (or core list). Create separate
class attributes with core mask and number of cores.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: Ibfae7770aea2f2c1c720abf567400deb50028ab4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6535
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
2021-04-15 16:33:26 +00:00
Karol Latecki
128df53566 scripts/nvmf_perf: save subsystem info
Save info about subsystems created subsystems (target
side) and remote subsystems (initiator side) into
a list.

Change-Id: I7ec1c6a4d1f75060b69b68e0b8e0dd61789fc9e0
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6374
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-04-15 16:33:26 +00:00
Karol Latecki
696d55bf94 scripts/nvmf_perf: run set_irq_affinity
Run set_irq_affinity script after irqbalance service
is disabled and manually tune IRQs to be run only on
the CPUs local to NIC NUMA node.

set_irq_affinit.sh script must be manually downloaded
as part of https://github.com/Mellanox/mlnx-tools/tree/master/ofed_scripts
package before running tests.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I67b3c851bcf283caea29ad36f6d4bc0322ca0d27
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6534
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
2021-04-15 16:33:26 +00:00
Karol Latecki
3992eb2896 scripts/nvmf_perf: set cpu governor to performance
In performance tests always use best performance policy.
Previously this was manually managed on test systems,
but it's better to do this automatically every time.

Change-Id: Iff81863cf8d9cc713a3c4cce1d8edf7ebbf81c84
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6373
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
2021-04-15 16:33:26 +00:00
Karol Latecki
2f9103304f scripts/nvmf_perf: add configure_tuned method
Modify CPU power management settings to achieve high,
consistent results.

Change-Id: I3602ae7523c5b83878238928caaeb453f7d2533e
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6372
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
2021-04-15 16:33:26 +00:00
Karol Latecki
81ccc81dde scripts/nvmf_perf: add configure_sysctl method
Tune sysctl parameters for TCP testing. Restore previous
settings after tests have finished. For ADQ-enabled tests
also set proper value for busy_read option.
Commit includes a fix to initiator exec_cmd() method to
allow using command parameters which contains whitespace,
otherwise it's not possible to set some of sysctl params.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I3376b69b8d7c0d8a282765db4fe55824f55f9e05
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6265
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
2021-04-15 16:33:26 +00:00
Karol Latecki
a35f2b34dd scripts/nvmf_perf: add configure_services method
Disable all unnecessary services which might affect
tests performance.
Managing services implemented in a very simplified way
usign subprocess and configparser modules (because json
output is not available). Proper implementation would
need to use a proper dBus API, but this seems like an
overkill for such a script.

Change-Id: I72feec4293b77442a9d2c9b4afaa032df1c4d5e1
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6264
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
2021-04-15 16:33:26 +00:00
Karol Latecki
2c5849c7be scripts/nvmf_perf: add adq_configure_nic method
Configure NIC settings as per ADQ configuration guide.

Change-Id: I957aada267474c2a1448c89a1b7c81d4dd261ca6
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6263
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-04-15 16:33:26 +00:00
Karol Latecki
3c09b2fb99 scripts/nvmf_perf: add adq_load_modules method
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: If2784b74ece6c48b81cc53e63412cd2bc618ffef
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6262
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-04-15 16:33:26 +00:00
Karol Latecki
26d7e3d4bb scripts/nvmf_perf: use common method name for calling subprocesses
Use common method name and parameters for calling
subprocesses for local (Target) and remote systems
(Initiators) instead of using "check_output" (from
subprocess module) and "exec_command" (from paramiko)
separately.
Having these functions wrapped by a single common
method will allow to create common methods in Server
class more eaisly, instead of creating two copies in
Target and Initiator classes.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I1c10f6a88f3d7300c227e969ad6fd901763ac52c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6261
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
2021-04-13 08:40:32 +00:00
Karol Latecki
f9dd94b515 scripts/nvmf_perf: add adq_enable option to configuration file
Parse the config file for "adq_enable" flag and enable
the option in SPDK Target and SPDK Initiator configuration.
At this point the flag has no effect on the generated
traffic, as the system and priority queues are not
configured properly. These settings will be added
gradually in following patches.

Change-Id: I5d7bc892f3aa7bbe691b7e0983294b56fcf994f5
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6260
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-04-13 08:40:32 +00:00
Karol Latecki
ddb4d24b1d scripts/nvmf_perf: get detailed info about NICs in servers
Get detailed info about NICs installed in servers. This
is using lshw utility to make things easier and not
implement the whole logic for parsing /sys objects and
pci.ids listing in system.

Change-Id: I97871fdc9feaae1c2485574a7b488b88ac3afc4f
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6259
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
2021-04-13 08:40:32 +00:00
Karol Latecki
9fec8853fb scripts/nvmf_perf: re-work managing test options
The number of options for test execution grew high,
and we still need to add more. This results in a lot
of parameters to pass around in constructors, which
is hard to read and causes pylint to complain.

Instead of passing each option individually as a
separate parameter, pass them as dictionaries just
like they're defined in .json config file. This
makes managing the default values a bit harder, but
is more readable.

Change-Id: I5d88a2b7fe51d2df93edd9130678a937d34facdd
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6207
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-04-13 08:40:32 +00:00
Maciej Wawryk
1fc0c2d8b9 test: move spdk_test_image.qcow2 to spdk_dependencies dir
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I8d75f5fa22e79c93d1f8929b6df048b440d9fb8a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6940
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-04-13 05:13:50 +00:00
Maciej Wawryk
0872c7edbd nvmf/perf: Fix type error
TypeError: expected str, bytes or os.PathLike object, not int

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I0b395035b43fd4ef5e10ae152889c737ccf66c03
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7054
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-03-29 07:40:16 +00:00
Karol Latecki
78ce3152b5 scripts/nvmf_perf: add function to get interface names
Interface names will later be needed for additional
test options.

Change-Id: Ic16d305a97087ab1fdc35d5fd5a3d1ed2a021d90
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6206
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Mellanox Build Bot
2021-02-19 11:29:18 +00:00
Karol Latecki
c4a330c937 scripts/nvmf_perf: add remote_nic_ips field to configuration
So far the information about remote target IP addresses was
stored in "nic_ips" attribute. Add a new "remote_nic_ips"
attribute to be able to differantiate which IPs are local
to given Server class object.

Change-Id: I03dac140241341996674b350c51c18da6e2ee447
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6205
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-02-19 11:29:18 +00:00
Simon A. F. Lund
f79d4b8baf scripts/nvmf_perf: fix pylint E741
Signed-off-by: Simon A. F. Lund <simon.lund@samsung.com>
Change-Id: I039a2e22a665e0d52082e6876f2c7c1cd9a336c1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6386
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
2021-02-16 08:43:24 +00:00
Karol Latecki
4dd1548be3 scripts/nvmf_perf: check "extra_params" key berofe accessing it
If config without this field was used then
script resulted in KeyError exception.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: Icf0b270d28dcc6bf44b66c4b9ed583a6b3ef08b6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6204
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
2021-02-04 08:33:43 +00:00
Karol Latecki
62c2c1c110 scripts/nvmf_perf: fix pylint E1111 error - assignment on no return
Fix assigning variables from functions which return nothing.

Change-Id: Ic54dc0d42e3339cda23970ed347ac6f7bf92e795
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6203
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
2021-02-04 08:33:43 +00:00
Karol Latecki
6882e04563 scripts/nvmf_perf: fix pylint C0411 wrong import order
Standard lib modules first, then pip installed modules
and local modules last.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: If0159fad29bab5bef0b69aa803e3cea429cc5f25
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6202
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2021-02-04 08:33:43 +00:00
Karol Latecki
22e240eb87 scripts/nvmf_perf: do not use "shell" for python subprocess
Using shell=True was unnecessary, as it spawned additional
shell process for any subprocess with this option set.
We don't need that in this script and it's generally
discouraged.
Also removing check=True from some calls as don't have
these subprocess calls in try/excepy anyways.

Change-Id: Ibb9b5d71119c9dd877209ea77b8351f9b610cade
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6046
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2021-01-22 18:32:19 +00:00
Karol Latecki
abe273b8d1 scripts/nvmf_perf: unify PCM options into single setting
Instead enabling each measurement (cpu/power/memory) just
use a single flag and enable them all at once.

Change-Id: I8a639b697285aa741f28b5ddec11ce70c34fc760
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5980
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-01-22 18:32:19 +00:00
Karol Latecki
90761053ef scripts/nvmf_perf: stop pcm-memory using sigterm
Use terminate() instead of kill() to exit more gracefully.
Using kill() resulted in (sometimes) output not being saved.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I3ff1dec8cb0804f6e0674c5ed61bbe4ec9ea70ef
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6045
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-01-22 18:32:19 +00:00
Karol Latecki
79199d4164 scripts/nvmf_perf: create results dir early
"measure_*" threads in script use the same results dir
as the results parsing function. To avoid race try to
create results destination directory early.

Change-Id: Ic391a7c2fb3decff64f1fdf51782abb940706d38
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6044
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-01-22 18:32:19 +00:00
Karol Latecki
55e5baf993 scripts/nvmf_perf: add PCM power measurements
Enable pcm-power measurements with appriate option.
This will be used for scheduler tests.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I57acc9905c6f1acd7bc3ecdbf886b209a2cdb653
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5979
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2021-01-21 14:24:19 +00:00
Maciej Wawryk
e7fb0e9a8a scripts/nvmf: Add scheduler_setting option in NVMe-oF performance scripts
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I1933b1c3026c86b55c3e2b712bff02fa8f711bc7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5446
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-01-05 09:38:00 +00:00
Maciej Wawryk
da5c9b328a scripts/nvmf: enable zcopy on target side
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I5fb99ebf57e7989f23d877798feddddd70179d4b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5297
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-01-05 09:38:00 +00:00
Maciej Wawryk
632703f681 scripts/nvmf: Capture configuration in perf tests
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I343e568f3c4f6d16a7159c8cdc460df2d2c842ca
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5252
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-12-11 11:01:58 +00:00
Karol Latecki
1bc2a2d9e9 scripts/nvmf_perf: fix clat percentile calculations
In case of latest fio versions "percentile" leaf for clat
latency measurements does not get created in case no IO
was run (e.g. write clat will not have "percentiles"
created in case rw mode was set to randread). This
caused KeyError exception when trying to access non-
existent dict key.

Change-Id: Ie0c28dc932481e5aab971505ba0f38b6798bff36
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5466
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-12-10 14:50:09 +00:00
Jim Harris
9289521630 test/vhost: replace master_core with main_core
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ibdf9612a27266ef739d9d10db3fd1d353729644a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5353
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-12-03 09:41:07 +00:00
Maciej Wawryk
c4c37f1cf1 Modifications for using universal qcow2 image in tests
We want to replace few qcow2 images with one universal.
This commit contains:
 - change password in autotest.sh
 - change image path
 - change image name
 - use snapshot mode in hotplug.sh instead of copying base image

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I75c457fe75f005b0ab43ca909be7886529ed115b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4551
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2020-10-28 15:07:56 +00:00
Karol Latecki
dc45a73d9e perf/nvmf: replace bdev INI configs with JSON
INI is deprecated, use JSON configs for fio plugin instead.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I3075e79ed7601116e837536a818821372362eb8d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4762
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-10-22 10:12:02 +00:00
Karol Latecki
a8e5914833 perf/nvmf: remove pylin no-member error
Add an abstract method in Initiator class to get
rid of pylint "no-member" error.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I83ac1a498d55f6390a5b36038a9532121a9e5d0a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3922
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-08-28 07:30:36 +00:00
Karol Latecki
f252ff1f16 perf/nvmf: add option for TCP dif_insert_or_strip
Change-Id: Id5b6082f0ac87e76335f807d1758693d6813bf98
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3920
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-08-28 07:30:36 +00:00
Karol Latecki
66f42f3703 perf/nvmf: set protection information on null bdevs
Use --dif-type and --md-size option for bdev_null_create
rpc when creating null bdevs. Needed for DIF/DIX comparison
tests.

Change-Id: I15de25d498ff7c239cfca291e10aa16acb4f09eb
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3919
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-08-28 07:30:36 +00:00
Karol Latecki
7049fe936f perf/nvmf: Allow using multiple nullblock devices
So far only single null block device was allowed in
the setup for the purpose of latency measurements.
Rename setting to "null_block_devices" and use an int,
rather than a bool, so that multiple null block
devices are created.

Change-Id: Id543cf3bfbc7d9c171f8605909e1971cabbdfcc7
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3900
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-08-28 07:30:36 +00:00
Karol Latecki
c7f8e73906 scripts/nvmf_perf: fix syntax error
Missing comma causes Python interpreter to
raise syntax error.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: Id86bd10fb55ae2284ebc1c0c36babfe946302d1a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3897
Reviewed-by: Seth Howell <seth.howell@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-08-21 17:59:13 +00:00
Maciej Wawryk
d1fc288376 scripts/nvmf: add dpdk memory measurements
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I6c836114b71b1c5ee1dbbf7900c7314bb5619a48
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3551
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-08-21 08:25:22 +00:00
Maciej Wawryk
e90d35578a scripts/nvmf: Remove unnecessary function call
Function get_nvme_devices_count in kernel mode is
unnecessary called, this caused fails.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Ie1c613f9a46707b9e4f821d03644a8b45bf52966
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3508
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-07-24 15:36:04 +00:00
Maciej Wawryk
bf601d5ae4 scripts/nvmf: Add Linux to grep nvme list
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I1a0d11e14df481f62752da7dcf622e5eb38e26e0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3499
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-07-24 15:36:04 +00:00
Maciej Wawryk
efd3f83d72 scripts/nvmf: move get_nvme_devices_count function
Getting number of nvme devices before check if this is
null_block test caused fails.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I55b0399561ddb11b2f6c36408a56c525e813d239
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3497
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-07-24 15:36:04 +00:00
Maciej Wawryk
cb94476cfc scripts/nvmf: Set minimal job_section_qd value
job_section_qd cannot be equal 0 because tests will fails.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I910ae8dbbe7e18bc79f8d69a8ae157162b3b861b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3478
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2020-07-24 15:36:04 +00:00
Maciej Wawryk
7dbaf54bf5 scripts/nvmf: Fixed python syntax error
Missing comma in nvmet_cfg caused this error.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I71864c8dc443e2bb73cd9f575eb6540e16bab40e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3470
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-07-23 07:29:01 +00:00
Maciej Wawryk
1cab8c9f19 scripts/perf: Choose only nvmf device
During nvmf perf test, on machines with have local
nvme device figure out that they are added to
fio.conf file, that can caused bad for results.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Ibb6aa7b29ee63a8c41cd80909bc26d1dc224dd0c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3113
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-07-16 07:30:06 +00:00
Maciej Wawryk
3c8451e69a scripts/perf: Add bandwidth measurement to nvmf test
Bandwidth measurement is realized by bwm-ng tool.
Using this measure tool will help us to check network saturation.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I7b95b9b4ac3ca3b1847ec72ab7ea333807befe89
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3190
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-07-14 07:55:08 +00:00
Maciej Wawryk
9de6d03ee4 scripts/perf: Fix kernel fio filename generator
Also check nullblock subsystem number.
There was missing subsystem number definition in tgt_start
for null_block tests, and wrong filenames generator in
kernel section.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Iae5bc6f497176d6c2a12ab430f4c6c92104fa4a5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2570
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-07-07 07:30:44 +00:00
Maciej Wawryk
af0057e6e1 scripts/perf: Add effective QD counting
During perf test we figured out, that global qd
settings is split between disks number in filename.
Setting this parameter (multiplied by section disks number
and numjobs) in filename section give as expected value.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I89a44e48f10da131e2e9128f3ff2fbfb197076ad
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2749
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2020-07-01 07:53:54 +00:00
Karol Latecki
c0ff295493 perf/nvmf: add option to skip SPDK build on initiators
Skip copying and building SPDK on host systems if
specified in configuration file.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I4f9c0dd77e020dfd866f52dc138526e63dd9222d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3075
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
2020-07-01 07:53:32 +00:00
Karol Latecki
d3f46b279b perf/nvmf: aggregate read and write results
Aggregate separate read/write results into
a single stat. Reduces the number of additional
fields and formulas in spreadsheets when preparing
performance reports.

Change-Id: I5fdab35c1bb69aad98f7b9808885801db0416449
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3068
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-07-01 07:53:32 +00:00
Karol Latecki
9228dd5dd7 perf/nvmf: sum results from all test initiators properly
Test result IOPS and BW are a sum from across all initiators,
but latencies should be an average from accross initiators.

In previous version it was mistakenly a sum, and that had
to be manually handled in Excel spreadsheets when gathering
data for performance reports.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I17e38e3cdd593dcda9730c19decc2d6a72a11130
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3067
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
2020-07-01 07:53:32 +00:00
Karol Latecki
baa1f8a4a1 perf/nvmf: save per initiator results to a file
Save each initiator results to a .csv file. Useful
for a bit more insight than aggregated results.

Change-Id: I1fafe52ba78f2c62abfd0e3c863505515a363cd3
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3053
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
2020-07-01 07:53:32 +00:00
WANGHAILIANG
74050eacd7 script/perf: move "make clean" before "./configure"
I suggest putting "make clean" before "./configure". Like Migration-tc3a.sh (spdk/test/vhost/migration):
ssh_remote $MGMT_INITIATOR_IP "cd $spdk_repo_share_dir/spdk; make clean; ./configure --with-rdma --enable-debug; make -j40"
Otherwise, once the configuration is changed, some outdated compiled files may remain after "make clean".

Change-Id: I7300749986129cbfaef44f8ff72fad0449f4e081
Signed-off-by: WANGHAILIANG <hailiangx.e.wang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2805
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Liang Yan <liang.z.yan@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-06-26 07:21:15 +00:00
Ben Walker
c3b5ae0c60 build: Place fio plugins into build/fio
This makes them easier to find. They are now named
spdk_nvme and spdk_bdev.

Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: I1f6736d8f7f5b9669583731fb0b710ba54f0d50a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2679
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2020-06-15 15:27:16 +00:00
Ben Walker
6b9b448e53 build: Output executables from the app directory to build/bin
Automatically place binaries produced from the app directory
into build/bin. This matches with the output in build/lib
that already exists.

Change-Id: I13cd2da71d2f88592e22308fe8a907bf458458b5
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2379
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2020-06-15 15:27:16 +00:00
Maciej Wawryk
6f702d4ea7 scripts/perf: Fix type error in run_nvmf.py
TypeError: unsupported operand type(s) for /: 'int' and 'range'

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I8266c80cc886a51b43d9719b8bf845ffd0730b03
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2259
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-05-29 14:59:40 +00:00
Michal Berger
ae7300425b autobuild: Use mktemp() for spdk workspace
With issues similar to https://github.com/spdk/spdk/issues/1293 it
would be better to use temporary directories for spdk workspace
instead of depending on the hardcoded path (/tmp/spdk). This will
mitigate potential scenario where permissions of the target dir
are suddenly changed in-between autobuild runs.

Change-Id: If44a2dcce712a185287186f33e7361223dc6d451
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1863
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-05-07 10:41:08 +00:00
Maciej Wawryk
6ec892634e scripts/perf: Enable percentile 999 measurements in fio config
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I32c534196108c4795bcd6cd8392e3a1b0a5dd6e9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1842
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2020-04-17 07:31:55 +00:00
Maciej Wawryk
451c9e7b69 scripts/perf: Add frequency-info after changing cpu frequency
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Icf45fbdff547fcf45c361b1d9c95b352756ad126
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1775
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-04-16 08:07:14 +00:00
Maciej Wawryk
282b1505e5 scripts/perf: Fix cpu_frequency parametr passing
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I9617cd92d20b07c32761fb67ce6a1b30e0985831
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1697
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-04-09 08:08:38 +00:00
Maciej Wawryk
a61af6cfc9 scripts/perf: replace dots by floor sign in percentile variable
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I34e912dd207f16b1926edb37df41128a0524bc4c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1711
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-04-09 08:08:17 +00:00
Maciej Wawryk
7c1e69b3fa scripts/nvmf: Fix subsystem divide and format error
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I874c3a28c453737d0ba6ab42ec0d0a9fbc4719f7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1679
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-04-08 06:43:16 +00:00
Karol Latecki
db43b387ba test: do not use iter_pci_class_code in tests
Do not use iter_pci_class_code function in tests to
iterate over NVMe drives. This function can return
drives which at the moment of execution can not be
whitelisted for use.

This can result in test errors (such as simply
bdev_nvme_attach_controller RPC command failing) or
even using and deleting data from NVMe drive which
was not meant to be used in tests.

Fixes #1235

Change-Id: I82b9935fc88605b636c2096be6c71d4880a567c8
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1309
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-03-18 08:03:21 +00:00
Maciej Wawryk
0172f27974 perf/nvmf: Add more latency measurements
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I1ff5e6440ebef50d0488f020edfd8d5b7e274162
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1281
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-03-18 08:03:02 +00:00
Maciej Wawryk
3509fc12df Add cpus_allowed_policy param for NVMeOF benchmark tests
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I82a1632d4bb8742bd93f33a5a094a9cb4aa2e927
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1259
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2020-03-17 16:38:16 +00:00
Maciej Wawryk
0ebd0dce19 Add subsystem division in Kernel initiator
This will allow us to use num_cores and cpus_allowed option also
in Kernel initiator side.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Iad4e8bf7cd8a614ada8f443a8b1036dc2c4803e4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1173
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2020-03-16 08:45:50 +00:00
Maciej Wawryk
0eab351528 Add cpu_frequency param for NVMeOF benchmark tests
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Id594da091d7bcd5d23a752ccfab9bfa5b7726b49
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1113
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-03-16 08:45:50 +00:00
Maciej Wawryk
e068db83f1 Add cpus_allowed param for NVMeOF benchmark tests
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I0d113e39c1ae9ff284f408998f5bba82352a2891
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/993
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-03-16 08:45:50 +00:00
Maciej Wawryk
23f78f22cf scripts/perf: extract SKT section to new file
PCM cpu measure give as result 1306 columns, we want check only ones which contains
SKTdata and SKTtraffic values.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I52b6781a7ff99e0d894f7f1c2b25b655fa274eb1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/946
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-03-02 10:33:48 +00:00
Karol Latecki
93be26a51d Fix Markdown MD022 linter warnings - headers blank lines
MD022 Headers should be surrounded by blank lines

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I768324b00fc684c254aff6a85b93d9aed7a0cee5
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/656
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-02-17 10:07:21 +00:00
Maciej Wawryk
db65d58359 Add pcm measurement to nvmf tests
PCM - Processor Counter Monitor is an application to monitor performance
and energy metrics of Intel® Core™, Xeon®, Atom™ and Xeon Phi™ processors.

Using this measure tool will help us to capture memory bandwidth consumed
on Intel processors.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Ie580d2aad6ddd98c5256df65f65e478dd78317e5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/541
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-02-11 08:37:42 +00:00
Karol Latecki
3d8a0b19b0 doc: Fix Markdown MD032 linter warnings
"MD032 Lists should be surrounded by blank lines"
Fix this markdown linter error by inserting newlines or
adjusting text to list points using spaces.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I09e1f021b8e95e0c6c58c393d7ecc11ce61c3132
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/434
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
2020-02-06 09:46:06 +00:00
Seth Howell
f257be1313 doc: replace all cloudfront refs with ci.spdk.io
It's good to have a human readable domain again.

Change-Id: If773e28a267b635a94a45b07fba9a32d62d0c248
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/483146
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-31 12:36:19 +00:00
Karol Latecki
f1d0fc75cf scripts/perf: use additional kernel NVMe-oF initiator params
Add configuration option "extra params" to run kernel initiator
with nvme connect options like -i, -I, -D, etc.

Change-Id: I01d9fd0ed15dc5607f0c0f58d97581c5e0b821ba
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/477767
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-12-24 08:13:14 +00:00
Karol Latecki
f610df8d52 scripts/perf: remove hardcoded NVMeOF initiator parameter
Removing hardcoded "-i 8" value for kernel NVMe-oF initiator
parameter "nr-io-queues".

This was previously merged by mistake and had to be removed
manually in test runs.

Change-Id: I454ad468863f93ac2ce93a1f0bd00f678ed4995c
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/477766
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-by: Seth Howell <seth.howell@intel.com>
2019-12-24 08:13:14 +00:00
Karol Latecki
9dfe2a4bce scripts/perf: clean up constructor parameters
Move code around so that lines containing arguments
for constructors are better aligned with how they're
inherited.

Change nvmet_dir and fio_dir to nvmet_bin and fio_bin to
point directly to executable files. There is no advantage
in using directories.
Additionally fix how fio_bin (previously fio_dir) is passed
to parent classes - it was broken and caused exceptions.

Update README.md accordingly.

Change-Id: I10524593d543e236bf318a8cd51ea5375c334e8b
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/477015
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-12-24 08:13:14 +00:00
John Kariuki
f161e76dff scripts/perf/nvmf: remove hard-coded fio directory
Added Initiator configuration option to point to the
fio source directory used when compiling SPDK with the --with-fio
option and run SPDK test with the fio plugin.

Signed-off-by: John Kariuki <John.K.Kariuki@intel.com>
Change-Id: I097cb17691a72d7c2a0e198dccd12cf3d1c3474c
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/468890
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2019-10-10 16:19:23 +00:00
Maciej Wawryk
0772af4de9 RPC: rename get_nvmf_transports to nvmf_get_transports
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I088070d130f46113efe4f89baae2508060ef33b3
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/469114
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-09-26 15:19:43 +00:00
Maciej Wawryk
7538af7034 RPC: rename nvmf_subsystem_create to nvmf_create_subsystem
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Ic3aa9f168b2e612e461c450ef1e5e1ae81fa5635
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/468963
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-09-24 16:42:41 +00:00
Maciej Wawryk
80a0c5be99 RPC: rename get_nvmf_subsystems to nvmf_get_subsystems
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Ief52fefa547535b23f2a191b1dbc9b9659893c09
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/468955
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-09-24 16:42:41 +00:00
Pawel Kaminski
f54df84058 rpc: Rename construct_nvme_bdev to bdev_nvme_attach_controller
Change-Id: I1831eb39d3d357594968271b2ee69bb48502a59c
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/466138
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2019-09-18 21:53:25 +00:00
Maciej Wawryk
2c49e910cb RPC: rename rpc get_bdevs to bdev_get_bdevs
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I899c05963f4dd5f29df0b10fc030c27029c2c246
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/468057
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-09-13 22:39:15 +00:00
Pawel Kaminski
60563dfe7f scripts/rpc: Rename rpc related to null bdev
Rename construct_null_bdev to bdev_null_create
Rename delete_null_bdev to bdev_null_delete

Change-Id: I2390b14ac62130eaf20b2c43ebd30c763eca34de
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/465789
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-08-28 15:19:58 +00:00
Ben Walker
4a770fa9f1 test/vhost: Export default VM_IMAGE from common.sh
Also, construct this path by grabbing the home directory
of the current user.

Change-Id: Ia8d5cc2be40c0c3ac693c80a3132b970f7124183
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456704
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-03 04:15:18 +00:00
Liang Yan
4ebf5cef63 perf/nvmf: add link files into zipped SPDK source codes
Change-Id: I9581c936a40be8ccf352778c62bba09ab5a9428e
Signed-off-by: Liang Yan <liang.z.yan@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459091
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-06-28 05:36:37 +00:00
Liang Yan
f240e2bf48 perf/nvmf: add config file path handling codes
Change-Id: I27c8c4a03237c75761c75cce824a65fe3e4776db
Signed-off-by: Liang Yan <liang.z.yan@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/454760
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-06-19 07:14:06 +00:00
John Kariuki
d601374387 scripts/nvmf performance: fix config param and connection error
Fixed the following:
1) Missing python paramiko module in package dependecies.
2) Paramiko connection error: TypeError: missing_host_key() missing 1 required positional argument: 'key'
The set_missing_host_key_policy call should take a class instance AutoAddPolicy(), instead of the class name.
3) JSON config file defines "rdma_ips" instead of "nic_ips"
4) Added result file location in the documentation
5) Added env variable for PYTHONPATH to documentation
6) config.json changed default rw to randrw instead of read

Change-Id: I96624e2912131f62254c684a6c03a53a7f806cde
Signed-off-by: John Kariuki <John.K.Kariuki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455285
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Liang Yan <liang.z.yan@intel.com>
2019-05-30 22:01:13 +00:00
Karol Latecki
4b8405685a scripts/vhost: Modify parameters for vhost benchmark job.
Add missing parameter numjobs. It is used in one of the cases.
Remove unnecessary arguments:
- cpumask: we do not want to restrict that
- percentile list: we do not want to restrict that, all
  percentule buckets can be generated.

Change-Id: I6a4431af0defe92920f16d26091d67161e56dc5a
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/450519
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-04-10 21:05:43 +00:00
Karol Latecki
0153a2e9d6 scripts/perf: wrap remote ssh command into a function
Wrap ssh_connection.exec_command() and related decode calls
into a function.

Change-Id: Ia56d453358e086df53fa179be643da6416218bec
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/450018
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-by: Paweł Niedźwiecki <pawelx.niedzwiecki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-04-08 23:20:51 +00:00
Karol Latecki
7e2052e309 scripts/perf: Update NVMeOF scripts to support TCP transport layer.
For future NVMeOF benchmarking scripts must support TCP layer.
This is a recent feature so also added support of using custom
built nvme-cli.
Example config file was updated with appropiate fields.

Rename rdma_ips to nic_ips, as NICs no longer need to be
RDMA-enabled for this test.

Change-Id: I368b48eca08a30b4221f198cb76134b8836dbba9
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449879
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-04-08 23:20:51 +00:00
Seth Howell
33197a1bbd doc: update links to the performance reports.
The changes I made to the doc directory in spdk.github.io got
overwritten.

Change-Id: I468c7eeb8c36ccd969909e179c216d5def812602
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449360
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Paweł Niedźwiecki <pawelx.niedzwiecki@intel.com>
2019-03-29 18:54:37 +00:00
Karol Latecki
02d7812f46 scripts/perf: Update results parsing in nvmf benchmark scripts
Wrong indent level caused results to be duplicated in final
output file if configuration file was used to run more than one
workload combination.

Change-Id: I0fadc10eba38898df648ec2da5fbd0440a3b9693
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449150
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-03-29 04:41:02 +00:00
Karol Latecki
a42dfab18e scripts/perf: SPDK NVMeOF benchmark scripts
Automation scripts for reproducing scenarios described in
NVMeOF performance report by Vishal.
https://ci.spdk.io/download/performance-reports/SPDK_nvmeof_perf_report_18.04.pdf

Change-Id: I006f68b67b5f42f2cc1ec82a158fc86e76371ba1
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/441898
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-03-13 23:19:10 +00:00
Karol Latecki
80a9ef4042 test/vhost: vhost benchmark scripts update
- Add option to throttle iops in VMs using cgroups
- Add option to measure CPU utilization in VMs using SAR
- Add option to limit kernel vhost CPU cores (not NUMA optimized)
- Add option to do lvol preconditioning using fio bdev plugin
  before running IO performance test

Change-Id: I7e0fcf977be96ecf837385c2abc9d5dabbe2f8c5
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/434229
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Paweł Niedźwiecki <pawelx.niedzwiecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-01-29 20:23:50 +00:00
Karol Latecki
1250a41664 scripts/perf: fix nvme perf scripts
Due to change of defauly Python interpreter to Python3
we need to decode byte object from check_output()
to utf-8, otherwise there is an error.

Change-Id: I83e2d79ec8c3934c5c6d00768288fbb4c5a50914
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/428172
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-10-11 20:46:18 +00:00