Add RPC bdev_nvme_get_path_iostat to show I/O statistics for IO paths
of the NVMe bdev.
Change-Id: I22e5ad112d5cfa6d96cf246dcd0e511ae71dc839
Signed-off-by: Richael Zhuang <richael.zhuang@arm.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14745
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Recent changes done to iobuf and accel framework
require us to adjust iobuf pool sizes when running
tests with high number of NVMe-oF subsystems.
Change-Id: I72ce6e71446bea0af44bdad6258e7d7f5d765656
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16288
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
This is meant to improve performance of the heaviest checks, like
the shellcheck one, where instead of running against the contents of
the entire repo simply go through files that are meant to be
committed.
This mode has to be requested through the environment via the
CHECK_FORMAT_ONLY_DIFF.
For now, get_diffed_dups() is hooked into least performant
checks (permissions, shfmt, shellcheck).
Also, the git grep command in the main check_permissions() is dropped
as the idea of piping its output to git ls-files was faulty - the git
ls-files does not read its arguments from the stdin.
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I098dcbbe18c08c08a8216857c7cf2c80c4021d31
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16049
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This is the port of the vbdev compress logic into the accel
framework. It includes just one enhancement, to only fill each
mbuf in either src or dst array with max "window size" param to
avoid QAT errors. Note that DPDK ISAL PMD was not ported as we
have native ISAL compression in accel now.
Note: ISAL w/DPDK is still built w/this patch, that can't be
removed until the vbdev module moves to accel fw as it still
depends on DPDK ISAL PMD.
Follow-on patches will include addition C API for PMD selection,
this patch just gets equivalent functionality going. Upcoming
patches will also convert the vbdev compress module to use the
accel framework instead of talking directly to compressdev.
More patches will also address comments on vbdev common code
that addressed here would make the review challenging.
This patch also fixes a bug in the ported code that needs to
be fixed here to pass CI. Capability discovery was incorrect
causing all devices to appear to not support chained mbufs,
with the mbuf splitting code this is important to get right.
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I7f526404819b145ef26e40877122ba80a02fcf51
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15178
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
This patch adds virtio_blk_get_transports matching the
virtio_blk_create_transport RPC. Allowing for querying
existing virtio_blk transports and displaying their options.
Signed-off-by: Krystyna Szybalska <krystyna.szybalska@gmail.com>
Change-Id: I0ec49c5f2ad11962feb5087dd376407ad125c349
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16303
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Currently we have stat per bdev I/O channel, but for NVMe bdev
multipath, we don't have stat per I/O path. Especially for
active-active mode, we may want to observe each path's statistics.
This patch support IO stat for nvme_io_path. Record each nvme_io_path
stat using structure spdk_bdev_io_stat.
The following is the comparison of bdevperf test.
Test on Arm server with the following basic configuration.
1 Null bdev: block size: 4K, num_blocks:16k
run bdevperf with io size=4k, qdepth=1/32/128, rw type=randwrite/mixed with 70% read/randread
Each time run 30 seconds, each item run for 16 times and get the average.
The result is as follows.
qdepth type IOPS(default) IOPS(this patch) diff
1 randwrite 7795157.27 7859909.78 0.83%
1 mix(70% r) 7418607.08 7404026.54 -0.20%
1 randread 8053560.83 8046315.44 -0.09%
32 randwrite 15409191.3 15327642.11 -0.53%
32 mix(70% r) 13760145.97 13714666.28 -0.33%
32 randread 16136922.98 16038855.39 -0.61%
128 randwrite 14815647.56 14944902.74 0.87%
128 mix(70% r) 13414858.59 13412317.46 -0.02%
128 randread 15508642.43 15521752.41 0.08%
Change-Id: I4eb5673f49d65d3ff9b930361d2f31ab0ccfa021
Signed-off-by: Richael Zhuang <richael.zhuang@arm.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14743
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Support to specify rr_min_io for multipath round-robin policy,
which makes I/O switches to another io path after rr_min_io I/Os are
rounted to current io path.
Change-Id: I09f0d8d24271c0178ff816fa63ce8576b6e8ae47
Signed-off-by: Richael Zhuang <richael.zhuang@arm.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15445
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Support selecting io path according to number of outstanding io of
each path in a channel. It's optional, and can be set by calling
RPC "bdev_nvme_set_multipath_policy -s queue_depth".
Change-Id: I82cdfbd69b3e105c973844c4f34dc98f0dca2faf
Signed-off-by: Richael Zhuang <richael.zhuang@arm.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14734
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Each fio job section will get unique CPU cores
assigned based on the number of numjobs value.
In case of high number of fio jobs or high numjobs
parameter value we might eventually run out of
CPU cores to assign. Allow regenerating list
of available CPU cores. This allows to run the
test, but as a result CPU cores might be shared
between fio threads.
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I69b1713e640d9343d68b26317f4caccdcec3cddd
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16240
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Allow user to specify the way IRQ alignment for
network interfaces used in test should be done.
Change-Id: Ib835d2ac2bc0c7b79474e617de32b96a483e436b
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15737
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Fio "numa_cpu_nodes" parameter (which tells fio
which NUMA node to use to pick cpus for the job)
seemed promising, but manually adjusting
"cpus_allowed" seems to have better results.
Change-Id: I113566ad0dd1d32fb5a1e989725b257e3ccff4c0
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14842
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This will enable NVMe mDNS discovery implementation
and testing in future patches.
Signed-off-by: Parameswaran Krishnamurthy <parameswaran.krishna@dell.com>
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ic8de8e463b07b5b9d260781175d117766acdab9e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16254
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Boris Glimcher <Boris.Glimcher@emc.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This application has not been maintained for at least
two years, let's just remove it rather than updating
it. Original intent was to monitor iSCSI I/O
processed on each core, if we decide it's important
to still provide this kind of analysis for iSCSI it
would be better to incorporate it into spdk_top
instead most likely.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I4093ab1357ccc06c8a6770f7f87238b510bb20d4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16219
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
The new module replaces functionality in vbdev_crypto.
This module is bdev agnostic, so some inernal parts
were reworked.
io_channel: contains a qp of every configured DPDK PMD
crypto key: for mlx5_pci we register a key on each available
device since keys are bound to Protection Domain.
Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: If1845cb87eadacbb921c593ba82207a97f2209a3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14859
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
This is done to make clear what data is included with the core dump.
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I67f7da7f342354df4688147eaf56283b33ba2863
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15996
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
As stated in the core(5) the RLIMIT_CORE is not enforced when kernel
is piping core to the collector, hence it will be written out even
when it's set to 0. Instead of playing nice and checking the validity
of RLIMIT_CORE, simply ignore it and apply our own limit (2GB).
This is done for convenience and to make sure that any changes to the
way how given distro applies limits (see fedora37) to the processes
doesn't affect our core collection.
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: Iab8757f548f2a4f0f4b27b4ce66b0a4a84ed2579
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15994
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Error counters for NVMe error was added in the generic bdev layer but
we want to know more detailed information for some use cases.
Add NVMe error counters per type and per code as module specific
statistics.
For status codes, the first idea was to have different named member
for each status code value. However, it was bad and too hard to test,
review, and maintain.
Instead, we have just two dimensional uint32_t arrays, and increment
one of these uint32_t values based on the status code type and status
code. Then, when dump the JSON, we use spdk_nvme_cpl_get_status_string()
and spdk_nvme_cpl_get_status_type_string().
This idea has one potential downside. This idea consumes 4 (types) *
256 (codes) * 4 (counter) = 4KB per NVMe bdev. We can make this smarter
if memory allocation is a problem. Hence we add an option
nvme_error_stat to enable this feature only if the user requests.
Additionally, the string returned by spdk_nvme_cpl_get_status_string()
or spdk_nvme_cpl_get_status_type_string() has uppercases, spaces, and
hyphens. These should not be included in JSON strings. Hence, convert
these via spdk_strcpy_replace().
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I07b07621e777bdf6556b95054abbbb65e5f9ea3e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15370
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
* added raid1 module functions
* raid1 logical volume can be created using standard SPDK RPC
* strip size parameter not supported by raid1
Signed-off-by: Krzysztof Smolinski <krzysztof.smolinski@intel.com>
Change-Id: Id3ee1ba0ec28540ca8eb67b04c3ff655a16b1f19
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13444
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Some Ceph builds, as seen under fedora37, have their default
osd_class_dir parameter set improperly to a relative path. As a
result, Ceph is not able to open and load .so libs that are part
of rados-classes. Example of an error seen in the ceph logs:
could not stat class lib64/rados-classes/libcls_rbd.so: (2) No such file or directory
To workaround it, always slap "/" at the beginning of the path.
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I08c2c94f7469748489ef4a9f0e65f3faccbecfb7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16064
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Kamil Godzwon <kamilx.godzwon@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
To achieve the highest possible IO-rates and lowest latency, then CPU
cycles must be spent processing IO. This commit introduces three
different techniques dependendent on the 'io_mechanism' used.
For 'libaio', xNVMe is instructued via 'opts.poll_io' to not wait for
completions, thus xnvme_queue_poke() returns immmediatly, the user can
then call xnvme_queue_poke() as frequently as desired. This requires
xNVMe v0.5.0. Earlier versions will ignore 'opts.poll_io' for 'libaio'.
For 'io_uring', xNVMe is instructed via 'opts.poll_io' to enable
IORING_SETUP_IOPOLL.
For 'io_uring_cmd', xNVMe is instructued via 'opts.poll_sq' to enable
IORING_SETUP_SQPOLL, this sets up a kernel-side thread reaping
completions from user-space on the ring. Note that 'io_uring_cmd'
requires liburing >= 2.2.
This commit enables the above by default. The above can be disabled by
setting 'conserve_cpu' to true on the bdev-setup.
Signed-off-by: Simon A. F. Lund <simon.lund@samsung.com>
Change-Id: Id54f1e59733ce9ae3b174ad4562904d868d4ef4f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14678
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Under fedora, install_liburing() would put .pc under
/usr/lib/pkgconfig, whereas the pkg-config, by default, looks under
/usr/lib64/pkgconfig.
For the tests, make sure both paths are included in the environment
in case any entity (like xnvme) attempts to use pkg-config during
build.
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: Iab3a4d1f69afc12c56c9fa6b7dd99daaca670e45
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15853
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
When gdb_macros.py is used with unit tests some of the globals it
expects are not present. This commit handles the relevant exceptions so
a missing symbol does not prevent the use of macros that are initialized
later.
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: Ic81f0dfa705839c9a03fb76e934684716b710390
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15999
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
A bunch of python files in rpc and scripts directories were missing two
blank lines after the opening comments. python3-pycodestyle-2.5.0-2.el8
was unhappy with this
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: Iee7f2bd4f0ddd96b2be89949a3aa324b2f9ea43a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16003
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
This is only relevant when rpc.py is running in server mode. Recent
changes in argparse under python3.11 force it to raise the
ArgumentError exception whenever subparser of the same name is being
added.
With that in mind, avoid extra call to spdk_rpc_plugin_initialize()
by keeping track of all plugins that were already successfully
initialized.
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I89124738b995ba774b6619f97726a5bd69fe1fd9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15807
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Kamil Godzwon <kamilx.godzwon@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Starting with DPDK 22.11, PCI API are no longer public.
In order to implement out-of-tree driver like
SPDK NVMe driver that is compatible with DPDK,
a copy of PCI API headers are required in SPDK.
check_dpdk_pci_api.sh script is intended simplify
the maintanance of the compatibility between
SPDK copies of the headers and multiple DPDK versions.
Please see the included README
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ief028c13564441560425761e7802c6cf07460876
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15857
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
This gives us a place to initialize iobuf pools, specify subsystem
dependencies, and execute RPCs to configure the sizes of the pools.
We allow users to configure the size of the pools either through the
options in spdk_bdev_opts or through the new RPC, iobuf_set_options.
The second option has higher priority, so it will override the options
set by the bdev layer.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I7c45ace04bc71c8343658f98248fc7babcf24e5d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15765
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This check was put in place as a workaround for some specific kernel
builds which were mainly shipped under centos8. Since we already
dropped centos8 in the CI and replaced it with rocky8 (where kernel
is actually stable) there's no value in maintaining that anymore.
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: Idd39b5bae888a991d7d628a19bc18acac6570f81
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15819
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
There may be some timing issue where attempt to rebind the ctrl from
vfio_pci driver back to nvme, right after issuing the
bdev_nvme_detach_controller() call, fails as the vfio_pci might have
not fully released the device yet.
To mitigate, simply kill the application (as it's not needed anymore
at at point) before starting the kernel_target test - this should give
enough time for the device to be properly released.
As a precaution, make setup.sh to retry the probe attempt in case it
fails.
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: Ifc4f4c18a90605154bf33b078575c8b41129f1f3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15767
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Kamil Godzwon <kamilx.godzwon@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Found with misspell-fixer.
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: If062df0189d92e4fb2da3f055fb981909780dc04
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15207
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Using new_id attribute is global in scope, meaning that depending
on the kernel's setup seen prior running setup.sh, single write
to it may re-bind ALL matching devices. This doesn't play well
with our PCI_{ALLOWED,BLOCKED} options as they can't be enforced
in such a case. Consider the following example:
> modprobe -r nvme # all nvme ctrls are detached from the kernel
> echo 0xdead 0xbeef >/sys/bus/pci/drivers/uio_pci_generic/new_id
# setup.sh-wise
> modprobe -r nvme
> PCI_BLOCKED=some:dead:beef.bdf setup.sh
# PCI_BLOCKED device still ends up bound to userspace driver.
After this single write, ALL matching devices will end up bound to
uio_pci_generic. To avoid this, we should override preferred driver
on per-bdf basis.
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: Ic4613e33321303b92b47ce3f4d7e1f29ecca3036
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13813
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Kamil Godzwon <kamilx.godzwon@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
This package is required to install grpcio:
./scripts/pkgdep.sh --developer-tools
...
Could not find <Python.h>. This could mean the following:
* You're on Ubuntu and haven't run `apt-get install python3-dev`.
* You're on RHEL/Fedora and haven't run `yum install python3-devel` or
`dnf install python3-devel` (make sure you also have redhat-rpm-config
installed)
* You're on Mac OS X and the usual Python framework was somehow corrupted
(check your environment variables or try re-installing?)
* You're on Windows and your Python installation was somehow corrupted
(check your environment variables or try re-installing?)
----------------------------------------
ERROR: Failed building wheel for grpcio
Failed to build grpcio
Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: I1a9d6154de9ab86ba8062b53ba56002e226daa94
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15751
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: <qun.wan@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Do this before the attempt to bind these devices is made. This is to
make sure script works on subsequent runs.
Scripts itself executes the qat_service which disables potential VFs
of the target PF. In case VFs were previously bound to one of the
uio drivers (via new_id) the next write to that attr will fail not
rebinding the devices again. To avoid that, remove the VFs' id first
and then attempt to bind them to the uio driver.
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I4818a2e3e663a4136084ce4bbb85ccfcfde1ae42
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15427
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
The warning appeared during Fedora 37 VM image testing
Signed-off-by: Kamil Godzwon <kamilx.godzwon@intel.com>
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I2ceff22d08614fec0c27b9931d71753f57cf99bd
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15794
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
This is in prep for adding a new compressDev accel_fw
module that will contain all of the DPDK compressDev specifics
on it, the vbdev will make calls to the accel_fw instead.
As the accel_fw has SW based compression, we want the configure
option to apply to building the vbdev module but not the accel_sw
software implementation or the upcoming compressdev module.
Renamed to "compress" as reduce is a term specific to the vbdev
implementation of the compression to be provided by the accel_fw
and thus the same reason why we leave the test flag called REDUCE
because it's controlling tests for the reduce library as well as
the vbdev module that is using reduce. The flag does not apply
to the SW implementation of compression.
This does not affect upcoming accel_fw compressdev module, that
will have its own configure option.
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: If8ed3e48e1e3dabcaad1cd161289e78122cd9d58
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15179
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Both max and min should be reset periodically. We can use the queue
depth sampling poller to reset these but the queue depth sampling poller
is optional. We extend the bdev_reset_iostat RPC to support mode to
reset all or only max/min fields.
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I9ce54892f6e808f6a82754b6930092f3a16d51ff
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15444
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Add a helper function bdev_reset_device_stat() to reset I/O statistics.
This funciton is used for the bdev_reset_iostat RPC.
We do not have any plan to use bdev_reset_device_stat() outside
lib/bdev. Hence, we do not add this as a public API.
Then, add a new RPC bdev_reset_iostat to reset I/O statistics of a
single bdev or all bdevs.
Resetting I/O statistics affects all consumers. Add a note to CHANGELOG
and doc/jsonrpc.md.
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I97af09107b5c3ad1f9c19bf3cbf027457c4fbae7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15350
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Add rpc method trace_get_info to show name of shared memory file,
list of the available trace point groups and mask of the available trace points for each group.
Fixes#2747
Signed-off-by: Xinrui Mao <xinrui.mao@intel.com>
Change-Id: I2098283bed454dc46644fd2ca1b9568ab2aea81b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15426
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Community-CI: Mellanox Build Bot
Make sure all attributes are initialized in both classes.
If something is not supported or works a bit differently -
move it to children's class method.
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I963ef1e9fe4288992a15f65671c1b85e8e7f7fd6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14882
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Michal Berger <michal.berger@intel.com>
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I4183549cc55b437a435fb024c4a9d234c456ccab
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15041
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Michal Berger <michal.berger@intel.com>
"num_cores" is described as optional in readme file.
For SPDK Initiator class it was required instead of
optional, and Kernel Initiator did not even take this
parameter into consideration.
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: Id6874ee8614aefdd9ee4a55dd200a6deab504b7a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14881
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Michal Berger <michal.berger@intel.com>
Find matching SAR and PM script output files, parse
them and put calculated average from accross all
workload iterations into final CSV with aggregated
results.
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I9d3c445941e54eefbe93dd47d152d2ede22c9560
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15467
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Michal Berger <michal.berger@intel.com>
Wrong key was used to assign value from json config.
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I3814b385824b435ae39cdf0a47cca42735873e86
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15757
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Kamil Godzwon <kamilx.godzwon@intel.com>
Reviewed-by: Jaroslaw Chachulski <jaroslawx.chachulski@intel.com>
Reviewed-by: Michal Berger <michal.berger@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This should make it easier later to group matching
result files when parsing results.
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I8d3c5641c8b71b6027e1f200ce53426a46e5234d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15466
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Michal Berger <michal.berger@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Run additional measurements (PM, SAR, PCM, BWM-NG, DPDK mem)
for each configured test run.
For example: until now if fio parameter "retry" was set to 3
fio would run the workload 3 times, but additional measurements
would only be done for the first run.
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I5a8aaa8eeb28f2a24f47a41650f9e2bd14a298dd
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15380
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Enable PM measurements by default.
Delay time and sample count variables are now taken
from fio ramp_time and run_time parameters, while
sample interval is hardcoded to 1 second.
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: Ibc08c690a8c8eea7591a5553b2478283c7574eec
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15520
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Group and move all restore_* functions to classes
stop method.
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I92cde5865a28343dbace702f54e4e06e4c2ba6cb
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14855
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Enable PCM cpu, power and memory measurements as
default and make them run during fio workload.
Change-Id: I4d42cf9df8385bfaa3b5e8fac59283ef8be0e98b
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15341
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Michal Berger <michal.berger@intel.com>
bwm-ng interface bandwidth monitoring is now enabled
by default. Samples are captured every second during
fio run_time duration.
Change-Id: I868f01d23e648e855161e6711236a68dcff8ff5a
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15340
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Michal Berger <michal.berger@intel.com>
Enable SAR CPU measurements by default.
Delay time and sample count variables are now taken
from fio ramp_time and run_time parameters, while
sample interval is hardcoded to 1 second.
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: Icf2866820250dd6bbc5619b2777f6fe139f2a588
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15339
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Michal Berger <michal.berger@intel.com>
Groupt variables used for enabling addtional
measurements together for better readibility.
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: If61de427294fea6ae9a2d5c53f74b6c9e5aa3e6a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15338
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Use "allowlist" and "blocklist" options in JSON
configuration file to specify which NVMe drives
are OK to use when setting up Target side. If no
list is specified the user can continue with the
test and use all available NVMe drives by passing
"-f" option when running the script.
Previously the default behaviour was to use all
NVMe drives available in the system for test, which
could cause data loss on those drives.
This is a failsafe primarily aimed at KernelTarget
class, as SPDKTarget is already safer to use because
of existing checks in "setup.sh" script which must
be run prior to executing NVMe-oF performance test
script.
Change-Id: I5ff93672a92cb09f2aef8355542ad197d96e14e1
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14854
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
"log_print" was removed in previous commit and
switched to Python's logging module instead.
This line was probably a result of a faulty rebase.
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I6bbb2c37a947d0169689a014b2e110884bb70fdc
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15519
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michal Berger <michal.berger@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I0d38256afb918f1273def36b3d4915d5a1911df9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15422
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
They were missed by the initial set of patches which introduced this
header as a mandatory one across different types of files.
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I3f9b37d41298c843e1648e72fe8593768ccd37e0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15423
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
per Intel policy to include file commit date using git cmd
below. The policy does not apply to non-Intel (C) notices.
git log --follow -C90% --format=%ad --date default <file> | tail -1
and then pull just the year from the result.
Intel copyrights were not added to files where Intel either had
no contribution ot the contribution lacked substance (ie license
header updates, formatting changes, etc)
For intel copyrights added, --follow and -C95% were used.
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I2ef86976095b88a9bf5b1003e59f3943cd6bbe4c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15209
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
CalledProcessError was removed from common.py and
not moved to run_nvmf.py in:
https://review.spdk.io/gerrit/c/spdk/spdk/+/14740
Change-Id: Ib35c60aa779711f40c6fda990c721491ae000b43
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15543
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Michal Berger <michal.berger@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Signed-off-by: Sebastian Brzezinka <sebastian.brzezinka@intel.com>
Change-Id: Ieb89027d41567f000f0f1a8c7375fde5201b06db
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15561
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Michal Berger <michal.berger@intel.com>
Signed-off-by: Sebastian Brzezinka <sebastian.brzezinka@intel.com>
Change-Id: I64473e4904c82fa06d41e8a02d56070ebf966adc
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15559
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Michal Berger <michal.berger@intel.com>
Under latest 13.1 builds ceph package is now available as "ceph14"
so try to install it when the first attempt fails.
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I4301f357d2c04881a1e8839c644cb000ba74e4e6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15495
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Kamil Godzwon <kamilx.godzwon@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Look for all declarations instead of only the ones introduced in the
tested patch. It makes this test pass if a patch adds definition for a
function that already has a declaration in the header file (e.g.
the implementation depends on some #ifdef).
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I735cf10d4524318009f8c4282a86e4137bf8c769
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15428
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Add an option "--generate-uuids" to bdev_nvme_set_options
RPC to enable generation of UUIDs for NVMes devices that
do not provide this value themselves. The identifier is
based on a serial number of the device, so a bdev
using this NVMe will always be assigned the same UUID.
Part of enhancement from #2516.
Change-Id: I86d76274e5702d14ace89d83d1e9129573f543e2
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15151
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
This uses RAPL's powercap interface under sysfs. The alternative is
to use MSRs directly, but with this we don't have to bother about
different cpu models, etc. as kernel does that for us here.
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I91ed5d67edf2669b9d7b271bbc02ecc61a6a3ea2
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15182
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Run collect-bmc-pm as a thread with execution time limited by count
and interval. This aligns it with the way how other tools - sar,
pcm - are used during the test.
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I48fa4ea1be69f9dcae71bf01c3214102ab329bac
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14628
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
These functions are used only by SPDKTarget class, so
move them out of common module.
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I489443f76fbdf4cb3def19a557f4a953a098bbce
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14853
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Michal Berger <michal.berger@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
This function is used only by this class, remove it
from common module.
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I6cb01e68f0005ddc1604ff4014c6b346da3a970e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14852
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Michal Berger <michal.berger@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Results parsing is not Target specific. Result
files get copied from initiator the system where
the script is run. In future we might want to
add a feature to work with remote Targets which
would make the parse method fail.
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I215391e0ff7a3ac0881f5a092513dd087fbd2e2c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14845
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Michal Berger <michal.berger@intel.com>
Remove unused imports and unused function.
Move nvmet_command function to Kernel Target
as it's specific only to this class.
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I30699ca1d8541ff2f57ea609e5caf0304feb4282
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14740
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Michal Berger <michal.berger@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Rename "kernel_init_connect/disconnect" methods
and create equivalents in SPDK Initiator class.
This allows to skip conditional checks in main
section of the script.
Change-Id: I23f620670d628c602dfdc1f24b2ddf493993ae12
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14739
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Michal Berger <michal.berger@intel.com>
Information about Target subsystems (port / nqn name /
ip address) is already saved at the time of their creation.
Avoid spawning nvme-cli processes and instead just filter
matching subsystems based on available information.
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I24d16315d67e5eaade48790b7fae45664110afcb
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14737
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Michal Berger <michal.berger@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Use tuple instead of list as this information is
supposed to be immutable later on.
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: Id9806202d06368f09ede0c0d903924b101d3b86c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14736
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Michal Berger <michal.berger@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Permissions on /etc/systctl.conf file might be
for owner-only (root). Use sudo to make sure we
can read the file.
Change-Id: I98d77ac87f196aac82000cb42553285f22f8cdcd
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14390
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Kamil Godzwon <kamilx.godzwon@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jaroslaw Chachulski <jaroslawx.chachulski@intel.com>
Reviewed-by: Michal Berger <michal.berger@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
bdevperf application is utilized in multiple test scenarios
and most prominently in SPDK performance reports.
Similar to perf and fio_plugins, it is used to measure performance.
It is intended to represent the expected behavior of users
application that use SPDK bdev layer.
Applications under --enable-tests are intended for specific test
scenarios and user interaction is usually via a test scripts.
Meanwhile --enable-examples are used more broadly to teach and
show usage of SPDK libraries.
As an added benefit with this patch, its possible to compile
bdevperf without need to satisfy additional requirements to
compile all the test applications.
Change-Id: I9aaec1f9d729a1cdee89b5fdc365d61c19b03f82
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14558
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Do this prior installing any pip package to make sure they are stable.
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I842252fe33b62dddd16858806a7d334cf2139fac
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15292
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Kamil Godzwon <kamilx.godzwon@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Adds "--rbd" option to install build dependencies for the rbd bdev.
This will be used by the Ceph NVMe-oF gateway.
Signed-off-by: Mykola Golub <mykola.golub@clyso.com>
Change-Id: I7926433aaa398873133dfffea3c401ce3c1f7813
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11134
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
per Intel policy to include file commit date using git cmd
below. The policy does not apply to non-Intel (C) notices.
git log --follow -C90% --format=%ad --date default <file> | tail -1
and then pull just the year from the result.
Intel copyrights were not added to files where Intel either had
no contribution ot the contribution lacked substance (ie license
header updates, formatting changes, etc)
Note that several files in this patch didn't end the license/(c)
block with a blank comment line so these were added as the vast
majority of files do have this last blank line. Simply there for
consistency.
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I6cd3f18d1b469d5ef249d26ddb2923ca6b970bd4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15208
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
per Intel policy to include file commit date using git cmd
below. The policy does not apply to non-Intel (C) notices.
git log --follow -C90% --format=%ad --date default <file> | tail -1
and then pull just the 4 digit year from the result.
Intel copyrights were not added to files where Intel either had
no contribution ot the contribution lacked substance (ie license
header updates, formatting changes, etc). Contribution date used
"--follow -C95%" to get the most accurate date.
Note that several files in this patch didn't end the license/(c)
block with a blank comment line so these were added as the vast
majority of files do have this last blank line. Simply there for
consistency.
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Id5b7ce4f658fe87132f14139ead58d6e285c04d4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15192
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
Allow CPU core locks to be enabled and disabled
during runtime. This feature will be useful
in cases like SPDK hot upgrade, where
locking should be disabled temporarily.
Change-Id: I9bc7292fd964abffc7214d074d191f38b13583c3
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15031
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
When running SPDK application on a given set of
CPU cores, create lock files for each of them.
This wil prevent user misconfiguration and
assigning a core to more than one SPDK instance.
The introduced mechanism is based on device locks
implemented in spdk_pci_device_claim() function.
Add a command line option to disable lock files.
This feature will be useful in cases where differing
CPU cores is impossible (eg. setup with only one core
available).
The patch also fixes all existing cases of overlapping
core masks.
Change-Id: Ie9aacb7523a3597b9aa20f2c3fa9efe4db92c44c
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14919
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
The structs io_device and spdk_io_channel were moved from using tailq list to rb tree,
moving the parsing in gdb functions to use those too.
Fixes issue #2763
Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Change-Id: I8fa5a69b5004e3d86fdfa201c2ba47e06fa47cfa
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15330
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michael Haeuptle <michaelhaeuptle@gmail.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
There was a code that attempted to to use pip3 before we try to
actually install it. Change the order.
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I5dc74b79b3adf132589bbf9ef10360d2eeed51ba
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15293
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
The actual target files come with a .* suffix.
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I1c6056c81261ae97bd272d0d99ad915ab4bb870e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15294
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Support injecting data corruption for processing read/write I/O.
Add two parameters, corrupt_offset and corrupt_value, and an error
type corrupt_data.
For write I/O, inject data corruption before submitting write I/O
to the underlying bdev. For read I/O, inject data corruption after
read I/O successfully returned.
Cause the data corruption by xoring the value at the corrupt_offset
with the corrupt_value. corrupt_value is required to be non-zero to
cause data corruption necessarily.
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I67d8d252b06d7a221413e74996f7f894e6519556
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15028
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Community-CI: Mellanox Build Bot
