This is part of a larger series enabling failover at the bdev
layer for NVMe.
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: Ic5c128244699c1a47275145ca7e41aa5f1366259
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3044
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
This patch paves the way for introducing a tailq containing
multiple alternative paths to the same controller.
Change-Id: I13d30c12b8e0ce38eae687f9e76740be1d11e4d1
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3259
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
RDMA target can't handle bidirectional xfer type, in debug build
it throws an assert in nvmf_rdma_setup_wr function. NVMF controller
performs checks od opcodes, but the failure happens before this
check. Add similar validation in TCP transport.
Change-Id: I14400b9c301295c0ae1d35a4330189d38aeee723
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3436
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Update reference build path to latest build.
Change path to use a generic directory name instead
of using specific SPDK version in the name. That way
in future we can update the build just after the
release, and still make ABI checks for patches which
were not rebased.
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I32850a871274773e334057b60b9df238ef85a999
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3532
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
This becomes a problem when the qpair is reconnected.
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I6677b396cf766684a4891ffbee93aa3e4e83374d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3391
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Some of the tests are skipped when using Soft-RoCE.
Test executed as part of run_test should never
skip a test, as it results in false assumption
that it was executed. See supplemental logs in CI output.
Before this patch bdevperf and srq_overwhelm (if executed),
resulted in misreporting this.
This patch moves all check for Soft-RoCE to top-level
nvmf test script.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I6d8941bbadaca2dbf8ab178e30e87d3786fea94c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3597
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Using Soft-RoCE is only relevant when testing
RDMA transport. For all other cases, we might
treat the IP address as non-rxe.
This simplifies the checks a little bit in couple places.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I996afb364070fbe107bbe97207952f99e4089308
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3596
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Soft-RoCE sometimes returns non-spec compliant values
in the WC entries after the qpair has been disconnected.
This can result in segmentation faults. In order to avoid
this, don't test shutdown on Soft-RoCE.
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I604ffaf38aa8da5efa349df0b1b51e7e391d17f8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3582
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Fixes#1498
When shutting down the application, it was possible to
reference stale ocf_cache pointer. This was the case
when two or more vbdev_ocf devices were based on top
of single cache bdev.
This issue did not occur outside of the shutdown case,
since RPC only allows deletion of the vbdev_ocf.
This erases on disk metadata and next run of the application,
would not detect such vbdev_ocf.
Shutdown meanwhile works different, by first stopping
the instance of running "ocf_mngt_cache" and later detaching
"core" devices (the ones being cached). This prevented
erasing the on disk metadata and allowed for restarted
application to detect vbdev_ocf.
See patch (1292ef2) for details.
Since references to ocf_cache are copied between vbdev_ocf
[see start_cache()], the reference count inside ocf_cache
was limited to original ocf_mngt_cache_start() and
management queue creation. First call into ocf_mngt_cache_stop()
released all references to ocf_cache. Leaving other
vbdev_ocfs pointing to released memory.
This patch works around this issue by increasing ref cnt
on ocf_cache for each vbdev based on top of it.
It allows to call into ocf_mngt_cache_stop(), but not
release the memory for ocf_cache until last vbdev.
Note:
A proper redesign here is in order:
- either rearranging structures to be based around single ocf_cache,
rather than multiple vbdev_ocf instances
- better use of OCF API to reduce book keeping logic in vbdev
There are plans to implement detach/attach in RPC,
so it should be a focus during that effort.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I560a7fbb1c052bf53970e655bdb60803c561a252
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3574
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Some images (especially cloud/containers) may do not
contain this package in default. So we need to add this package
install support.
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I2620026534009e8b6c320ecf61b8970947d5a9fb
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3218
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
To specify the desired logical block size. Must be 4K or 512.
If no block size is provided a default of 0 means to use the
underlying bdev block size. For cases where something other
than 4K or 512 is desired, format the underlying device
accordingly and don't specify a logical block size on creation
of the compress vol.
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I58b71e210cfa77b3237c0c454585c734e2e22aea
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3177
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Makefile support in DPDK was deprecated and will be removed soon,
so switch to the officially supported way of building DPDK -
with meson and ninja. Two new tools. Basically, our Makefiles
will invoke meson+ninja for DPDK, no other SPDK components are
affected.
Apparently DPDK wanted to move away from an octopus-like config
system and the ideology behind meson configuration is simple now:
build everything by default. Some PMDs can be explicitly disabled
with meson command line, but all libraries (both static and shared
versions) and test apps are built unconditionally.
How long does it take to build minimal DPDK with meson? Too much.
On my machine half of the total build time is spent on libraries
we don't need at all. (I have some hacks up my sleeve to disable
building those libraries - see the subsequent patch.) As for the
official way of building a minimal DPDK, there was a patch [1]
on dpdk mailing list to introduce more specific configuration,
but it was rejected:
> We talked about this a few times in the past, and it was actually one
> of the design goals to _avoid_ replicating the octopus-like config
> system of the makefiles. That's because it makes the test matrix
> insanely complicated, not to mention the harm to user friendliness,
> among other things.
>
> If someone doesn't want to use a PMD, they can just avoid installing it
> - it's simple enough.
>
> Sorry, but from me it's a very strong NACK.
Let's not follow that direction, hack the DPDK build system instead.
As for advantages of meson+ninja over Makefiles? I can't find any.
It's another build system that does a lot for you with some functions,
magic options, and a built-in dependency system. It seems nice if you know
the syntax, but it's another component that you need to learn, debug,
and possibly find bugs in (there's a lot of github issues open for meson).
I would compare it to CMake.
As for changes in this patch: rather that explicitly disabling
PMDs we don't need, specify a list of PMDs we do need and disable
everything else found in ./dpdk/drivers/*. This way we won't have
to disable the new PMDs as they're added to DPDK.
Meson configuration also sets RTE_EAL_PMD_PATH #define to a valid directory
with built PMD shared libs. When it's set, DPDK dynamically loads all shared
libraries inside. The drivers there depend on DPDK shared libs and fail to
load in static SPDK builds, so we disable them altogether by unsetting
RTE_EAL_PMD_PATH in the meson-generated config file - just like
DPDK Makefiles did. EAL checks for RTE_EAL_PMD_PATH being empty and skips
loading any external PMDs then. We do it for both static and shared libs.
We specify all PMDs at build time for now, so there's just no need to load
them dynamically.
We have three more hacks in our submodule:
* disable building dpdk apps by commenting-out a line in dpdk/meson.build
* disable building unnecessary libs (build everything that spdk *may*
need)
* build isa-l compress pmd with `-L[...] -lisal`. DPDK expects to find
libisal with pkg-config. We don't want to prepare a pkg-config file,
so comment-out a failing check in another meson.build file and provide
isa-l through CFLAGS and LDFLAGS.
We also need to make some changes to our test/external_code. First of
all, -ldpdk is no more. Meson build generates a pkg-config file with all
libs, but we'll switch to it in a separate patch - for now just specify
all -lrte_ libs one by one. -Wl,--no-as-needed has to be added to some
test cases, otherwise rte_mempool_ring isn't loaded. We don't use any
APIs from this library, it only has a static constructor that provides
a few callbacks used by rte_mempool_create(). Also, since DPDK now builds
both static and shared libraries, we need to add -Wl,-Bstatic to force
using static libswhere required. It's only needed for DPDK libs, but we
use it for SPDK libs as well since there's no harm.
As for performance:
$ ./configure --enable-debug --with-crypto --with-reduce
$ time make -j40 -C dpdkbuild all
with meson:
real 0m8.287s
user 1m7.983s
sys 0m10.548s
before, with the old DPDK makefiles:
real 0m20.232s
user 0m55.921s
sys 0m16.491s
The subsequent builds are much faster too:
$ time make -j40 -C dpdkbuild all
meson:
real 0m0.876s
user 0m0.663s
sys 0m0.217s
makefiles:
real 0m10.150s
user 0m11.740s
sys 0m6.772s
[1] http://inbox.dpdk.org/dev/1a07d1cd59d84dce84e56c10fdabf5e5504560a6.camel@debian.org/
Change-Id: Ic65db563014100bafb12e61ee0530cc2ae64401d
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1440
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
With update to DPDK 20.05 EAL requires rte_telemetry,
external make tests were missing specifying this lib.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I9f5eaea46b47da48fc81b4fff0199cd75ff6af78
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3483
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
related #1498
Updating DPDK 20.05 exposes issue listed in #1498.
Most likely the device removal path has to be re-worked
to accomodate multiple devices on the same cache instance.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I28f71ef2c2e1d9c2d19202cd33a932182bfb122c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3521
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Now that drivers can be registered from upper layers there's
no need to keep them centralized inside env.
(check_format.sh complains that spdk_pci_nvme_get_driver() shouldn't
start with the spdk_ prefix - to workaround that we move the function
declaration from one place in env.h to another - that's enough to
convince check_format it really is a public function)
Change-Id: If86aebd6c997349569c71430ec815b413eb44ef8
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3187
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This will allow to override job config filename from CLI.
Note that -T is required when using job config that has
missing filenames, but then it is optional if job config is not used.
Change-Id: I7d392ff2d0554524cfcb5072182ec806013083de
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3452
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Some global argument and some code from verify_test_params()
could be removed.
Change-Id: I923f5f23fafd31ff48049bd16476f01c801bfbb8
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3278
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Instead of calling bdevperf_construct_job() directly,
create a job_config instance that would express
the same semantics and then use
bdevperf_construct_config_jobs() to create and run actual jobs.
The goal is to unify methods and reduce code duplication.
Change-Id: I724e5788ea74868c9e9c8128d658e720d211f253
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3510
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This parameters allow for low cost refactoring
because with them we can simplify construct_multithread_jobs()
by creating instances of job_config during its invocation
and then handling those in construct_config_jobs().
Change-Id: I73057a9a65f58e48ac719f30e918d7a05ffc0f28
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3509
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
Implement parsing of config file,
including overall format of sections/parameters (using lib/conf)
as well as filename, bs, iodepth and cpumask parameters (based on FIO).
Also implement creating jobs based on config file.
The only required parameter that is missing is 'rw'.
It will be added in a separatate patch because it requires quite
different code than for other parameters.
Duplications that this patch introduces with respect to legacy
code are addressed in following patches.
Change-Id: Ie19e55dc66369e795a97b53a4223beafa10d8fe8
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3189
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
Accept job thread as argument
in bdevperf_construct_job().
This will allow for using single thread for
multiple jobs, as it is in FIO if multiple filenames are specified.
Change-Id: I0b6aa69aae1e1bf099170d3d495e0a83f97a857b
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3365
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
env_dpdk_get_mem_stats RPC has been requested to be
available in bdevperf.
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I10aea634851e21018b223d7331b1bbeb7630a09c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3520
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Parse fio output to CSV.
Calculate average read and write metrics while
parsing and present final results.
Change-Id: I8f6d7f84205be409f0ccbba7a42eb7888b2cb03d
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3253
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
There is a lot of extra lines at top of the output file
in case fio is used in client-server mode and with json
output mode. These lines make it resulting JSON file
not possible to parse.
Remove this lines so that the JSON file is ready to
be used in any later parsing functions.
Change-Id: Ie575941310f0d17b0285969f6b247f2fa653168c
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3237
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
app_repeat is used to test calling spdk_app_start/stop
repeatedly.
* "-t <num>" is used to specify the maximum number of
repeated rounds.
*SIGUSR1 can be used to trigger a new round of
spdk_app_stop/start.
*Shutdown signals like SIGTERM can be used to shutdown
the application.
Change-Id: I10a205ea2e9da3b4ecd6381bf25b457ce909a621
Signed-off-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2447
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
If intel_pstate is disabled in boot options, allow to
change CPU frequency using "userspace" cpu governor.
Change-Id: I390492c7be28df408f3cbb774ba5360859649fe0
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3376
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
For some reason the routine for generating fio
job/filename sections was different for kernel and
spdk fio modes. This made fio results hard to compare
because even configuration files themselves were not
comparable.
The function was modified to create the same fio
configuration files for kernel and spdk.
While at it, iodepth calculation has been slightly
modified as well. It is now being calculated for each
[ ] job section, so that each "filename" gets the
actual iodepth value specified in test run parameters.
While generating the config file the script will add
NUMA node information as a comment for "cpus_allowed"
and "filename" sections, so that it's clearly visible
how resources are aligned. In case of cross numa
configuration additional warnings will be printed to
screen as well.
For example, a configuration file for running test with
7 NVMe drives, 3 CPUs and effective (per filename) queue
depth of 32 would look like this:
run_perf.sh --cpu-allowed=0,1,2 --iodepth=32
--driver=kernel-classic-polling [...other params...]
--------
[filename0]
iodepth=96
cpus_allowed=0 #CPU NUMA Node X
filename=/dev/nvme0n1 #NVMe NUMA Node X
filename=/dev/nvme1n1 #NVMe NUMA Node X
filename=/dev/nvme2n1 #NVMe NUMA Node X
[filename1]
iodepth=64
cpus_allowed=1 #CPU NUMA Node X
filename=/dev/nvme3n1 #NVMe NUMA Node X
filename=/dev/nvme4n1 #NVMe NUMA Node X
[filename2]
iodepth=64
cpus_allowed=2 #CPU NUMA Node X
filename=/dev/nvme5n1 #NVMe NUMA Node X
filename=/dev/nvme6n1 #NVMe NUMA Node X
--------
Change-Id: Ida2f781fbb93c4a8c62154e711151152843ab997
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3370
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Configuration file generation was split into multiple
places/functions in scripts. Move it into a single
function. "time_based" fio parameter has been moved to
fio config template file as it is not configurable.
There is still a lot of room for improvement here, because
create_fio_config() function uses mixed local variables
(as function arguments) and global variables.
Change-Id: Ica3717d0d28692d24c861c2d32e729761c5c8647
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3345
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
In case spdk_dd was compiled with liburing support, rerun the tests
with forced AIO.
Change-Id: I26bee51b2671c19189a2678f48aef4809b7b1621
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3479
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
The new abort functionality doesn't take custom admin cmd
handlers into account.
This commit allows setting a custom admin cmd handler
for abort that provides the ability to influence the
bdev lookup to which the abort is sent to.
Signed-off-by: Michael Haeuptle <michael.haeuptle@hpe.com>
Change-Id: I3a66c6f863f5ee4d89cb2194dffdc6855945fa8a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3485
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
struct spdk_nvmf_request holds req_to_abort and so passing req_to_abort
separately is not really necessary now. The internal API
nvmf_ctrlr_abort_request() was added at the stage of prototyping.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I9ef2467d6f92422f044650c62a0777b95c0fc1ab
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3488
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Michael Haeuptle <michaelhaeuptle@gmail.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Now that we support only DPDK 18.11+ and always have
to register pci drivers to DPDK on initialization we
don't need that flag - it's always true.
Change-Id: Ibf1d79155595609fe9093f58e056bea25db6fdb2
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3446
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This follows struct rte_pci_id which had class_id as well.
We'll need it to make some additional DPDK APIs public through
the env abstraction.
Change-Id: I794a6cd6b17e48daf53b48fa5abe3d3dcfeaa403
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3182
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
You don't get notified when someone starts using your hooked
device, so there's not much gain from knowing when someone
stops.
Remove that callback and also move DPDK device detach under
the same lock which sets the pending_removal flag. This eliminates
a data race window when hotremove notification could arrive
after device was detached, but before it was scheduled to be
removed.
vmd and ioat nest the spdk_pci_device struct and abigail complains
even though the parent structs only have forward declarations in
public headers. Adding those two structs to the suppression list
doesn't help though. Abidiff still complains about the pci device
struct being changed, probably because ioat.h and vmd.h both include
env.h. Abidiff suppresion list should eventually be split per-lib,
but for now ignore struct spdk_pci_device changes globally.
$ abidiff [...]/libspdk_ioat.so [...]
'struct spdk_pci_device at env.h:652:1' changed:
type size changed from 1024 to 960 (in bits)
1 data member deletion:
<SNIP>
Change-Id: I9b113572c661f0e0786b6d625e16dc07fe77e778
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2939
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
A workaround for kernel deadlocks surfaced in #1275.
DPDK basically offers two APIs for hotplugging all PCI devices:
rte_bus_scan() and rte_bus_probe(). Scan iterates through
/sys/bus/pci/devices/* and creates corresponding rte_pci_device-s,
then rte_bus_probe() tries to initialize each device with the
supporting driver.
Previously we did scan and probe together, one after another, now
we'll have an intermediate step. After scanning the bus, we'll
iterate through all rte_pci_device-s and temporarily blacklist any
newly detected devices. We'll use devargs->data field to a store
a timeout value (integer) after which the device can be un-blacklisted
and initialized. devargs->data is documented in DPDK as "Device
string storage" and it's a char*, but it's not referenced anywhere
in DPDK. rte_bus_probe() respects the blacklist and doesn't do
absolutely anything with blacklisted ones.
The timeout value is 2 seconds, which should be plenty enough
for an NVMe device to reset, leave the critical lock sections in
kernel, and let us initialize it safely.
Note that direct attach by BDF doesn't respect the blacklist,
so an NVMe attach RPC won't be delayed in any way, it will continue
to work as it always did. Only the automatic discovery & enumeration
is deferred.
Change-Id: I62b719271bd0755bc2882331ea33f69897b1e5e5
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1733
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
There's no reason to keep usage and getopts
outside of the main script.
Other small changes while at it:
- use small letters for all "dir" variables
(consistent with most of other test scripts)
- try to move default & global variables into
one place.
- remove long comment from top of the main script.
It was basically the same as "help" for getopts.
Change-Id: Ic99f4bb1808b55dea80b534f7b4e0a65595ed053
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3335
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Read CPU list to use in test directly from option
value of from a file if it's an existing path.
This will make running tests in CI a bit easier.
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I174a38ab615c2f2cd2c3b4a95e0ebf886c478a51
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3333
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Allow to set custom size of bdev IO cache and pool
sizes.
Change-Id: I574d1e949f705d8f834539d5879f0a1e72865e00
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3297
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Something changed about the way we support
AERs in the target. Now, the AER commands remain
queued in the target side causing the fuzzer to
think it is timed out.
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I8e892a11d9922e84cd7a09e6adc8b69b6db4e8d2
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3453
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Community-CI: Mellanox Build Bot
Use a simple config file to select NVMes for test
instead of selecting them all via gen_nvme.sh.
HW platforms may differ significantly in setup and
capabilities, so using all disks in sequential order
is not always a good way.
(e.g. using "first" four available NVMes for test
may yield not optimal results if it turns out that
NVMes are connected to oversubscibed PCIe switch)
Change-Id: I55dd85799a5859c4764e94e1d8058e01ad9b84f0
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3296
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Script was written in such a way that tests would be
performed starting with all found NVMe disks in configuration.
Then the number of disks for the workload would be reduced in
loop until reaching 1. This is the scenario used in test
case 1, described in SPDK NVMe Performance Benchmark document.
--disk-no allowed to avoid running test in loop, and specify
just a single value for the number of disks to be tested.
Both loop logic and --disk-no are no longer required as
tests are scheduled and run via CI system which picks the
parameters for each test run.
Change-Id: Iad0f4d0d259ac64680fd5bb69d7cbb7e0afff79c
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3295
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
- Change divisor in nsec to usec calculation to "1000".
- Ignore MIX param when rw mode is set to 100% read or
100% write.
- Use MIX param as number in 0.0-1.0 range for calculating
summary read+write latency.
- Use jq "//" alternative operator when querying for fields
which might not exist in JSON object. Assume "0" for
non-existent fields, otherwise jq reports "null", which
in turn results in script errors.
Change-Id: Ibc8f91f058d9ae31ee1b60f4b253c42b743ae22f
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3294
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Previous changes moved some of the fio CLI parameters into
a bash heredoc to save into a text file, but it was put in
wrong place in the script.
Move global fio parameters a couple lines up so that the
resulting fio config file has the correct form of:
[global]
global_params
[testjob]
testjob_params
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I3a958fe5ce880ab63a079c345e6b24fd1d3bae32
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3293
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
There is no reason to continue processing these requests if the
qpair is not still active. We should complete them and free
any resources they are still holding.
Also, not doing so can cause issues with trying to access pointers
in the qpair after they are invalid. See issue #1460.
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I6e570a576983dfedf726dc4a9a83316209403e00
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3451
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
spdk/nvmecli git index happens to be corrupted and
results in a lot of unrelated failures in the CI.
This patch attempts only to mitigate the issue of
unwanted failures by restoring nvmecli repository
to clean state.
Fixes#1496
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: Ic17bffb0bc5bd3222c9f395fb217d2ffae99ceb3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3415
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Seth Howell <seth.howell@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Add a new sample application to test I/O abort for NVMe backend,
both PCIe and over Fabrics. And enable it to test pool by NVMe-oF
+ delay bdev + malloc bdev.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I5aa28b523bdb2c1d4d8c7af05f925c4dd01dda86
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2661
Reviewed-by: Michael Haeuptle <michaelhaeuptle@gmail.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Bdev underneath lvs has a certain value set as block size,
then lvolstore inherits that as `io_unit_size`,
which ends up as block size for the lvol bdev presents.
Before this patch lvs on device with block size of 512,
would create lvol bdev with block size of 512.
Meanwhile reporting in bdev_lvol_get_lvstores(),
that block_size is 4k instead.
This was result of addition of io_unit_size and
forgetting to update the value here. Previously
only I/O of page_size was allowed.
This patch adjusts the value reported to
io_unit_size.
Alternatively it could have been removed,
or name changed to "io_unit_size".
Yet for API compatibility reason this name
remains.
Info on actual page size which is always 4k,
is not useful to bdev users. No need to add
another field for that.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I19701c408c389f64e25c027c7bba789294bbed94
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3289
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Due to the firmware response time for REVERT action, currently
we only enable this test with P4510 drives.
Change-Id: I4b02008ec0b7d2caeba20c566b1cf1b69404ae12
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2816
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Call nvmf_ctrlr_abort_request() if the request whose CID matches
is found and its state is executing.
nvmf_rdma_qpair_abort_request() returns immediately if rc is
SPDK_NVMF_REQUEST_EXEC_STATUS_ASYNCHRONOUS, or calls
spdk_nvmf_request_complete() otherwise.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I1462a21db7270f3d63f8f293ad4be61d52e74da1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3011
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Michael Haeuptle <michaelhaeuptle@gmail.com>
Call nvmf_ctrlr_abort_request() if the request whose CID matches
is found and its state is executing.
nvmf_tcp_qpair_abort_request() returns immediately if rc is
SPDK_NVMF_REQUEST_EXEC_STATUS_ASYNCHRONOUS or calls
spdk_nvmf_request_complete() otherwise.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I1abceecc211ee79d8ac18a82dc63b13d313a6f27
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3008
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Michael Haeuptle <michaelhaeuptle@gmail.com>
State machine is different among NVMe-oF transports and is
encapsulated to the transport neutral NVMe-oF controller and
NVMe-oF qpair.
To implement abort operation for each NVMe-oF transport,
add a function pointer qpair_abort_request to struct spdk_nvmf_transport_ops
and a stub nvmf_transport_qpair_abort_request() to encapsulate
which transport is used.
The following patches will implement qpair_abort_request for each
transport. Each qpair_abort_request() is responsible to call
spdk_nvmf_request_complete() for the abort request.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I2beac959ed428c5108cf33691226b7fae5cd24d6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3007
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michael Haeuptle <michaelhaeuptle@gmail.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
We have no particular requirement to keep both conn->outstanding_r2t_tasks
array and conn->active_r2t_tasks list now.
To improve readability and maintaineability, unify two into the latter,
conn->outstanding_r2t_tasks list. Update unit test accordingly.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I25cf7cffbe39ac66e102eb3052340de6ef65c8f1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3115
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Previously iscsi_del_transfer_task() dequeued the task only from
the array conn->outstanding_r2t_tasks[].
process_non_read_task_completion() had dequeued the task from
the tailq conn->active_r2t_tasks then.
However abort_transfer_task_in_task_mgmt_resp had not dequeued the
task from the tailq conn->active_r2t_tasks then.
This was an apparent bug, and is fixed here. Update unit tests
accordingly.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I93f02b2fb670dcee4c32d61c264e3ad5b4f9f43e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3108
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Previously free connections had been managed by g_conns_array,
and allocate_conn() gets the lowest free connection. This had worked
almost as LIFO, and the just freed connection had been reused
immediately to the new connection.
Using TAILQ makes management of free connections FIFO, and this will
be more intuitive and simpler, and avoid potential issues due to the
fact that we do not know the state INVALID is the current connection
or the current connection is exited and the new connection is allocated.
This patch includes following updates.
Remove the test condition that the connection ID should be zero.
Connection ID is used as Target Transfer Tag (TTT) and TTT is opaque
number. Hence requiring connection ID to be zero is not meaningful.
iscsi_conn_free() calls free_conn() while holding g_conns_mutex, but
iscsi_conn_construct() does not call free_conn() without holding
g_conns_mutex. Hence add g_conns_mutex to the latter.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I204f66469f0bf54845c773da5b4ac86f3c8dca60
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3089
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
VPP is going to be deprecated in 20.07, thus remove support for it
from vm_setup.sh.
Change-Id: Ic5891178bc439ff08475adbb2925b81c4a33525f
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3199
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Using hardcoded path of /tmp/spdk was a bit problematic due to the
way how CI mixes root permissions with non-root's during test
execution. As an example, see:
https://review.spdk.io/gerrit/c/spdk/spdk/+/2560
Instead of using /tmp/spdk as a fallback, always try to create tmp
directory by generating its path via mktemp(). This way, as with
"autobuild: Use mktemp() for spdk workspace (ae730042)", different
permissions shouldn't be an issue anymore.
Lingering tmp directories will be purge at each set_test_storage()
run.
Change-Id: I93bc60c97b68e66606e98fc83463074fc020e372
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3238
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
This patch adds waitforblk() after rebinding the driver.
If timing was unfortunate, it was possible to issue
id-ctrlr command to a device that was not yet ready.
Meaning the support for fw command was not read properly.
No error was reported due to "set +e", so this patch moves
it till after id-ctrlr.
Example of the error:
# get_nvme_ctrlr_from_bdf 0000:82:00.0
# readlink -f /sys/class/nvme/nvme9
# grep 0000:82:00.0/nvme/nvme
# bdf_sysfs_path=/sys/devices/pci0000:80/0000:80:03.0/0000:82:00.0/nvme/nvme9
# [[ -z /sys/devices/pci0000:80/0000:80:03.0/0000:82:00.0/nvme/nvme9 ]]
# basename /sys/devices/pci0000:80/0000:80:03.0/0000:82:00.0/nvme/nvme9
# printf '%s\n' nvme9
# nvme_name=nvme9
# [[ -z nvme9 ]]
# set +e
# ctrlr=/dev/nvme9
# ns=/dev/nvme9n1
# /usr/local/src/nvme-cli/nvme id-ctrl /dev/nvme9
# grep oacs
# cut -d: -f2
/dev/nvme9: Resource temporarily unavailable
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I5728a5062cd553eb39d18d9869fdf56319a27777
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2950
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Accidentally, checkout command was included in said path, remove it.
Change-Id: I3d497b90552ddcc14be4a45b8794f79e185cfb9f
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3196
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Poller should return status > 0 when it did some work
(CPU was used for some time) marking its call as busy
CPU time.
Active pollers should return BUSY status only if they
did any meangful work besides checking some conditions
(e.g. processing requests, do some complicated operations).
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: Id4636a0997489b129cecfe785592cc97b50992ba
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2164
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
$rootdir is used already in this script, but couple instances
were left out.
We should always point to $rootdir or $testdir when refering to
scripts and test files.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ifeb4527c6fd42881a6e04c1406a765f57498ae7d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2937
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Usual way to call get_nvme_bdfs() is to iterate over resulting bdfs.
When no bdfs are returned, most of the scripts do no not execute tests
at all and assume success.
Instead of adding error checking for each and every script for no bdfs,
this patch makes get_nvme_bdfs() exit with test failure.
This assumes that there will never be a test that expects no devices
to be present.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I1e41a885c76ce6cc248fb076bc4f4efd4695ff3c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2836
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Only spdk/nvme-cli fork with spdk-1.6 branch is supported.
Even then this will be deprecated at some point in the future,
in favor of nvme-cuse.
This came up when fixing the tests to actually run.
If unsupported nvme-cli is used, compilation succeeds and tests
fail at the functional test part with non-obvious message :
# ./nvme id-ctrl 0000:84:00.0
0000:84:00.0: No such file or directory
Some of the CI machines need to be updated, but this
check will make it easier to gauge what went wrong.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ia19121a6c49b9618b8fea9e13304e27089f13dcf
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2948
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
nvme-cli tests with NVMe-oF could be silently skipped,
based on presence of the repo on system.
This patch makes it a requirement for tests to run,
and with that builds nvme-cli.
fixes#1469
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I581df5a00988b775e54110f718b95beb88487145
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2970
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
NVMe tests compiled the nvme-cli, but turns out that
NVMe-oF did not. This patch moves compilation to
common script. Meanwhile next one will use it in other tests.
Additionally return to the original directories with
pushd & popd.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ica60775a9b8e4347bed28d4c9dfae1acab2cae9f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2969
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tests should always have a reference to $rootdir absolute path.
Before this change, result of this function depended on current
working directory.
Test like test/nvme/spdk_nvme_cli.sh where directory was changed,
would fail.
With this change the path to get_nvme.sh script should always be
correct.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Id555188f032e179eedd406463446a85e13234b6d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2936
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
There are a lot of files in the nvme-cli repo on system that
either build up (*.core) or produce misleading logs (*.gcda).
This patch cleans the nvme-cli repository before compiling.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I8f169b9eaef2dc91b14b324a63b338611f82a6a5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2949
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Tests do not work on OS other than Linux, instead of
going through whole script - exit early with error.
Took this chance to move staring the stub, till after
nvme-cli compilation succeeds.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I5659888527bc2c25ab10bd871f7820b13f02e3fc
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2947
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
This patch is used to enable uring test in some VMs or machines
in the CI pool.
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I75c2ad477f5f648289d8dbb344b75b2408d56a38
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3107
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
headers can be installed in different directories but `/usr/include`
Signed-off-by: Jörg Thalheim <joerg@thalheim.io>
Change-Id: Ibc4641813f6f1c904560dfe7d05cb9b40109cea7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3134
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
The length of a keyed SGL data block is limited by 3 bytes.
Add a check to fail requests which length exceeds 3 bytes.
In other case we can send an incorrectly formed SGL request with
an invalid or zero length.
Fixes issue #1450
Change-Id: I77cdaff5fbf4be5754a3ac6008b8ccd532ac5905
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3056
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
It is possible to have yum on Fedora (either manual install
or via upgrade), yet dnf is the default pkg manager there.
Re-order the list of checks to pick dnf first if available.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I48dfda2a248a7798c9d0c8b3cb4b8a3aa615e2a7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3057
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Exit early from script execution when unsupported pkg manager is found.
Just not before the "--help" has a chance to be called.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I176bf5778dd7b2bc510f54842202113c8614a9d4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3054
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
When trying to find out parameters through "--help",
the $PACKAGEMNG is "undefined".
So the output from help does not make sense.
Change it so that the help message uses $package_manager that
was detected.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ia7ac47dec810588dfa01441b35204a7bf13280ed
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3049
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Removing verify_add_nvme_bdev_rpc_methods test from
rpc_config.py.
bdev_nvme_attach_controller RPC call is frequently used in
a lot of other low-level test cases. Also this test case
fails when run on a VM with multiple NVMe disks containing
0-namespace and OCSSD emualted NVMes, which is most
probably a result of a very generic regex used to obtain
NVMe BDFs.
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I091cc7fa4b426784136cda7423f829dc4a82eae0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3111
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
The 65b8c30d meant to use exec as a mean to "reload" the running
shell, however, exec simply replaces running bin image with a new
one, in this case, another shell. Since this call doesn't return
(unless it fails) this would suddenly leave the caller of the
vm_setup.sh inside the interactive shell.
Instead of re-executing the shell, simply export new PATH pointing
at where new git binary should reside.
Change-Id: I9ab5ba7f8f88602160c8be0f017477e0d1a01c08
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2790
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
This mainly concerns QEMU compilation. For system which have installed
tsocks, vm_setup.sh always tries to pass --with-git to QEMU's configure
even when tsocks is not really used (it determines that based on $CONF
which by default has tsocks enabled and presence of the tsocks binary).
Since in that scenario qemu compilation would simply fail, allow user
to force vm_setup.sh into simply ignoring tsocks alltogether.
This is an alternative for a change which potentially could break the
compatibility by removing tsocks from default $CONF list (I guess there
may be some systems out there that relay on tsocks and its default setup
in the $CONF).
Change-Id: Id3583357878002c79e1a554df664c297a0e39cb3
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2787
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Since most of the sources are executed in a background, there's a
race happening between different instances of package maangers
which are called to install extra packages (for QAT and QEMU).
This mainly concerns apt-get which will not wait for locks on
given files to be released, it simply exits when they are still
held.
To avoid this, simply install all the extra packages from a separate
function before calling routine for a given source.
Running these builds in a background is still not safe in that regard,
since code of the actual source may want to perform a similar tasks,
however, this patch should at least mitigate the issue in its current
form.
Change-Id: Iede83793e1cc739cd3bc5dd4dee90f9c81d55f63
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2786
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
In case vm_setup.sh is called on a bare system (e.g. cloud image), it
will instantly fail since gcc won't be on board, forcing user to either
install it by itself or by running our pkgdep.sh first.
Moving this check to pkgdep/git will mitigate similar issues since
before it's sourced, an $INSTALL portion of the script will have a
chance to run pkgdep.sh on its own (this still depends on user running
vm_setup.sh with -i arg, but still ;)).
Change-Id: I309ca42e8ea15fd14e416fc85fa0e244de2d8262
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2785
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Make sure that environment is preserved for each call to sudo to, e.g.
make sure proxy configuration is not lost when executing given package
manager.
Change-Id: If221d8d6546fc0ae4f02bc5814b6688c6d78cdd6
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2739
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Allow to specify a target place for all the git sources and use it
as a full path (instead of cding directly into $HOME). $HOME is
still kept as a default.
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Idb60ff8f8108368de2ac241971d221142198c89a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2528
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
We aren't going to run into any runtime issues when
linking against static libs, but we will when linking
against shared objects, so always test against shared
objects in autotest.
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I2d3d6e90f5fa4c38b3f44cf35e24b4f7e43b60ba
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2980
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This gets us ready for the case where we want to run with
shared objects.
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I84d22cf9def30a27aeb11fcf18eb80080a002ef9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3021
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Just set it once instead of doing it over and over.
This cleans up the code a lot.
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I91c6ea8c02d964693909b9c53795b0f1ee09c90f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2979
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
The previous revision to output parsing could
cause issues when more than one set of parenthesis
was included in the output.
fixes: 8661ded761
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I1cbfac1186fb5ee97518f4b687f38af59c0dcfb9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3092
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Community-CI: Mellanox Build Bot
As SPC4R31 5.9.3 "Exceptions to SPC-2 RESERVE and RELEASE behavior" says,
add a SPC2 RESERVE/RELEASE test case when PR is present.
Change-Id: Ibd7e703895cb910637677c6cb28b09e014976e9d
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2893
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Add a test case for SPC2 RESERVE/RELEASE from 2 hosts.
Change-Id: I7b2a6352aa8a90a291f495f059de8920d6cbc11b
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2892
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Some OS(Windows and VMWARE esxi) will send SPC2 RESERVE/RELEASE
commands when formatting the filesystems, but in our previous
implementation of reservation feature, we didn't add such support,
in specification SPC3/4, they all include one section "Exceptions to
SPC-2 RESERVE and RELEASE behavior" feature for compatible support of
SCP2 RESERVE/RELEASE, so we add this support now.
Fix issue #1414.
Change-Id: I65d85ffb3b8f824b0ae4e130f53f01d95735d700
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2802
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I59030c4570a03f8c6dfcf4a5f86426e3b5875744
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2527
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Perform it inside the a copy instead.
Also, don't copy the repo if target directory already exists.
Change-Id: Icdff80c978c7a8998b113db27bf744503bb29d0f
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2737
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
This is done in order to not taint the rest of the environment with
autotest settings.
Change-Id: Iad31096aa29d4893200e64c77f2cf22663f0dfc7
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2524
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Use 512B block size when running nvme_namespace_revert
routine. Need to use this block size because of
vhost_boot.sh test case.
Fixes#1452
Change-Id: I5f41cf2bf9d63b811849a99de6cba9b7b5a1d97d
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3003
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This is part of a larger series enabling failover at the bdev
layer for NVMe.
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I0d89e1afab8aeaa90237d0ba780d708154f6e3be
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2881
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
This API is a wrapper for rdma_accept which allows
to remove spdk_rdma_qp_init_attr::initiator_side.
Change-Id: Iba2be5e74e537c498fb11c939c922b2bbda95309
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2908
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
The old version of this test was using a function
that only detects NVMe drives bound to the kernel
driver so it was not finding any work to do.
Instead use identify to find NVMe namespaces.
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I937fc364254a926b8d251ec0cb38ea026593d977
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2788
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
If for any reason the cuse tests do not revert state
of NVMe namespaces to the original state, the test
machines would be stuck in until manual intervention.
Assumed default state is single namespace,
encompassing available space formatted as 4k block size.
This patch adds nvme_namespace_revert(), which
searches for NVMe devices that support Namespace Managment
and have some space unallocated.
When such device is found, all existing namespaces are removed
and single one created.
Fix#1435
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ifc3b04b3f166bf450b884674fa6a482e2fbc4c29
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2829
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
When the block size is 520(512 + 8) Bytes, we should check the data
length before the tests.
Change-Id: I785e395c9427460238205a52220e8333eee99ec0
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2905
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
In case when no device supporting Namespace Managment was found,
the test case exits without producing an error.
For example QEMU NVMe emulated device does not support it, thus
only physical devices have to be used right now.
This goes against design of "run_test" function, that when executed
should always run and never silently skip.
This patch adds check whether there is a device support and triggers
the test case only then.
Note that original code for selecting the device is still in original
script. This is to enable running nvme_ns_manage_cuse.sh on its own.
Right now CUSE tests are only enabled on VMs so nvme_ns_manage_cuse,
was added to exception list.
This will soon be reverted as the tests are moved to job that
runs on physical machines.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ie784696c2628b5fe45ce751b3978abcc03118cd9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2828
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Added more comments for clarity.
Moved namespace size calculation to the clean_up() function.
Changed size calculation from nvmcap (id-ns) to tnvmcap (id-ctrlr).
There is noneed to cache the size before deleting namespace,
if we assume that all devices had single namespace encompassing the
whole device.
Changed blocksize from 512 to 4k. All devices in CI should support that,
and usually thats the default.
This could be further improved by reading LBAF and checking the Relative
Performance (RP), but that would convolute the test scripts. This
is good enough.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I8a38d00ae1442edf5480e4e72a8a8aa90380cd79
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2827
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
It is already assumed that enviroment was prepared,
setup.sh ran binding the drives.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I864590b57db880b33c7b32deba8d38e4d5637b5b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2825
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This is used couple times in cuse test, so move it to common script.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I1e9f973e7fb53b0024fb320d37632e1e011ba3e9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2824
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
This will help with managing the test execution and preparation.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I0bd3612381d031a4015335d162f1296fd6dd213d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2823
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
"nvme_cuse" name for run_test was already used by one
of the tests in nvme.sh.
This removes the naming conflict and will prepare for further
restructure of nvme-cuse tests, where "nvme_cuse" name will
encompass all cuse tests.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: If141fe0ef492e0cde8e68490550de5d48fecb5e7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2822
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
In SPDK 20.07 will be deprecated, this means that CI
will no longer build or test the VPP component.
With SPDK 20.10 VPP the implementation and test scripts
will be removed.
autorun_post.py processes all run_test instances to
verify that they were executed. Since CI will skip
VPP tests, it needs to be added to list of exceptions.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I06c76643e6189d32471d9f53a0987e52f167844b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2771
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This will help us to change the host identifier if required, moreover,
some drives sometimes report set host identifier failed when startup,
so set host identifier at runtime making more sense.
Change-Id: I939c059d890b06aa2d51e82e0dede9630a6a1b51
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2860
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: wanghailiang <hailiangx.e.wang@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Enable Packed Ring option in performance tests. Only BLK
stack for now, as SCSI doesn't have this option yet.
Change-Id: Ie70642dd846ada35926e17204770ea2436089432
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2716
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Use system-wide fio if a specific binary was not provided.
Additionally add 1 second delay after finishing fio run
to make sure fio client-server finishes correctly.
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: Id7d1e7f70017755206c8fe9942ef8d6051a73bbe
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2559
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Make the variable local, so that we don't unnecessarily
read it's value from base script in case it's defined.
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: Ib4d95d3c67cfa8918c67f0b2642e9a3fe2ca3260
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2558
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
This makes them easier to find. They are now named
spdk_nvme and spdk_bdev.
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: I1f6736d8f7f5b9669583731fb0b710ba54f0d50a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2679
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Automatically place binaries produced from the app directory
into build/bin. This matches with the output in build/lib
that already exists.
Change-Id: I13cd2da71d2f88592e22308fe8a907bf458458b5
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2379
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
We will skip the test case if the drive didn't report such
capability, this isn't treated as an error case.
For vagrant environment, the namespace size is set to 1GiB or
2GiB, so here we also change the test LBA start from 0.
Also use printf instead of fprintf for normal exit.
Change-Id: Ib9f845ae548452921564aafebd2b23da6310b071
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2768
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
See commit msg from db43b387:
"Do not use iter_pci_class_code function in tests to
iterate over NVMe drives. This function can return
drives which at the moment of execution can not be
whitelisted for use.
This can result in test errors (such as simply
bdev_nvme_attach_controller RPC command failing) or
even using and deleting data from NVMe drive which
was not meant to be used in tests."
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ifa70a51822f78e95b6cddbfe665ea95ec1b423c1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2837
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
