btrfs-progs and xfsprofs are required by some of the tests, e.g.
the nvmf/target/filesystem.sh.
abigail package provides abidiff needed for check_so_deps.sh tests.
Change-Id: I5199fafef2ac03ce04ea3709e9ff76fefa1ac717
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2923
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
These tests take significant amount of time on bare-metal systems
causing some random timeouts in the CI pool. Reclaim some time
by lowering the number of test iterations.
Change-Id: Id70e00230e0e49dc0f2f1ccc4d499420fc84d7dd
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3843
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Make the footprint of the test smaller by creating smaller AIO file -
2G instead of 8G. Adjust offseting as well to not fall outside of the
device|file.
Also, cap in half the size of the source file.
Change-Id: I625eebe545477239606cd65ac6949cca4a2b8c80
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3842
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
This test is unnecessarily redundant since posix tests cover this
functionality in detail.
Change-Id: Ice7a6616a2106c098ff32595547371ea4c716d45
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3841
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
When opal_revert_and_init() is interrupted for some reason,
the spdk_tgt still exists, but it should be killed at the same time.
Change-Id: I8546d3b0b4d6a0fda1687558a664decb535ef2b4
Signed-off-by: WANGHAILIANG <hailiangx.e.wang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3830
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This allows for much more granular control over the timeout.
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: Ib23de21e60eec4207c55320579699edf284f4e16
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3794
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Originally, config->offset was defined as int type.
When the capacity of SSD is very large, such as 8T(P4510),
then bdev->blockcnt2 is 7814037168, config->offset is 3907018584.
At this time, it exceeds the maximum int range of 2147483647 and
becomes a negative number, resulting in core dumped.
Debug info:
config->filename is Nvme1n1.
make_cli_job_config offset is -387948712.
This should be:
config->filename is Nvme1n1.
make_cli_job_config offset is 3907018584.
Change-Id: Ia83d88cc4e56d6c97a6d3fc1a2593b6fc31655b2
Signed-off-by: WANGHAILIANG <hailiangx.e.wang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3818
Community-CI: Broadcom CI
Reviewed-by: GangCao <gang.cao@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Add MaxR2TPerConnection to iSCSI global options and make it configurable
by JSON RPC.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ida95e5c7dac301a22520656709e1aa4d611f31ef
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3777
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
By the recent refactoring, we have no static size array for outstanding
R2Ts per connection. It looks that we do not have any critical reason
to prohibit us from making max outstanding R2Ts per connection configurable.
There are some use cases to use large write I/O intensively (e.g. 128KB).
Let such use cases change the value of max R2Ts per connection by their
responsibility to do performance tuning.
Maximum outstanding R2Ts per task are defined both for iSCSI target
and NVMe-TCP target but maximum outstanding R2Ts per connection is
unique for iSCSI target.
The next patch will add the corresponding iSCSI option.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I4f6fd3c750a9a0a99bcf23064fe43a3389829aa9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3776
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Add MaxLargeDataInPerConnection to iSCSI global options and make
it configurable by JSON RPC.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ibcd16da2eac64241217bedeb89a7929bbdc67871
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3756
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
For some use case that there is heavy large read I/O, the performance
bottleneck due to MAX_LARGE_DATAIN_PER_CONNECTION was reported.
The following assumes that all I/Os are large read.
Large read primary task whose I/O size is more than
SPDK_BDEV_LARGE_BUF_MAX_SIZE (=64KB) is split into multiple
read subtasks.
spdk_iscsi_globals::MaxQueueDepth limits maximum number of outstanding
read primary tasks, and MAX_LARGE_DATAIN_PER_CONNECTION (=64)
limits maximum number of outstanding read subtasks.
MAX_LARGE_DATAIN_PER_CONNECTION is also used to calculate PDU pool.
To remove the performance bottleneck, change the macro constant
MAX_LARGE_DATAIN_PER_CONNECTION to a global variable
spdk_iscsi_globals::MaxLargeDataInPerConnection.
We don't see any negative side effect if we set
spdk_iscsi_globals::MaxLargeDataInPerConnection to 64.
The use case that reported the performance issue will change the
value of spdk_iscsi_globals::MaxLargeDataInPerConnection by its own
responsibility.
The next patch will add the value of
spdk_iscsi_globals::MaxLargeDataInPerConnection to iSCSI options,
and make it configurable by JSON RPC.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ifc30cdb8e00d50f4d3755ff399263cf5d0b681b6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3755
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This is to avoid including non-matching pattern as an actual item.
Change-Id: Ie4fbb27e66efa1f56618959bb7db6f0fccfc2847
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3290
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Get rid of WITH_DPDK_DIR and SPDK_RUN_INSTALLED_DPDK,
introduce SPDK_RUN_EXTERNAL_DPDK which can point to
a DPDK dir. It's an empty string by default.
Change-Id: Iff2b3773a4614db07f4196165087a79472e02b9a
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/867
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Originally the idea was to disable error checking,
to match output from Kernel and SPDK NVMe cuse.
This includes passing test commands and failures.
Any discrepancy would be caught by log output diff
at the end.
Flaw in this logic is that test command itself might
be incorrect. We shouldn't depend on that, nor
attempt to cover up some of the failures even if
they occur on both interfaces.
Most probable cause for this at all, was NVMe emulated
in QEMU not really working with all the nvme-cli commands
from this test.
Since the original creation of this test, CUSE executes
on physical devices (to be able to support namespace management).
The behavior there is predictable and works with current
test commands, thus the test exits on any error with this patch.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I086faf38b2cbbb6225935cc50d4fad14e81f1972
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3032
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
These suppressions are only valid for the course
of one release. Now that we have started with a
new release, remove all of the suppressions.
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I1a36bc49bb3a16f98de870cc06e56dbfa75d72d6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3722
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
The target changed how we deal with custom AERs
so we can't post them and expect to get an
immediate return anymore. The fuzzing application
doesn't expect to not get a return value from the
target so it just spins forever until it times out
and fails. The solution is to not send AERs over
the wire with the fuzzer.
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I24fc890dfc05601953552c98690950d0981d70fc
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3739
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Put the NVMF_SECOND_PORT and NVMF_THIRD_PORT to common file.
And clean some wrong comments.
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Change-Id: I2ccd420551bfa686a0d46e317b7a8767b171c2d9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3761
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
VPP implementation and tests are no longer present,
the skips can be safely removed.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I7a5f194ae87caae897bab53eca1370f56584ba94
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3735
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
All VPP tests were the same tests as usual posix iSCSI tests.
TEST_TYPE was used to direct which tests to run and in some cases
with what parameter.
Since VPP is deprecated, this type of selection is no longer needed.
All tests will run with default socket abstraction.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I292f1b5141ec8c4dacb4a43c6f51e4d30501a92a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3730
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Since VPP is deprecated, this is no longer required.
rpc_config tests will work for posix and uring
the same for this test.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ibc7bd76ea47ca137f3eb4484782cc64ff748fe22
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3729
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
We can do better than "sleep 5". Instead use waitforfile(),
that will wait until the namespace is available.
Moved around the variables with names and checks for the ctrlr/ns
existence after all the tests complete.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I8326c2cec0d0cf5aec88b5f80c1126c81eb09436
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2951
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Enviroment for VPP and posix tests was prepared differently.
Since VPP is no longer tested this patch removes path for
starting VPP application and configuring interfaces there.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ie55646207d07bbbd7edddd3caf2f2cc23bf27e23
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3728
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Other common scripts for nvmf and vhost already used
TEST_MODE set by autotest_common.sh.
Meanwhile iscsi was processing it for each test script
separately.
This patch simplifies iscsitestinit/iscsitestfini
functions to just use TEST_MODE.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Iacbdb552207eff72b41f2adb505e83348567df57
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3727
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
If $output_dir is not set, cwd is used as a designated output dir
throughout the runtime of the script that sourced autotest_common.sh.
This, however, may fail in some particular cases, e.g. when calling
autobuild.sh directly from within the spdk repo. As a result some
*.txt files that are created by scanbuild_make() will end up in the
root of the repo. This causes porcelain_check() to fail as it does
pick these files up after the test is done.
To avoid similar scenarios, simply make sure the desired output dir
is always created and that it resides outside of the repo.
Change-Id: I5b2e142ce9c2a0b8d24d5331f2b52110c9634c67
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2791
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Test steps were disabled due to issue #1146.
Which has since been resolved by patch:
(991a56f) fio: Continue polling spdk_threads on the init thread after shutdown
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Id5c2849a94b23e95bb222b8a70c538f392896a30
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3726
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: GangCao <gang.cao@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Host connects to nvmf in multiple paths, that we can use
different ports on one nic to test this case.
Then this case can be tested on all machines even though they have only
one nic.
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Change-Id: I021e2e1923361ecb4de1e260b2c277f7719f8335
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3590
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
lsblk used in waitforblk() works on blk device names
withouth full path (e.g. "nvme0n1"). This results in
a timeout because we pass "/dev/X" as an argument.
waitforfile() is the way to go in this case.
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I47a2b76bce6f15e0ffa2070e581bbf16f9647bdd
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3694
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This setting may be crucial on some systems in keeping core dump files
in order, that's why we should always make sure it's main value is
restored upon exiting from autotest.sh.
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I0c81a8c96c2d8f061e95c2614d9c30f4e3cdd5a6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3605
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This patch will pass when CUSE tests are switched from
job executing on virtual machines to physical machines.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I9c13106084197d8d197ee3c50c920c889c86e645
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2830
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
If the portal group map of the target has a redirect portal,
iscsi_tgt_node_is_moved() fills the buffer by the redirected address
and returns true.
iscsi_op_login_check_target() calls iscsi_tgt_node_is_redirected() before
calling iscsi_tgt_node_access() because login redirection can be
checked before any or after all security check.
If iscsi_tgt_node_is_redirected() returns true, notify login redirection
to the corresponding initiator.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I4573a69c0a32eafcfe48080a033c135e127da321
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3221
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
In SPDK iSCSI target, portal group works almost as identifier of
portal.
To support iSCSI login redirection, we need to have two types of
portal groups, public and private portal groups.
We need portals of public portal groups to redirect to a portal in
a private portal groups at login via temporary login redirection
funciton, and we need to make SendTargets return only portals in
public portal groups.
To do these simply, we mark primary or secondary portal group expicitly
at its creation by this patch.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Iccf87a4b9dd1f4a8fbb857a399b8f2dbc7c0b3ab
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3491
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
The NOT() function is meant to report a success when given test
fails. This negative testing is meant for specific cases, however,
there may be some circumstances where application failed, but not
for the reasons the tests was aiming for, e.g. upon receiving
signals from the kernel, like SIGILL, SIGSEGV, etc.
To make sure the test is aborted in such a scenario, check exit
status of the execution more carefully. If it matches any of the
defined signals make sure to fail the test. By default, all signals
which generate cores are looked up and SIGKILL which may be sent
out by the kernel as well (in case of the oom-killer).
Also, $EXIT_STATUS var is provided to specifically declare which
exit status NOT() should be looking for if process terminates with
a failure (i.e., for any other reason then receiving a signal).
Change-Id: I67b42ef038ef4553aceaf96d4da139858d819f22
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3448
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Make sure that we patch only a specific version of the driver that
we know works with given patches.
Check if $DRIVER_LOCATION_QAT looks like something we know and
proceed with patching if it does.
Change-Id: I27d82793ecfd6ad4cfb625cf14fcdda9fe822856
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3507
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
The pci_cleanup_aer_uncorrect_error_status() was renamed to
pci_aer_clear_nonfatal_status() in kernel 5.7, details here:
git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=894020fdd88c1e9a74c60b67c0f19f1c7696ba2f
Change-Id: I233718e02d95f46ccef4d78a4d1f626950cd8503
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3506
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
timespec and other time related types were hidden in 5.6 kernel,
details here:
git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=c766d1472c70d25ad475cf56042af1652e792b23
git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=412c53a680a97cb1ae2c0ab60230e193bee86387
Change-Id: I02ed16f9047337c77cc9ad344bbb40c52b3202be
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3505
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
This is done as a preperation for oncoming patches which will
require additional pieces of the kernel version to work with.
Change-Id: I365201d8254ee68a0201790dff29474545a4f50a
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3504
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
This patch is not needed for the newer version of the QAT driver which
vm_setup.sh installs - it successfully compiles against older kernels
(tested against 4.4.x sources).
Change-Id: Icd3e3171ddcdc1b5e4eb7c45736461cfcaa86cf7
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3503
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Previously we might have any possibility to need iaddr in
iscsi_send_tgts(), but it is highly unlikely now. Let's remove
the param iaddr from iscsi_send_tgts().
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ide3405706b727a9d6ebaa92ed83509196ff675da
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3442
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
The upcoming new feature, iSCSI login redirection will requests
connections whose portal group tag matches to logout asynchronously.
Hence add pg_tag to the second parameter of iscsi_conns_request_logout()
and iscsi_conns_request_logout() checks if conn->pg_tag is equal to
the passed pg_tag.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Iaea37f28046396404c5b4faed01d748f2944288c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3160
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Previously we might have any possibility to need conn in
iscsi_append_text(), but it is highly unlikely now. Let's remove
the param conn from iscsi_append_text(). This patch includes a
minor simplification not to use rc but return directly.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ifa009c1374e878f8d3b9c0043ba0352f59010b59
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3157
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This is part of a larger series enabling failover at the bdev
layer for NVMe.
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: Ic5c128244699c1a47275145ca7e41aa5f1366259
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3044
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
This patch paves the way for introducing a tailq containing
multiple alternative paths to the same controller.
Change-Id: I13d30c12b8e0ce38eae687f9e76740be1d11e4d1
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3259
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
RDMA target can't handle bidirectional xfer type, in debug build
it throws an assert in nvmf_rdma_setup_wr function. NVMF controller
performs checks od opcodes, but the failure happens before this
check. Add similar validation in TCP transport.
Change-Id: I14400b9c301295c0ae1d35a4330189d38aeee723
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3436
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Update reference build path to latest build.
Change path to use a generic directory name instead
of using specific SPDK version in the name. That way
in future we can update the build just after the
release, and still make ABI checks for patches which
were not rebased.
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I32850a871274773e334057b60b9df238ef85a999
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3532
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
This becomes a problem when the qpair is reconnected.
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I6677b396cf766684a4891ffbee93aa3e4e83374d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3391
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Some of the tests are skipped when using Soft-RoCE.
Test executed as part of run_test should never
skip a test, as it results in false assumption
that it was executed. See supplemental logs in CI output.
Before this patch bdevperf and srq_overwhelm (if executed),
resulted in misreporting this.
This patch moves all check for Soft-RoCE to top-level
nvmf test script.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I6d8941bbadaca2dbf8ab178e30e87d3786fea94c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3597
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Using Soft-RoCE is only relevant when testing
RDMA transport. For all other cases, we might
treat the IP address as non-rxe.
This simplifies the checks a little bit in couple places.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I996afb364070fbe107bbe97207952f99e4089308
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3596
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Soft-RoCE sometimes returns non-spec compliant values
in the WC entries after the qpair has been disconnected.
This can result in segmentation faults. In order to avoid
this, don't test shutdown on Soft-RoCE.
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I604ffaf38aa8da5efa349df0b1b51e7e391d17f8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3582
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Fixes#1498
When shutting down the application, it was possible to
reference stale ocf_cache pointer. This was the case
when two or more vbdev_ocf devices were based on top
of single cache bdev.
This issue did not occur outside of the shutdown case,
since RPC only allows deletion of the vbdev_ocf.
This erases on disk metadata and next run of the application,
would not detect such vbdev_ocf.
Shutdown meanwhile works different, by first stopping
the instance of running "ocf_mngt_cache" and later detaching
"core" devices (the ones being cached). This prevented
erasing the on disk metadata and allowed for restarted
application to detect vbdev_ocf.
See patch (1292ef2) for details.
Since references to ocf_cache are copied between vbdev_ocf
[see start_cache()], the reference count inside ocf_cache
was limited to original ocf_mngt_cache_start() and
management queue creation. First call into ocf_mngt_cache_stop()
released all references to ocf_cache. Leaving other
vbdev_ocfs pointing to released memory.
This patch works around this issue by increasing ref cnt
on ocf_cache for each vbdev based on top of it.
It allows to call into ocf_mngt_cache_stop(), but not
release the memory for ocf_cache until last vbdev.
Note:
A proper redesign here is in order:
- either rearranging structures to be based around single ocf_cache,
rather than multiple vbdev_ocf instances
- better use of OCF API to reduce book keeping logic in vbdev
There are plans to implement detach/attach in RPC,
so it should be a focus during that effort.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I560a7fbb1c052bf53970e655bdb60803c561a252
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3574
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Some images (especially cloud/containers) may do not
contain this package in default. So we need to add this package
install support.
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I2620026534009e8b6c320ecf61b8970947d5a9fb
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3218
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
To specify the desired logical block size. Must be 4K or 512.
If no block size is provided a default of 0 means to use the
underlying bdev block size. For cases where something other
than 4K or 512 is desired, format the underlying device
accordingly and don't specify a logical block size on creation
of the compress vol.
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I58b71e210cfa77b3237c0c454585c734e2e22aea
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3177
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Makefile support in DPDK was deprecated and will be removed soon,
so switch to the officially supported way of building DPDK -
with meson and ninja. Two new tools. Basically, our Makefiles
will invoke meson+ninja for DPDK, no other SPDK components are
affected.
Apparently DPDK wanted to move away from an octopus-like config
system and the ideology behind meson configuration is simple now:
build everything by default. Some PMDs can be explicitly disabled
with meson command line, but all libraries (both static and shared
versions) and test apps are built unconditionally.
How long does it take to build minimal DPDK with meson? Too much.
On my machine half of the total build time is spent on libraries
we don't need at all. (I have some hacks up my sleeve to disable
building those libraries - see the subsequent patch.) As for the
official way of building a minimal DPDK, there was a patch [1]
on dpdk mailing list to introduce more specific configuration,
but it was rejected:
> We talked about this a few times in the past, and it was actually one
> of the design goals to _avoid_ replicating the octopus-like config
> system of the makefiles. That's because it makes the test matrix
> insanely complicated, not to mention the harm to user friendliness,
> among other things.
>
> If someone doesn't want to use a PMD, they can just avoid installing it
> - it's simple enough.
>
> Sorry, but from me it's a very strong NACK.
Let's not follow that direction, hack the DPDK build system instead.
As for advantages of meson+ninja over Makefiles? I can't find any.
It's another build system that does a lot for you with some functions,
magic options, and a built-in dependency system. It seems nice if you know
the syntax, but it's another component that you need to learn, debug,
and possibly find bugs in (there's a lot of github issues open for meson).
I would compare it to CMake.
As for changes in this patch: rather that explicitly disabling
PMDs we don't need, specify a list of PMDs we do need and disable
everything else found in ./dpdk/drivers/*. This way we won't have
to disable the new PMDs as they're added to DPDK.
Meson configuration also sets RTE_EAL_PMD_PATH #define to a valid directory
with built PMD shared libs. When it's set, DPDK dynamically loads all shared
libraries inside. The drivers there depend on DPDK shared libs and fail to
load in static SPDK builds, so we disable them altogether by unsetting
RTE_EAL_PMD_PATH in the meson-generated config file - just like
DPDK Makefiles did. EAL checks for RTE_EAL_PMD_PATH being empty and skips
loading any external PMDs then. We do it for both static and shared libs.
We specify all PMDs at build time for now, so there's just no need to load
them dynamically.
We have three more hacks in our submodule:
* disable building dpdk apps by commenting-out a line in dpdk/meson.build
* disable building unnecessary libs (build everything that spdk *may*
need)
* build isa-l compress pmd with `-L[...] -lisal`. DPDK expects to find
libisal with pkg-config. We don't want to prepare a pkg-config file,
so comment-out a failing check in another meson.build file and provide
isa-l through CFLAGS and LDFLAGS.
We also need to make some changes to our test/external_code. First of
all, -ldpdk is no more. Meson build generates a pkg-config file with all
libs, but we'll switch to it in a separate patch - for now just specify
all -lrte_ libs one by one. -Wl,--no-as-needed has to be added to some
test cases, otherwise rte_mempool_ring isn't loaded. We don't use any
APIs from this library, it only has a static constructor that provides
a few callbacks used by rte_mempool_create(). Also, since DPDK now builds
both static and shared libraries, we need to add -Wl,-Bstatic to force
using static libswhere required. It's only needed for DPDK libs, but we
use it for SPDK libs as well since there's no harm.
As for performance:
$ ./configure --enable-debug --with-crypto --with-reduce
$ time make -j40 -C dpdkbuild all
with meson:
real 0m8.287s
user 1m7.983s
sys 0m10.548s
before, with the old DPDK makefiles:
real 0m20.232s
user 0m55.921s
sys 0m16.491s
The subsequent builds are much faster too:
$ time make -j40 -C dpdkbuild all
meson:
real 0m0.876s
user 0m0.663s
sys 0m0.217s
makefiles:
real 0m10.150s
user 0m11.740s
sys 0m6.772s
[1] http://inbox.dpdk.org/dev/1a07d1cd59d84dce84e56c10fdabf5e5504560a6.camel@debian.org/
Change-Id: Ic65db563014100bafb12e61ee0530cc2ae64401d
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1440
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
With update to DPDK 20.05 EAL requires rte_telemetry,
external make tests were missing specifying this lib.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I9f5eaea46b47da48fc81b4fff0199cd75ff6af78
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3483
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
related #1498
Updating DPDK 20.05 exposes issue listed in #1498.
Most likely the device removal path has to be re-worked
to accomodate multiple devices on the same cache instance.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I28f71ef2c2e1d9c2d19202cd33a932182bfb122c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3521
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Now that drivers can be registered from upper layers there's
no need to keep them centralized inside env.
(check_format.sh complains that spdk_pci_nvme_get_driver() shouldn't
start with the spdk_ prefix - to workaround that we move the function
declaration from one place in env.h to another - that's enough to
convince check_format it really is a public function)
Change-Id: If86aebd6c997349569c71430ec815b413eb44ef8
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3187
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This will allow to override job config filename from CLI.
Note that -T is required when using job config that has
missing filenames, but then it is optional if job config is not used.
Change-Id: I7d392ff2d0554524cfcb5072182ec806013083de
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3452
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Some global argument and some code from verify_test_params()
could be removed.
Change-Id: I923f5f23fafd31ff48049bd16476f01c801bfbb8
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3278
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Instead of calling bdevperf_construct_job() directly,
create a job_config instance that would express
the same semantics and then use
bdevperf_construct_config_jobs() to create and run actual jobs.
The goal is to unify methods and reduce code duplication.
Change-Id: I724e5788ea74868c9e9c8128d658e720d211f253
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3510
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This parameters allow for low cost refactoring
because with them we can simplify construct_multithread_jobs()
by creating instances of job_config during its invocation
and then handling those in construct_config_jobs().
Change-Id: I73057a9a65f58e48ac719f30e918d7a05ffc0f28
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3509
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
Implement parsing of config file,
including overall format of sections/parameters (using lib/conf)
as well as filename, bs, iodepth and cpumask parameters (based on FIO).
Also implement creating jobs based on config file.
The only required parameter that is missing is 'rw'.
It will be added in a separatate patch because it requires quite
different code than for other parameters.
Duplications that this patch introduces with respect to legacy
code are addressed in following patches.
Change-Id: Ie19e55dc66369e795a97b53a4223beafa10d8fe8
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3189
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
Accept job thread as argument
in bdevperf_construct_job().
This will allow for using single thread for
multiple jobs, as it is in FIO if multiple filenames are specified.
Change-Id: I0b6aa69aae1e1bf099170d3d495e0a83f97a857b
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3365
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
env_dpdk_get_mem_stats RPC has been requested to be
available in bdevperf.
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I10aea634851e21018b223d7331b1bbeb7630a09c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3520
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Parse fio output to CSV.
Calculate average read and write metrics while
parsing and present final results.
Change-Id: I8f6d7f84205be409f0ccbba7a42eb7888b2cb03d
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3253
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
There is a lot of extra lines at top of the output file
in case fio is used in client-server mode and with json
output mode. These lines make it resulting JSON file
not possible to parse.
Remove this lines so that the JSON file is ready to
be used in any later parsing functions.
Change-Id: Ie575941310f0d17b0285969f6b247f2fa653168c
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3237
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
app_repeat is used to test calling spdk_app_start/stop
repeatedly.
* "-t <num>" is used to specify the maximum number of
repeated rounds.
*SIGUSR1 can be used to trigger a new round of
spdk_app_stop/start.
*Shutdown signals like SIGTERM can be used to shutdown
the application.
Change-Id: I10a205ea2e9da3b4ecd6381bf25b457ce909a621
Signed-off-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2447
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
If intel_pstate is disabled in boot options, allow to
change CPU frequency using "userspace" cpu governor.
Change-Id: I390492c7be28df408f3cbb774ba5360859649fe0
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3376
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
For some reason the routine for generating fio
job/filename sections was different for kernel and
spdk fio modes. This made fio results hard to compare
because even configuration files themselves were not
comparable.
The function was modified to create the same fio
configuration files for kernel and spdk.
While at it, iodepth calculation has been slightly
modified as well. It is now being calculated for each
[ ] job section, so that each "filename" gets the
actual iodepth value specified in test run parameters.
While generating the config file the script will add
NUMA node information as a comment for "cpus_allowed"
and "filename" sections, so that it's clearly visible
how resources are aligned. In case of cross numa
configuration additional warnings will be printed to
screen as well.
For example, a configuration file for running test with
7 NVMe drives, 3 CPUs and effective (per filename) queue
depth of 32 would look like this:
run_perf.sh --cpu-allowed=0,1,2 --iodepth=32
--driver=kernel-classic-polling [...other params...]
--------
[filename0]
iodepth=96
cpus_allowed=0 #CPU NUMA Node X
filename=/dev/nvme0n1 #NVMe NUMA Node X
filename=/dev/nvme1n1 #NVMe NUMA Node X
filename=/dev/nvme2n1 #NVMe NUMA Node X
[filename1]
iodepth=64
cpus_allowed=1 #CPU NUMA Node X
filename=/dev/nvme3n1 #NVMe NUMA Node X
filename=/dev/nvme4n1 #NVMe NUMA Node X
[filename2]
iodepth=64
cpus_allowed=2 #CPU NUMA Node X
filename=/dev/nvme5n1 #NVMe NUMA Node X
filename=/dev/nvme6n1 #NVMe NUMA Node X
--------
Change-Id: Ida2f781fbb93c4a8c62154e711151152843ab997
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3370
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Configuration file generation was split into multiple
places/functions in scripts. Move it into a single
function. "time_based" fio parameter has been moved to
fio config template file as it is not configurable.
There is still a lot of room for improvement here, because
create_fio_config() function uses mixed local variables
(as function arguments) and global variables.
Change-Id: Ica3717d0d28692d24c861c2d32e729761c5c8647
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3345
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
In case spdk_dd was compiled with liburing support, rerun the tests
with forced AIO.
Change-Id: I26bee51b2671c19189a2678f48aef4809b7b1621
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3479
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
The new abort functionality doesn't take custom admin cmd
handlers into account.
This commit allows setting a custom admin cmd handler
for abort that provides the ability to influence the
bdev lookup to which the abort is sent to.
Signed-off-by: Michael Haeuptle <michael.haeuptle@hpe.com>
Change-Id: I3a66c6f863f5ee4d89cb2194dffdc6855945fa8a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3485
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
struct spdk_nvmf_request holds req_to_abort and so passing req_to_abort
separately is not really necessary now. The internal API
nvmf_ctrlr_abort_request() was added at the stage of prototyping.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I9ef2467d6f92422f044650c62a0777b95c0fc1ab
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3488
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Michael Haeuptle <michaelhaeuptle@gmail.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Now that we support only DPDK 18.11+ and always have
to register pci drivers to DPDK on initialization we
don't need that flag - it's always true.
Change-Id: Ibf1d79155595609fe9093f58e056bea25db6fdb2
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3446
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This follows struct rte_pci_id which had class_id as well.
We'll need it to make some additional DPDK APIs public through
the env abstraction.
Change-Id: I794a6cd6b17e48daf53b48fa5abe3d3dcfeaa403
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3182
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
You don't get notified when someone starts using your hooked
device, so there's not much gain from knowing when someone
stops.
Remove that callback and also move DPDK device detach under
the same lock which sets the pending_removal flag. This eliminates
a data race window when hotremove notification could arrive
after device was detached, but before it was scheduled to be
removed.
vmd and ioat nest the spdk_pci_device struct and abigail complains
even though the parent structs only have forward declarations in
public headers. Adding those two structs to the suppression list
doesn't help though. Abidiff still complains about the pci device
struct being changed, probably because ioat.h and vmd.h both include
env.h. Abidiff suppresion list should eventually be split per-lib,
but for now ignore struct spdk_pci_device changes globally.
$ abidiff [...]/libspdk_ioat.so [...]
'struct spdk_pci_device at env.h:652:1' changed:
type size changed from 1024 to 960 (in bits)
1 data member deletion:
<SNIP>
Change-Id: I9b113572c661f0e0786b6d625e16dc07fe77e778
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2939
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
A workaround for kernel deadlocks surfaced in #1275.
DPDK basically offers two APIs for hotplugging all PCI devices:
rte_bus_scan() and rte_bus_probe(). Scan iterates through
/sys/bus/pci/devices/* and creates corresponding rte_pci_device-s,
then rte_bus_probe() tries to initialize each device with the
supporting driver.
Previously we did scan and probe together, one after another, now
we'll have an intermediate step. After scanning the bus, we'll
iterate through all rte_pci_device-s and temporarily blacklist any
newly detected devices. We'll use devargs->data field to a store
a timeout value (integer) after which the device can be un-blacklisted
and initialized. devargs->data is documented in DPDK as "Device
string storage" and it's a char*, but it's not referenced anywhere
in DPDK. rte_bus_probe() respects the blacklist and doesn't do
absolutely anything with blacklisted ones.
The timeout value is 2 seconds, which should be plenty enough
for an NVMe device to reset, leave the critical lock sections in
kernel, and let us initialize it safely.
Note that direct attach by BDF doesn't respect the blacklist,
so an NVMe attach RPC won't be delayed in any way, it will continue
to work as it always did. Only the automatic discovery & enumeration
is deferred.
Change-Id: I62b719271bd0755bc2882331ea33f69897b1e5e5
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1733
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
There's no reason to keep usage and getopts
outside of the main script.
Other small changes while at it:
- use small letters for all "dir" variables
(consistent with most of other test scripts)
- try to move default & global variables into
one place.
- remove long comment from top of the main script.
It was basically the same as "help" for getopts.
Change-Id: Ic99f4bb1808b55dea80b534f7b4e0a65595ed053
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3335
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Read CPU list to use in test directly from option
value of from a file if it's an existing path.
This will make running tests in CI a bit easier.
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I174a38ab615c2f2cd2c3b4a95e0ebf886c478a51
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3333
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Allow to set custom size of bdev IO cache and pool
sizes.
Change-Id: I574d1e949f705d8f834539d5879f0a1e72865e00
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3297
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
