Commit Graph

14380 Commits

Author SHA1 Message Date
Ziye Yang
cd8b9455ad test: Enable uring test in CI pool.
This patch is used to enable uring test in some VMs or machines
in the CI pool.

Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I75c2ad477f5f648289d8dbb344b75b2408d56a38
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3107
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2020-07-03 07:31:08 +00:00
Maciej Wawryk
7303918290 scripts/pkgdep: Add uring install function
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Idabeee49f9a0b9093136cde214e8528f1abc37e9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2702
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
2020-07-03 07:31:08 +00:00
Jörg Thalheim
e7a86c42a4 ut/blob: more portable cunit version check
headers can be installed in different directories but `/usr/include`

Signed-off-by: Jörg Thalheim <joerg@thalheim.io>
Change-Id: Ibc4641813f6f1c904560dfe7d05cb9b40109cea7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3134
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-07-03 07:30:48 +00:00
Sochin Jiang
7137bde120 lib/blob: fix a potential wrong release of md page
blob_insert_cluster_msg() will release the already claimed
md page(intended for extent page) if the corresponding extent page
is found to be allocated. But later blob_insert_extent() may fails,
and this cause blob_insert_cluster_cpl() to release the same md page again,
this could be wrong if this specific md page is clamied by others, thus
cause data corruption. So, put it to zero after released in
blob_insert_cluster_msg().

Signed-off-by: Sochin Jiang <jiangxiaoqing.sochin@bytedance.com>
Change-Id: I46eba79b24b1950318002dcb27cb51b01ca566ec
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3152
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-07-03 07:30:30 +00:00
Jim Harris
16364fdc14 nvme: copy 8 bytes at a time into virtualized CMB
QEMU won't emulate wide (>8 byte) instructions on an MMIO
region.  We tried to fix this previously but it wasn't sufficient.
So this version adds an explicit copy function that will only
copy 8 bytes at a time from the source cmd pointer into the
SQ cmd point in the virtualized CMB.

Fixes issue #1458.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I804005a2e8cf3f7365c1608674f82a3f843e59ba

Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3135
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-07-02 07:21:38 +00:00
Alexey Marchuk
e762508854 nvme_rdma: Add check for keyed SGL length
The length of a keyed SGL data block is limited by 3 bytes.
Add a check to fail requests which length exceeds 3 bytes.
In other case we can send an incorrectly formed SGL request with
an invalid or zero length.

Fixes issue #1450

Change-Id: I77cdaff5fbf4be5754a3ac6008b8ccd532ac5905
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3056
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-07-02 07:21:31 +00:00
Tomasz Zawadzki
d09acccfe2 test/vm_setup.sh: prioritize dnf over yum
It is possible to have yum on Fedora (either manual install
or via upgrade), yet dnf is the default pkg manager there.
Re-order the list of checks to pick dnf first if available.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I48dfda2a248a7798c9d0c8b3cb4b8a3aa615e2a7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3057
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-07-01 09:13:03 +00:00
Tomasz Zawadzki
8bfb8c6a65 test/vm_setup.sh: exit with error when unsupported pkg manager is detected
Exit early from script execution when unsupported pkg manager is found.
Just not before the "--help" has a chance to be called.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I176bf5778dd7b2bc510f54842202113c8614a9d4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3054
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-07-01 09:13:03 +00:00
Tomasz Zawadzki
82540a6fd1 test/vm_setup.sh: use $package_manager var instead of $PACKAGEMNG
When trying to find out parameters through "--help",
the $PACKAGEMNG is "undefined".
So the output from help does not make sense.

Change it so that the help message uses $package_manager that
was detected.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ia7ac47dec810588dfa01441b35204a7bf13280ed
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3049
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-07-01 09:13:03 +00:00
Maciej Szwed
7ea417e5a3 spdk_top: Issue crash when pollers number changes
Root cause: We hit assert because we don't find
previous poller run counter. This happens when
new poller is created but 'last_run_counter' is
not being initialized for that poller.

Solution:
Force running store_last_run_counter() (which
inits last_run_counter) inside copy_pollers() by
simulating page change by changing value of
g_last_page and calling refresh_pollers_tab() again.

Fixes issue #1436

Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I983154ba1f7a46a92cba41c83691f71be8ab40b0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2847
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2020-07-01 07:54:03 +00:00
Maciej Wawryk
af0057e6e1 scripts/perf: Add effective QD counting
During perf test we figured out, that global qd
settings is split between disks number in filename.
Setting this parameter (multiplied by section disks number
and numjobs) in filename section give as expected value.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I89a44e48f10da131e2e9128f3ff2fbfb197076ad
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2749
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2020-07-01 07:53:54 +00:00
Karol Latecki
c0ff295493 perf/nvmf: add option to skip SPDK build on initiators
Skip copying and building SPDK on host systems if
specified in configuration file.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I4f9c0dd77e020dfd866f52dc138526e63dd9222d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3075
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
2020-07-01 07:53:32 +00:00
Karol Latecki
d3f46b279b perf/nvmf: aggregate read and write results
Aggregate separate read/write results into
a single stat. Reduces the number of additional
fields and formulas in spreadsheets when preparing
performance reports.

Change-Id: I5fdab35c1bb69aad98f7b9808885801db0416449
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3068
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-07-01 07:53:32 +00:00
Karol Latecki
9228dd5dd7 perf/nvmf: sum results from all test initiators properly
Test result IOPS and BW are a sum from across all initiators,
but latencies should be an average from accross initiators.

In previous version it was mistakenly a sum, and that had
to be manually handled in Excel spreadsheets when gathering
data for performance reports.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I17e38e3cdd593dcda9730c19decc2d6a72a11130
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3067
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
2020-07-01 07:53:32 +00:00
Karol Latecki
baa1f8a4a1 perf/nvmf: save per initiator results to a file
Save each initiator results to a .csv file. Useful
for a bit more insight than aggregated results.

Change-Id: I1fafe52ba78f2c62abfd0e3c863505515a363cd3
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3053
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
2020-07-01 07:53:32 +00:00
Karol Latecki
784553b681 test/iscsi: remove nvme bdev controller test case
Removing verify_add_nvme_bdev_rpc_methods test from
rpc_config.py.

bdev_nvme_attach_controller RPC call is frequently used in
a lot of other low-level test cases. Also this test case
fails when run on a VM with multiple NVMe disks containing
0-namespace and OCSSD emualted NVMes, which is most
probably a result of a very generic regex used to obtain
NVMe BDFs.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I091cc7fa4b426784136cda7423f829dc4a82eae0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3111
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-07-01 07:51:29 +00:00
Karol Latecki
afa0404c4b Revert "scripts/vagrant: Remove inline shell scripts"
This reverts commit 436b9be9d0.

Reason:
these change makes Vagrantfile rely on env variables (eg. spdk_dir),
which are only set when running scripts/vagrant/create_vbox.sh script.
After create_vbox.sh has finished env variables are not accessible,
but still needed to work with Vagrantfile using Vagrant commands
like 'vagrant ssh' or 'vagrant reload', etc. This results in
configuration errors from Vagrant and scripts being unusable.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I135c42d248c13dfa93c2210dabce49df8116e8f0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3055
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2020-07-01 07:51:29 +00:00
Ziye Yang
cb98b2ab3e nvme/tcp: Report that we have free entries if send_ack is set.
Without this patch, we will face the following warning code
when compiled with (--with-uring --enable-debug) while testing
big I/O size: 256KB, e.g.,

"nvme_qpair.c: 474:nvme_qpair_resubmit_requests: *ERROR*:
Unable to resubmit as many requests as we completed"

The reason is because the nvme_tcp_request structure is not freed yet
if send_ack is not set, so there will be no entries when there
are other requests submit again.

And this patch can mitigate such issue.

Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I4c7616fbd3c82a883b4e9facd257a1a4f66e876d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3123
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-07-01 07:51:17 +00:00
Ziye Yang
a213592c89 nvmf/tcp: Fix the error return code
It should return "NVME_TCP_PDU_FATAL". I think that
this issue is introduced after we move the data
copy from tcp transport layer to the socket
layer. And it should return "NVME_TCP_PDU_FATAL now",
and it will be consistent with the logic in the same
function.

With this patch, it will fix the big I/O size write
from the initiator.

Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: Ide018adb603eb13d002fc98886258dd1e2424f7c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3122
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
2020-07-01 07:51:17 +00:00
Karol Latecki
65d14fd9dd doc: add 20.04 nvme-of rdma performance report link
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I2891ed952d66fe14ccd710c47be7d6bfab8c7bea
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3033
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: GangCao <gang.cao@intel.com>
2020-07-01 07:51:09 +00:00
Changpeng Liu
021ebb3529 bdev_nvme/opal: remove unnecessary opal configuration
For base bdevs, we maintain a global list g_opal_base, and
for part bdevs, there is also a global list g_opal_vbdev,
so it's not necessary to use the configuration again, just
unpack it to opal part bdev.

Change-Id: Ia0943610dee3c194a80e44ed8eed8f2370d780fc
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3106
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-07-01 07:50:59 +00:00
Changpeng Liu
c3ace4f40c bdev_nvme/opal: use spdk_bdev_unregister() when deleting a vbdev
The base bdev's reference number is maintained in the bdev
layer, so we don't need to maintain it in this module.

Change-Id: Iffadbde2dbf35d934f273fdcb4b7d19e96394343
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2986
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: wanghailiang <hailiangx.e.wang@intel.com>
2020-07-01 07:50:59 +00:00
Michal Berger
0a0fe3df5c vm_setup: Remove exec() call from install_git
The 65b8c30d meant to use exec as a mean to "reload" the running
shell, however, exec simply replaces running bin image with a new
one, in this case, another shell. Since this call doesn't return
(unless it fails) this would suddenly leave the caller of the
vm_setup.sh inside the interactive shell.

Instead of re-executing the shell, simply export new PATH pointing
at where new git binary should reside.

Change-Id: I9ab5ba7f8f88602160c8be0f017477e0d1a01c08
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2790
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2020-07-01 07:50:49 +00:00
Michal Berger
4892b9353d vm_setup: Add arg for disabling use of tsocks
This mainly concerns QEMU compilation. For system which have installed
tsocks, vm_setup.sh always tries to pass --with-git to QEMU's configure
even when tsocks is not really used (it determines that based on $CONF
which by default has tsocks enabled and presence of the tsocks binary).
Since in that scenario qemu compilation would simply fail, allow user
to force vm_setup.sh into simply ignoring tsocks alltogether.

This is an alternative for a change which potentially could break the
compatibility by removing tsocks from default $CONF list (I guess there
may be some systems out there that relay on tsocks and its default setup
in the $CONF).

Change-Id: Id3583357878002c79e1a554df664c297a0e39cb3
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2787
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2020-07-01 07:50:49 +00:00
Michal Berger
a66c52fd5c vm_setup: Install extra dependencies from a separate function
Since most of the sources are executed in a background, there's a
race happening between different instances of package maangers
which are called to install extra packages (for QAT and QEMU).
This mainly concerns apt-get which will not wait for locks on
given files to be released, it simply exits when they are still
held.

To avoid this, simply install all the extra packages from a separate
function before calling routine for a given source.

Running these builds in a background is still not safe in that regard,
since code of the actual source may want to perform a similar tasks,
however, this patch should at least mitigate the issue in its current
form.

Change-Id: Iede83793e1cc739cd3bc5dd4dee90f9c81d55f63
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2786
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2020-07-01 07:50:49 +00:00
Michal Berger
0ccf1fc36b vm_setup: Move gcc version check to git sources
In case vm_setup.sh is called on a bare system (e.g. cloud image), it
will instantly fail since gcc won't be on board, forcing user to either
install it by itself or by running our pkgdep.sh first.

Moving this check to pkgdep/git will mitigate similar issues since
before it's sourced, an $INSTALL portion of the script will have a
chance to run pkgdep.sh on its own (this still depends on user running
vm_setup.sh with -i arg, but still ;)).

Change-Id: I309ca42e8ea15fd14e416fc85fa0e244de2d8262
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2785
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2020-07-01 07:50:49 +00:00
Michal Berger
7d2b98a770 vm_setup: Preserve environment when calling sudo
Make sure that environment is preserved for each call to sudo to, e.g.
make sure proxy configuration is not lost when executing given package
manager.

Change-Id: If221d8d6546fc0ae4f02bc5814b6688c6d78cdd6
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2739
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2020-07-01 07:50:49 +00:00
Michal Berger
70dfde6f1e vm_setup: Introduce $GIT_REPOS
Allow to specify a target place for all the git sources and use it
as a full path (instead of cding directly into $HOME). $HOME is
still kept as a default.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Idb60ff8f8108368de2ac241971d221142198c89a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2528
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2020-07-01 07:50:49 +00:00
Seth Howell
40d20acb72 autotest: always test against SPDK shared objects
We aren't going to run into any runtime issues when
linking against static libs, but we will when linking
against shared objects, so always test against shared
objects in autotest.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I2d3d6e90f5fa4c38b3f44cf35e24b4f7e43b60ba
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2980
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-07-01 07:50:27 +00:00
Seth Howell
3852afd6d8 test/nvme: in hotplug test copy lib files.
This gets us ready for the case where we want to run with
shared objects.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I84d22cf9def30a27aeb11fcf18eb80080a002ef9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3021
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-07-01 07:50:27 +00:00
Seth Howell
36e140b89e mk/spdk.nvmecli.mk: add some missing conditions.
We need to add options to include ocfenv, rte_vhost,
and libiscsi for building with shared objects.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I2ba144a6f0101cb84c5ae2d9595f4d58997a3a23
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3000
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-06-30 07:56:40 +00:00
Seth Howell
013b6b5234 move env.sh test under unittest label.
The env tests are actually mainly unit tests
and they can't run against shared objects because
we don't build unittests with shared objects.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I0eb1f8cc071539881a545f4e2c36559a825a36fc
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2981
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-06-30 07:56:40 +00:00
Seth Howell
05d92eaff7 autobuild.sh: configure without shared libraries for unittests
This will allow us to run the env tests with the other tests.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I1716ff25f088c49a010005ae883c57bf09b27bb5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2982
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-06-30 07:56:40 +00:00
Seth Howell
150b9f6219 test/external_code: set LD_LIBRARY_PATH once.
Just set it once instead of doing it over and over.
This cleans up the code a lot.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I91c6ea8c02d964693909b9c53795b0f1ee09c90f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2979
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-06-30 07:56:40 +00:00
yidong0635
c75dfb4201 nvme/perf: Free resources when abnormal exit.
There are anbormal cases, from callocated nvme qpairs until
connect io qpairs successfully .
We should free these resources when abnormal exit, and codes
just return there.

Signed-off-by: yidong0635 <dongx.yi@intel.com>
Change-Id: I08de9c1fe1757a11a118ebedc326d81e2c68b077
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2985
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2020-06-30 07:55:33 +00:00
Tomasz Zawadzki
4c156e6379 lib/scsi: downgrade SCSI SO version
Internal headers (such as one in SCSI) should not trigger the
SO versoning fialures. This was addressed by patch:

(f53bf76)test/check_so_deps: limit scope of header checks.

Additionally quite recently another patch increased the SO version
for SCSI:
(6b6a3ff)scsi: add SPC2 RESERVE/RELEASE support

They were not in the same relation chain when being submitted,
so they didn't test against each other.

This resulted in:
SO name for libspdk_scsi.so changed without a change to abi. please revert that change.

For this reason this patch revs down the SO version for SCSI lib.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I69caae6c139e546afe2eeeae3395dc80ff898339
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3114
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-06-29 15:01:57 +00:00
Darek Stojaczyk
59a91950ec Pull two additional patches into the DPDK submodule
Pull the following (from branch spdk-19.11.2):
083306680 build/meson: disable libraries we don't need
8c977596d build: don't parse build configs of explicitly disabled drivers

Change-Id: Ib38a0136d3ef6984a67dbb5a092e9e166ae90167
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1524
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-06-29 09:20:23 +00:00
Yan Wang
abeae98424 lib/blob: check return value after malloc() and strdup()
Signed-off-by: Yan Wang <wangyan122@huawei.com>
Change-Id: Ieaaecfd12ddf7826c2e4ce6dc5f47411e7e077da
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2907
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2020-06-29 09:20:12 +00:00
Darek Stojaczyk
1befc43b2d json_config: fix error message for loading failures
resp->error is usually a JSON object, but it was printed as
a string. For objects, resp->len was the number of fields in
this object, definitely not a length of the string to print.
There was usually just 3 characters printed with no newline.

To print this object we should stringify it using our json_write
library, so that's this patch does.

Before:
json_config.c: 192:rpc_client_poller: *ERROR*: error response: {"co
(missing newline at the end)

After:
[2020-06-19 13:33:57.060869] json_config.c: 220:rpc_client_poller:
*ERROR*: error response:
{
  "code": -32601,
  "message": "Method not found"
}
[2020-06-19 13:33:57.061067] app.c: 714:spdk_app_stop: *WARNING*:
spdk_app_stop'd on non-zero

Change-Id: I5d7aeba2b972782f18b8da0141ed4bfd79833f80
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2971
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-06-29 09:20:01 +00:00
Seth Howell
f53bf7676c test/check_so_deps: limit scope of header checks.
This prevents us from triggering false positives on internal
headers.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I4e7f5a8acb7da4351f21dd7619695a6a59ab1881
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2914
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Community-CI: Mellanox Build Bot
2020-06-29 09:19:46 +00:00
Seth Howell
03b8afa669 test/check_so_deps.sh: fix output parsing.
The previous revision to output parsing could
cause issues when more than one set of parenthesis
was included in the output.

fixes: 8661ded761

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I1cbfac1186fb5ee97518f4b687f38af59c0dcfb9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3092
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Community-CI: Mellanox Build Bot
2020-06-29 09:19:46 +00:00
Seth Howell
27f435bc1d lib/nvmf: raise RDMA acceptor backlog to 100.
10 is kind of unreasonable. You could easily start
seeing failures if you had just 3 intiators trying to
connect with 4 io qpairs at once.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I9985ffa3b03ebb33880eb5934b60eaafab57c82d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3096
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-06-29 09:19:09 +00:00
Seth Howell
203ed4f673 lib/nvme: report rdma_connect errors up the stack.
This will allow applications to discern specific connect
behavior and make choices relative to it.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I46182c285367ceb8a72511defe4508b3592b4572
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3095
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-06-29 09:19:09 +00:00
Alexey Marchuk
8421f83973 rdma: Fix qpair desctruction in error flow
rdma_qp may not be initialized when qpair is not fully
created. When such a qpair is being destroyed we may pass
a NULL pointer to spdk_rdma_qp_disconnect or spdk_rdma_qp_destroy
and hit an assert. This patch fixes this problem for NVMEoF
target and initiator.

Change-Id: I84787dc1b1211293c2a19f59d47727eaecd9d5a1
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3050
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-06-29 09:18:52 +00:00
Michael Haeuptle
89013903fe NVME: Fixes stuck IOs during hot remove (#1451)
When a NVMe device is hot removed, subsequent calls to
nvme_qpair_submit_request can fail with ENXIO.
The failure path handling for ENXIO did not free the request which
exhausts the qpair's free_req list eventually and all IOs are stuck
going forward.

This fix adds the same cleanup handling to nvme_qpair_submit_request
for this error case as it is done in _nvme_qpair_submit_request.

Signed-off-by: Michael Haeuptle <michael.haeuptle@hpe.com>
Change-Id: I5677d53965bdbd6d339c013483cdf42ce782099a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3018
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2020-06-29 09:18:29 +00:00
Ziye Yang
ceb07eb8f4 nvme/tcp: Fix send_cb and recv pdu function contention when there is R2T.
When using uring socket, we see following assert

nvme_tcp.c:1018: nvme_tcp_capsule_resp_hdr_handle:
Assertion `tcp_req->state == NVME_TCP_REQ_ACTIVE' failed.
Detailed info is in

https://ci.spdk.io/results/autotest-per-patch/builds/19205/archive/nvmf-tcp-vg-autotest/build.log

We face this issue, because there is also code execution ordering
between "sending callback function" and "pdu receving function".
We did not find it in physical machine testing, but finding it
in vagrant machine in CI.

Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I5eb241d564c0fc42ce0601b7c85999a2550f0de3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3046
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-06-29 09:18:13 +00:00
Ziye Yang
c833729b22 sock/uring: Fix the nvmf_shutdown_tc3 failure issue.
If the socks parameter(passed in uring_sock_group_impl_poll) is NULL,
we do not need to handle the sock_flush and prep the pollin task,
otherwise it will cause the assert issue when we reap the task when
we handle the nvmf_shutdown_tc3 issue.

Because in uring_sock_group_impl_remove_sock, we finally
set sock->group = NULL. Without this patch,
when we call uring_sock_group_impl_poll in this function,
pollin_task or write_task are prepared, then in the next round,
we will reap those tasks again.

PS: Error info can be found in
https://ci.spdk.io/results/autotest-per-patch/builds/19186/archive/nvmf-tcp-vg-autotest/build.log

Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I7e6deaa05e958b52e71e0bbf0ccdd20e35583685
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3031
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-06-29 09:18:13 +00:00
WANGHAILIANG
74050eacd7 script/perf: move "make clean" before "./configure"
I suggest putting "make clean" before "./configure". Like Migration-tc3a.sh (spdk/test/vhost/migration):
ssh_remote $MGMT_INITIATOR_IP "cd $spdk_repo_share_dir/spdk; make clean; ./configure --with-rdma --enable-debug; make -j40"
Otherwise, once the configuration is changed, some outdated compiled files may remain after "make clean".

Change-Id: I7300749986129cbfaef44f8ff72fad0449f4e081
Signed-off-by: WANGHAILIANG <hailiangx.e.wang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2805
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Liang Yan <liang.z.yan@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-06-26 07:21:15 +00:00
WANGHAILIANG
a1c00096dd example/hello_blob: Drop .ini config in favor of json
Change-Id: Iea222f4ef072676cbb5e9014a7d98ac08a4e617a
Signed-off-by: WANGHAILIANG <hailiangx.e.wang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2967
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-06-25 07:16:19 +00:00
Changpeng Liu
b18895d025 UT/scsi_reserve: add Persistent Reserve with SPC2 RESERVE/RELEASE test case
As SPC4R31 5.9.3 "Exceptions to SPC-2 RESERVE and RELEASE behavior" says,
add a SPC2 RESERVE/RELEASE test case when PR is present.

Change-Id: Ibd7e703895cb910637677c6cb28b09e014976e9d
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2893
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-06-25 07:16:08 +00:00