Commit Graph

18947 Commits

Author SHA1 Message Date
Jim Harris
8dc878483d env_dpdk: allow 2211.c file to build against older DPDK
The 2211 implementation only gets used when runtime
detects the DPDK version is DPDK 22.11. But we still
compile this file even if it gets built against an
older DPDK.

This is typically fine, except there are some interrupt
APIs that changed in DPDK 21.11, so older DPDKs don't
have some of the functions used in this file.  We need
to use ifdefs to allow this to compile.

We will need some more work to handle this case properly,
but this patch at least fixes the 2211.c case for now.
We will probably need a 2108.c file that exactly matches
the 2207.c file except for this interrupt API changes.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I6055694ccbb79845798e750ebb7127ec6c160e2e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15236
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michal Berger <michal.berger@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2022-11-15 08:31:28 +00:00
Jun Wen
74da5ee257 test/lvm_nvme_fuzz: return error code in TestOneInput
When get `spdk_nvme_connect() fail` error, need to stop fuzz, so return -1 here.
Call `spdk_app_stop()` in `start_fuzzer`

Fix #2744

Signed-off-by: Jun Wen <junx.wen@intel.com>
Change-Id: I301012931a4f618edd84f3ee6f371c187755012e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15160
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: wanghailiang <hailiangx.e.wang@intel.com>
Reviewed-by: <qun.wan@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: <yifan.bian@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2022-11-15 08:31:20 +00:00
Michael Piszczek
1473d3b8c2 env_dpdk: fix check for AMD iommu
Update code for read the virtual address width to use glob to locate the
Intel and AMD iommu capability registers. This code should work for all
AMD numa configurations.

Fixes issue 2730

Signed-off-by: Michael Piszczek <mpiszczek@ddn.com>
Change-Id: Ibf5789087b7e372d892b53101e4c0231809053f0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14961
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Mellanox Build Bot
2022-11-15 08:31:13 +00:00
John Levon
0d0de8e7d9 lib/rpc: add RPC allow list
Add an optional allowlist for RPC methods: if the method is not listed,
it is not allowed to be called or visible. This can be used to restrict
accidental mis-configurations, and generally helps locking down the
configuration surface.

Signed-off-by: John Levon <john.levon@nutanix.com>
Change-Id: Ied78fc4b14b60cb94ed0852b92deb6df545cbec4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15275
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-11-15 08:31:02 +00:00
John Levon
1139cb1415 lib/util: add strarray utility functions
Add some basic utilities for handling arrays of strings.

Signed-off-by: John Levon <john.levon@nutanix.com>
Change-Id: I2333f3e4605175b1717a7f289847ff2d48745e8d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15274
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Thanos Makatos <thanos.makatos@nutanix.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-11-15 08:31:02 +00:00
Jim Harris
c3e62d7729 test: add new cmdline.sh test script
This can be used to test various app command line
parameters - in this case, the new RPC allowlist
parameter.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iaa0e90940ac4fb0b044ab96bc2912afa9f48ddae
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15310
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-11-15 08:31:02 +00:00
Karol Latecki
bfe724a069 doc: add NVMe-oF TCP 22.09 performance reports links
Change-Id: Ibac2412f16f62024a7bb4787b3ad4a515719951a
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15333
Reviewed-by: Michal Berger <michal.berger@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-11-15 08:12:50 +00:00
Michal Berger
66c590e0b4 pkgdep/rhel: Upgrade pip under centos8, rocky8
Do this prior installing any pip package to make sure they are stable.

Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I842252fe33b62dddd16858806a7d334cf2139fac
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15292
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Kamil Godzwon <kamilx.godzwon@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2022-11-10 13:20:20 +00:00
wanghailiangx
3afa8c2f43 doc/accel_fw: fix RPC accel_assign_opc -e to -m
./scripts/rpc.py accel_assign_opc -h
usage: rpc.py [options] accel_assign_opc [-h] [-o OPNAME] [-m MODULE]

optional arguments:
  -h, --help            show this help message and exit
  -o OPNAME, --opname OPNAME
                        opname
  -m MODULE, --module MODULE
                        name of module

Change-Id: I597da36ca14430a3d3a5054bd352b67201e8be4a
Signed-off-by: wanghailiangx <hailiangx.e.wang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15345
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-11-10 08:29:19 +00:00
Mykola Golub
875ccefcdb rpmbuild: improvements for building with rbd
Add librbd and librados install and build dependencies when "with-rbd"
config option is set.

This will be used by the Ceph NVMe-oF gateway.

Signed-off-by: Mykola Golub <mykola.golub@clyso.com>
Change-Id: I1534e224cdfc166061a166ceef18261eef2d8cd8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11135
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-11-10 08:29:10 +00:00
Mykola Golub
8a5d487d7e scripts/pkgdep: add RBD support
Adds "--rbd" option to install build dependencies for the rbd bdev.

This will be used by the Ceph NVMe-oF gateway.

Signed-off-by: Mykola Golub <mykola.golub@clyso.com>
Change-Id: I7926433aaa398873133dfffea3c401ce3c1f7813
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11134
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2022-11-10 08:29:10 +00:00
paul luse
17538bdc67 add (c) and SPDX header to python files as needed
per Intel policy to include file commit date using git cmd
below.  The policy does not apply to non-Intel (C) notices.

git log --follow -C90% --format=%ad --date default <file> | tail -1

and then pull just the year from the result.

Intel copyrights were not added to files where Intel either had
no contribution ot the contribution lacked substance (ie license
header updates, formatting changes, etc)

Note that several files in this patch didn't end the license/(c)
block with a blank comment line so these were added as the vast
majority of files do have this last blank line.  Simply there for
consistency.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I6cd3f18d1b469d5ef249d26ddb2923ca6b970bd4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15208
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-11-10 08:28:53 +00:00
paul luse
7fc8a7dd45 add Intel copyright notices (and/or SPDX lic) where missing
per Intel policy to include file commit date using git cmd
below.  The policy does not apply to non-Intel (C) notices.

git log --follow -C90% --format=%ad --date default <file> | tail -1

and then pull just the year from the result.

Intel copyrights were not added to files where Intel either had
no contribution ot the contribution lacked substance (ie license
header updates, formatting changes, etc)

Note that several files in this patch didn't end the license/(c)
block with a blank comment line so these were added as the vast
majority of files do have this last blank line.  Simply there for
consistency.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: If1bf46b4e1902585c318dc35b1673f980366076e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15199
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-11-10 08:28:53 +00:00
paul luse
a6dbe3721e update Intel copyright notices
per Intel policy to include file commit date using git cmd
below.  The policy does not apply to non-Intel (C) notices.

git log --follow -C90% --format=%ad --date default <file> | tail -1

and then pull just the 4 digit year from the result.

Intel copyrights were not added to files where Intel either had
no contribution ot the contribution lacked substance (ie license
header updates, formatting changes, etc).  Contribution date used
"--follow -C95%" to get the most accurate date.

Note that several files in this patch didn't end the license/(c)
block with a blank comment line so these were added as the vast
majority of files do have this last blank line.  Simply there for
consistency.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Id5b7ce4f658fe87132f14139ead58d6e285c04d4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15192
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
2022-11-10 08:28:53 +00:00
Mike Gerdts
30cbc5a0d2 remove spdk_bdev_create_bs_dev_from_desc from map file
Commit 3e15b356f3 removed spdk_bdev_create_bs_dev_from_desc but missed
removing it from the map file.

Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: I0e0932e787b688753478a08f3b5b79e441ba30e7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14584
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
2022-11-09 08:19:04 +00:00
Krzysztof Karas
344249069d event: add runtime cpu lock configuration
Allow CPU core locks to be enabled and disabled
during runtime. This feature will be useful
in cases like SPDK hot upgrade, where
locking should be disabled temporarily.

Change-Id: I9bc7292fd964abffc7214d074d191f38b13583c3
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15031
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
2022-11-09 08:18:32 +00:00
Krzysztof Karas
0af934b38c event: add CPU lock files
When running SPDK application on a given set of
CPU cores, create lock files for each of them.
This wil prevent user misconfiguration and
assigning a core to more than one SPDK instance.

The introduced mechanism is based on device locks
implemented in spdk_pci_device_claim() function.

Add a command line option to disable lock files.
This feature will be useful in cases where differing
CPU cores is impossible (eg. setup with only one core
available).

The patch also fixes all existing cases of overlapping
core masks.

Change-Id: Ie9aacb7523a3597b9aa20f2c3fa9efe4db92c44c
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14919
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
2022-11-09 08:18:32 +00:00
Kozlowski Mateusz
311d014d17 scripts: Update spdk_print_io_devices and spdk_print_threads gdb macros
The structs io_device and spdk_io_channel were moved from using tailq list to rb tree,
moving the parsing in gdb functions to use those too.

Fixes issue #2763

Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Change-Id: I8fa5a69b5004e3d86fdfa201c2ba47e06fa47cfa
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15330
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michael Haeuptle <michaelhaeuptle@gmail.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
2022-11-09 08:16:49 +00:00
Michal Berger
b35a6f39a6 pkgdep/rhel: Make sure pip3 is installed in first instance
There was a code that attempted to to use pip3 before we try to
actually install it. Change the order.

Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I5dc74b79b3adf132589bbf9ef10360d2eeed51ba
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15293
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-11-08 08:21:59 +00:00
Michal Berger
71605a520d scripts/setup: Fix extglob match
The actual target files come with a .* suffix.

Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I1c6056c81261ae97bd272d0d99ad915ab4bb870e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15294
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-11-08 08:21:59 +00:00
Artsiom Koltun
29416c0a54 sma: report device level QoS capabilities for virtio-blk.
Virtio-blk supports one to one mapping between devices and volumes.
Based on that, we can report underlying bdev capabilities as device
level QoS capabilities.

Signed-off-by: Artsiom Koltun <artsiom.koltun@intel.com>
Change-Id: If7c28c75d087fecec0105002994581fd7ac57f51
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15260
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-11-08 08:21:46 +00:00
Jim Harris
51ab849bdf configure: emit command line options to mk/config.mk
This can be useful to know which command line parameters
were passed to configure to generate the current
build environment.

Fixes issue #2758.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ia662c73072101abfc5f59c463f4a1c821c3fa0e0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15249
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: wanghailiang <hailiangx.e.wang@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Community-CI: Mellanox Build Bot
2022-11-08 08:21:35 +00:00
Shuhei Matsumoto
3d2a3ee4df bdev/error: Support injecting data corruption into I/O buffer
Support injecting data corruption for processing read/write I/O.

Add two parameters, corrupt_offset and corrupt_value, and an error
type corrupt_data.

For write I/O, inject data corruption before submitting write I/O
to the underlying bdev. For read I/O, inject data corruption after
read I/O successfully returned.

Cause the data corruption by xoring the value at the corrupt_offset
with the corrupt_value. corrupt_value is required to be non-zero to
cause data corruption necessarily.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I67d8d252b06d7a221413e74996f7f894e6519556
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15028
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Community-CI: Mellanox Build Bot
2022-11-08 08:21:22 +00:00
Shuhei Matsumoto
e0daee9840 bdev/error: Use spdk_bdev_part_submit_request_ext() to use custom completion callback
This is a preparation to inject data corruption at completion for
read I/O.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I839f454d643254f2b805f3d4c65282deee037d71
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15003
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-11-08 08:21:22 +00:00
Konrad Sztyber
cff39ee7d5 nvme: add missing \n in ctrlr init fail log
Additionally, print the string representation of the ctrlr state, as it
makes debugging init failures much easier.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I572ef3d6f7d5bbd52039a8872733578c92be4c4a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15305
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-11-08 08:20:26 +00:00
Tomasz Zawadzki
88f16162e7 test/scheduler: make spdk_for_each_reactor test optional
(84ab68c) test/scheduler: add a for_each_reactor shutdown test

Patch above added regression test for handling spdk_for_each_reactor
during shutdown, by adding constant repeat of this operation
while application is running.

Reactor event processing (especially constant) has impact on the
reactor_interrupt_run(). spdk_fd_group_wait() will almost always
execute an event, skewing the results of scheduler test.
Reactor that should have been idle, will show active usage via
/proc/stat.

Fixes #1950

This patch makes this regression test optional, and enables it
only in test that does not measure CPU utilization from the system.
The ./test/event/scheduler/scheduler.sh is the only one where it is
enabled, as it's purpose is to verify the test scheduler application.
Remaining ./test/scheduler/*.sh tests do verify CPU utilization,
so the regression test is disabled in those.

Modified the for_each_done, to for_each_reactor_start, to better
reflect the intention.

On my system enabling spdk_for_each_reactor test flag on the
scheduler application with no threads (except app thread),
consumes ~20-25% CPU from every core in CPU mask.
Meanwhile disabling it, idle cores are 100% idle
and active cores spend 100% of CPU time in usr.

Change-Id: I40eda15a748e76b95dc5441144cd8931e46edee5
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15210
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2022-11-08 08:19:44 +00:00
Richael Zhuang
cabbb25d5d bdev: add API to get submit tsc of a bdev I/O
Add API spdk_bdev_io_get_submit_tsc to get submit tsc of a bdev I/O,
which can be used in bdev modules to avoid calling expensive
spdk_get_ticks().

Change-Id: Ifbcecb1bc663344997c5e73b72a1dfb5d0422946
Signed-off-by: Richael Zhuang <richael.zhuang@arm.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14989
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2022-11-04 10:15:46 +00:00
Denis Nagorny
c273513401 nvme/rdma: Allows to use PCI Express Relaxed Ordering
This fix allows to use relaxed ordering feature where it is
supported. libibversb checks with the driver if relaxed ordering
access flag is supported and ignores it if not.

Experiments show that set by default it doesn't spoil performance but
allows to reach desired one on AMD EPYC systems. For example fio read
test (ConnectX-6, AMD EPYC 7763, two jobs, queue depth 32, block size
32K) can starve down to 6-7 GiB/s without it. Enabling this option
allows to get bandwidth more than 21 GiB/s.

Change-Id: I5983aed5d1f38ee7bec9c310597731c9a6a329da
Signed-off-by: Denis Nagorny <denisn@nvidia.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14885
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-11-04 10:15:31 +00:00
KanKuo
aba9c09c0b UT/nvme_pcie: test for nvme_pcie_ctrlr_map_io_pmr
Signed-off-by: KanKuo <kuox.kan@intel.com>
Change-Id: Iec9f5b4dc42a5a707b50f7b12226e0ea0c7eb39d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10463
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: wanghailiang <hailiangx.e.wang@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-11-04 10:11:16 +00:00
Thanos Makatos
e51dde79b8 thread: use existing EPOLL defines
Signed-off-by: Thanos Makatos <thanos.makatos@nutanix.com>
Change-Id: Ic8ed31c7c2a4b915ec969a0757fca73ebbfb4488
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14934
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-11-04 10:10:33 +00:00
Thanos Makatos
b8fc75c36e nvmf/vfio-user: ensure BAR5 isn't 0
Signed-off-by: Thanos Makatos <thanos.makatos@nutanix.com>
Change-Id: I60a39c8a311879b7d6c7c82df0abd7a69f9a2778
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14933
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-11-04 10:10:33 +00:00
Thanos Makatos
bad452d25e nvmf/vfio-user: calculate doorbells based on number of queue pairs
It doesn't make sense to have the size of the doorbells fixed and then
calculate the maximum number of queue pairs based on it, do it the other
way round. Also, add some sanity checks based on the spec.

Signed-off-by: Thanos Makatos <thanos.makatos@nutanix.com>
Change-Id: I17e3509fb0a011128ca089ce78b7a296262e6f8e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14932
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-11-04 10:10:33 +00:00
Alexey Marchuk
5d39b20a50 bdevperf: Call bdev*_with_md even if md is NULL
The bdev*_with_md APIs now allow to pass NULL md
pointer, so calling this function without checking
for metadata simplifies code

Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: I32b2f85e375b858c25beb7e2128132dd604ca040
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15092
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2022-11-03 14:54:41 +00:00
Alexey Marchuk
6fa5007edd bdev/zone: Call bdev*_with_md even if md is NULL
The bdev*_with_md APIs now allow to pass NULL md
pointer, so calling this function without checking
for metadata simplifies code

Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: Ie4137f7a6a7628a13d14c7c9a5e9aa1ceb99d322
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15091
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2022-11-03 14:54:41 +00:00
Alexey Marchuk
0fec09fc50 bdev/part: Call bdev*_with_md even if md is NULL
The bdev*_with_md APIs now allow to pass NULL md
pointer, so calling this function without checking
for metadata simplifies code

Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: I364a646630bd36120231ea87a41fea05df51befb
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15090
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2022-11-03 14:54:41 +00:00
Alexey Marchuk
c89891ea8c bdev/delay: Use ext bdev API
Fixes commit c3a5848 where support of memory
domains was added without usage of the ext API

Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: I7b318f515d7421b8876d4717c0ef293084401bbc
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15089
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2022-11-03 14:54:41 +00:00
Alexey Marchuk
d8d1a4dd38 bdev/passthru: Use ext bdev API
Fixes commit c3a5848 where support of memory
domains was added without usage of the ext API

Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: Ia0d7132f11c233e334965669ab0d237c24074745
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15088
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2022-11-03 14:54:41 +00:00
Alexey Marchuk
21db73f909 bdev_nvme: Return memory domains of each controller
Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: I7417dcf69bbb8a526308075459c5887283896823
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15087
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2022-11-03 14:54:41 +00:00
Shuhei Matsumoto
fa5e7d1b8d bdev/error: Use switch-case to process error injection at submission
The following patches will support data corruption.
For write I/O, data corruption will be injected before submission,
and for read I/O, data corruption will be injected after completion.

To do these cleanly, use switch-case and reorder to process error
injection at submission.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I3b830b4331cb4c7d0794a555957cdcc73902c14f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15026
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Community-CI: Mellanox Build Bot
2022-11-03 14:54:28 +00:00
Shuhei Matsumoto
7078874b80 bdev/error: Passthrough I/Os other than read, write, unmap, or flush
If we use error bdev in general use cases, the upper layer may submit
I/O commands other than read, write, unmap, or flush.

However, before this patch, the upper layer could submit only read,
write, unmap, and flush.

To improve the usability of error bdev, pass thorugh I/Os other than
read, write, unmap, or flush.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: Ia642b13771f42505055f1372733825153085b805
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15027
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
2022-11-03 14:54:28 +00:00
Shuhei Matsumoto
ffee98ddd9 bdev/error: Consolidate params for injection into a options structure
This will make it easier to add more parameters for error injection.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: Ie5b22c31b5ba9d8c256d369213fa8fb4b985fa26
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15025
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
2022-11-03 14:54:28 +00:00
Shuhei Matsumoto
972013e29d bdev/error: Use custom JSON decoders for bdev_error_injection_error
This is a small clean up.

Use custom JSON decoders for the io_type and error_type parameters
in the bdev_error_injection_error RPC.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I528fe4a31fac7eddb8ec33594b90e107d71693be
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15024
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
2022-11-03 14:54:28 +00:00
Shuhei Matsumoto
95e4ddce51 python/rpc: Change num arg to default for bdev_error_injection_error
If we add default arguments to the bdev_error_injection_error RPC,
it causes a syntax error, non-default argument follows default
argument.

The RPC handler initializes the value of the num argument to be 1
if it is omitted. Hence, the RPC client can make the num argument
from non-default to default by removing =1. Otherwise, we have to
insert new default argument before the num argument.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I0eb009ff3c0c5078f2afc4fa15e9ac97feeb0846
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15023
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
2022-11-03 14:54:28 +00:00
Shuhei Matsumoto
d683d7b792 bdev/part: Modify spdk_bdev_part_submit_request() to use custom completion callback
In the following patches, we will add a feature to inject data
corruption to the error bdev module. For read I/O, we will have
to inject data corruption at completion. However, if we use
spdk_bdev_part_submit_request(), it will not be possible because we
cannot add any custom operation into the completion callback.
To fix the issue, modify spdk_+bdev_part_submit_request() and
rename it to spdk_bdev_part_submit_request_ext().
Fortunately, we can use stored_user_cb in struct spdk_bdev_io.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I46d3c40ea88a3fedd8a8fef6b68ee417c814a7a1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15002
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2022-11-03 14:54:28 +00:00
Changpeng Liu
e6d258a64b include/vhost.h: fix comments
Change-Id: I0c4f52d8c30a325c9f4aa0626812bf9040292984
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15039
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-11-03 14:53:55 +00:00
Changpeng Liu
fabf6a83cc lib/vhost: remove session initialized flag
Session in vhost means an active socket connection from
client(e.g: QEMU or SPDK vhost initiator), but the device
state could be `started` or `stopped` because users may
remove the driver of the device in VM, so in
`foreach_session` we can always call the callback function
without checking the session state, and the callback function
may check the device state if necessary.

Change-Id: Id0fc8c7f6f0915a55a738f0c87ebe6539f7fb2db
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15038
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-11-03 14:53:55 +00:00
Changpeng Liu
9da4e15c5c lib/vhost: start device asynchronously
Now we will start the device(virtio-blk and virtio-scsi) when
there is a valid I/O queue(VRING_KICK message), the backend
device `start_session` callback will ensure this check, so
when processing VRING_KICK messages for each vring, we can
just call `new_device` if `started` is false, and if `started`
is true, it means the device is already started, it's safe
for us to add one more vring even the device is started.

With this change, we don't need to wait for the return value
of `start_session` in synchronous mode, just return is OK.

Fix #2518.

Change-Id: I92ba3d4e5c38422d7697c1d13180a4a48f0dd4cd
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14981
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-11-03 14:53:55 +00:00
Changpeng Liu
23baa6761d lib/vhost: don't restart device multiple times
We will stop/start the device multiple times when a new
vring is added, and also stop/start the device when
set vring's callfd, actually we only need to start
the device after a I/O queue is enabled, DPDK rte_vhost
will not help us to start the device in some scenarios,
so this is controlled in SPDK.

Now we improve the workaround to make it consistent with
vhost-user specification.

For each SET_VRING_KICK message, we will setup the new
added vring, and then we try to start the device.

For each SET_VRING_CALL message, we will add one more
interrupt count, previously this is done when enable
the vring, which is not accurate.

For each GET_VRING_BASE message, we will stop the
device before the first message.

With above changes, we will start/stop the device once,
any new added vrings after starting the device will be
polled in next `vdev_worker` poller.

Change-Id: I5a87c73d34ce7c5f96db7502a68c5fa2cb2e4f74
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14928
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-11-03 14:53:55 +00:00
Changpeng Liu
b7facb30f8 lib/vhost_scsi: don't start device before a valid I/O queue is enabled
Change-Id: I407c62df2117069ad1d8f6aba18cf316a3cf47bf
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14980
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-11-03 14:53:55 +00:00
Changpeng Liu
9cdd1a8a2c lib/vhost: remove vhost_session_used_signal function
`vdev_worker` in vhost-scsi is used to process request queues,
and `vdev_mgmt_worker` is used to process the event and control
queue, so we don't need to call `vhost_session_used_signal` in
`vdev_worker`, just remove it.

Change-Id: I86f3e90890e6defba69b01fec131afe1adad3a49
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14927
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-11-03 14:53:55 +00:00