Commit Graph

1740 Commits

Author SHA1 Message Date
paul luse
46ac1b8d73 scripts/setup: Update device name IDXD->DSA
In prep for upcoming addition of IAA.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I3044f1e07dbfc65a6a55a7d68bbaff6a43e909d0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12761
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-05-23 07:02:21 +00:00
paul luse
ffef30ae0d modules/accel_dsa: update IDXD references to DSA where it makes sense
IDXD has always been used everywhere but technically it stands for
the driver, not the HW (Intel Data Streaming Accelerator Driver)
where the X comes from "Streaming Accelerator" somehow.  Anyway, the
underlying hardware is just DSA.  It doesn't matter much now but
upcoming patches will add support for a new HW accelerator called
the Intel In-Memory Analytics Accelerator which we'll call IAA and
it will use the same (mostly) device driver (IDXD) as DSA.  So, calling
the HW what it is will lessen confusion when adding IAA support.

This patch just does renaming for the accel_fw module and associated
files (RPC, etc).

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Ib3b1f982cc60359ecfea5dbcbeeb33e4d69aee6a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11984
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-05-23 07:02:21 +00:00
Karol Latecki
bf642c08a4 scripts/nvmf_perf: add explicit listener for discovery subsystem
As introduced in 25bc221cee:
listeners for created subsystems should also be added
to discovery subsystem.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: Id37854ea9a7877eb3e4eb37675d6918ef2931e7e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12676
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2022-05-20 09:52:57 +00:00
John Levon
927f1fd57b libvfio-user: update to use meson
libvfio-user has switched from cmake to meson; update the submodule for
this, and switch over to building with meson.

Signed-off-by: John Levon <john.levon@nutanix.com>
Change-Id: Ia0749590271d0b74260c9118692c21e2a82dde23
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12640
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-05-20 09:21:46 +00:00
Sebastian Brzezinka
56657ad058 sma: vhost-user sma implementation
This patch utilize generic sma implementation by adding vhost-user
devices manager. It's allow to expose virtualized block devices to
QEMU instances or other arbitrary processes.

Max device capacity depend on available `pci-bridge`
```
devices:
  - name: 'vhost_blk'
    params:
      buses:
        - name: 'pci.spdk.0'
          count: 32
        - name: 'pci.spdk.1'
          count: 32
      qmp_addr: 127.0.0.1
      qmp_port: 9090
```

To attach `pci-bridge` at boot time we need to run qemu with this option
```
device pci-bridge,chassis_nr=1,id=pci.spdk.0
device pci-bridge,chassis_nr=2,id=pci.spdk.1
``

Change-Id: Idbe841b12cf59975ff0e6717f8dc905d38379357
Signed-off-by: Sebastian Brzezinka <sebastian.brzezinka@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12207
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Artek Koltun <artsiom.koltun@intel.com>
2022-05-20 09:16:54 +00:00
Konrad Sztyber
943088499b sma: add volume cleanup thread
Since SMA keeps track of attached volumes and a volume might get
disconnected at any time (in which case they're also removed from the
host side), a mechanism is needed to monitor if any of the volumes are
no longer accessible.

This patch implements that mechanism by adding a new thread running in
the background that will periodically (60s by default) send a
bdev_get_bdevs RPC and check that all previously attached volumes are
available.  If any of them are not, it'll remove it and stop the
associated discovery services (if their refcount goes down to zero).
The period can be changed through the `volume_cleanup_period` variable
in the config file.

One important thing to note is that we assume that any intermittent
connection issues are handled internally by the SPDK application and a
bdev is only removed after all reconnection attempts are performed.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I5b9e63698879527d9f79156a0eda1c8bc5e66def
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12699
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-05-20 09:16:54 +00:00
Konrad Sztyber
0c2b10f250 sma: initial support for volume connection via discovery
This patch adds support for connecting volumes via discovery service.
The user specifies a volume UUID/GUID and a list of discovery endpoints,
which are then used to start the discovery service on and attach the
volume to a device.

SMA will keep track of the attached volumes and will also refcount the
connections to discovery services.  So if two volumes are attached using
the same discovery endpoint, it'll be disconnected only after both of
them are detached.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ie8ea50a2a784cf0db8a5953234c6bb2b68685d7c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12413
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-05-20 09:16:54 +00:00
Konrad Sztyber
20e3fb3687 sma: pass config to StorageManagementAgent's __init__
This will make it easier to control various options through config file.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I19ab9c020d536521f5985227bc43965bd04d2707
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12724
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-05-20 09:16:54 +00:00
Konrad Sztyber
b5678ba805 sma: wait until SPDK process is responding
Most of the devices need to send RPCs during initialization (e.g. create
a transport), so we need to take care that they're initialized only
after we are certain that the SPDK process is listening.

The mechanisim is similar to the `waitforlisten` function used in our
test scripts - it sends a series of `rpc_get_methods` calls and times
out after a period of time if no response is received.  For now, the
timeout is hardcoded to 60s.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Iddadc04ad4c486d2894bc40e1a899a9d204400fc
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11802
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-05-20 09:16:54 +00:00
Konrad Sztyber
4ee3d468c7 sma: handle SIGINT and SIGTERM signals
If the app is killed with either one of these signals, it'll shutdown
gracefully and exit with zero.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I630f9e34c3ccb382c8e1b53d0f589f3ec4d1483b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11727
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-05-20 09:16:54 +00:00
Konrad Sztyber
add4a7ce26 sma: register devices based on config file
This allows a user to specify the types of devices to support.  Those
not specified in the config, won't get initialized and will not service
user's requests.

Additionally, each device manager can now receive its own configuration
in its init() method.

The device configuration is structured as a list of objects with two
properties: "name" and "params".  The former identifies a device to
enable, while the latter contains a set of options (if any) specific to
that device manager.  For instance:

```
devices:
  - name: 'nvmf-tcp'
    params:
      max_queue_depth: 256
      io_unit_size: 8192
  - name: 'nvmf-vfiouser'
```

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I967016502ad93c243b3a7af58992bde14c44953c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11713
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-05-20 09:16:54 +00:00
Konrad Sztyber
d2db395987 sma: load configuration from file
This patch adds support for loading configuration from a yaml-formatted
file.  The command-line options takes precedence over those from the
config file.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Iafb5b4aec0a64afe358fce2d6c55361fc4093223
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11712
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
2022-05-20 09:16:54 +00:00
wanghailiangx
c77f17a53e bdev_malloc and bdev_null : remove support for deprecated RPC names
These were deprecated in 2019, it's time to remove
support for them now.

Change-Id: Ic80ce74344b24814dad792cfff6a4791d0430527
Signed-off-by: wanghailiangx <hailiangx.e.wang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12741
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-05-19 13:07:41 +00:00
Jim Harris
95c589e6e9 setup.sh: redirect modprobe msr error messages to /dev/null
We try to modprobe msr so that DPDK can more easily figure
out TSC on x86 systems.  But that just results in a
"Module msr not found" message on non-x86 systems.  setup.sh
still works, but the error message makes it appear
otherwise.  So just redirect the error message to reduce
this confusion.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I71074c7836519b003933551ce51a193070b1bfe6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12671
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
2022-05-19 13:05:05 +00:00
Karol Latecki
e05299f1b2 scripts/nvmf_perf: spread out subsystems evenly
Spread out used block devices evenly between
used initiators and then between initiators NIC
interfaces, so that in result each initiator
connects to the same number of subsystems.

Previous version had problems spreading block
devices / subsystems between interfaces in case
division remainder was not 0.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I045dc9313b0613685fc997f821fe66238026a645
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12732
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
2022-05-19 08:29:12 +00:00
Karol Latecki
b6103a58a1 scripts/nvmf_perf: save initiator NIC info in target object
Save Initiator addressing information in Target object.
This will be needed later to properly spread created
subsystems between initiators and NICs.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I9e5545b7847bb16f3ce3d0ee56038003fda74e6d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12731
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
2022-05-19 08:29:12 +00:00
Sebastian Brzezinka
d045916f65 sma: loading out-of-tree plugins in sma-client
Change-Id: I7813a31060af7b2f9aea1b0a0d99fb699670509a
Signed-off-by: Sebastian Brzezinka <sebastian.brzezinka@intel.com>
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11667
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-05-18 07:24:06 +00:00
Konrad Sztyber
93a20e79ec sma: loading out-of-tree plugins
It's now possible to register SMA device managers defined in a module
outside of the regular directory.  To do that, a global variable called
`devices` containing a list of device manager classes needs to be defined
in a module that's added to SMA_PLUGINS environment variable.

For example:
```
$ cat /path/to/plugins/external_devices/__init__.py
import spdk.sma as sma

class MyDeviceManager1(sma.DeviceManager):
	pass

class MyDeviceManager2(sma.DeviceManager):
	pass

devices = [MyDeviceManager1, MyDeviceManager2]

$ SMA_PLUGINS=external_devices scripts/sma.py
```

Multiple plugins can be specified by separating them with a colon, e.g.:
```
SMA_PLUGINS=plugin1:plugin2
```
Of course, the location at which these modules reside must be in
PYTHONPATH.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I2bb4cfea5191710f97d66abe3a21e4deacc6338a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11412
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-05-18 07:24:06 +00:00
sberbz
f0f65d240d sma: make SMA parameters configurable
It's now possible to specify the gRPC interface listen address and the
SPDK-RPC socket path.

Change-Id: I957cf4ca4e03eac6f4f6f7ffab5ee95c5b6f7521
Signed-off-by: Sebastian Brzezinka <sebastian.brzezinka@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10979
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-05-18 07:24:06 +00:00
Konrad Sztyber
509241cef3 configure: add switch for enabling SMA
The SMA requires some extra python packages during the build to generate
the protobuf interface, so this option will select whether it's enabled
or not.  Additionally, the dependencies are added to `pkgdep.sh` in this
patch.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ib0d75c86d4d8575ec1245b219b4e96edb773b366
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10974
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2022-05-18 07:24:06 +00:00
Konrad Sztyber
048fb36ac3 sma: initial Storage Management Agent structures
Storage Management Agent is a gRPC server that provides an abstraction
layer above the SPDK RPC interface.  The interface aims to provide a set
of methods for managing various protocols (e.g. NVMe, virtio-blk) while
hiding the details of a particular transport.

The external API is defined by `lib/python/spdk/sma/proto/sma.proto`
protobuf file.  It defines the generic gRPC service methods and their
requests/responses.  Device-specific messages are defined in their own
files.  This patch also defines messages for creating NVMe and NVMe/TCP
devices.

This patch implements a gRPC service that delegates the work to a
specific device type.  A DeviceManager is a class that implements some
of the methods defined by the service for a given type of devices (e.g.
NVMe, virtio-blk, NVMe/TCP, etc.).  For now, only the RPC for creating a
device is implemented, others are added in subsequent patches.

The series implements the generic calls as well as their NVMe/TCP
implementation.  Support for other devce types could be easily added by
creating a new device manager and defining its protobuf parameter
definition.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I17cde3b31d3514878f1027cfcd112b48848f6123
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10273
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-05-18 07:24:06 +00:00
Konrad Sztyber
e5f9e82291 bdev/nvme: add timeout option to start_discovery
It's now possible to specify a time to wait until a connection to the
discovery controller and the NVM controllers it exposes is made.

Whenever that time is exceeded, a callback is immediately executed.
However, depending on the stage of the discovery process, we might need
to wait a while before actually stopping it (e.g. because a controller
attach is in progress).  That means that a discovery service might be
visible for a while after it timed out.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I2d01837b581e0fa24c8e777730d88d990c94b1d8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12684
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-05-18 07:24:06 +00:00
Shuhei Matsumoto
00d46b80b2 bdev/nvme: Disable automatic failback in multipath mode
By default, failback to the preferred I/O path is done automatically
if it is restored. Some users may want to keep using the backup I/O
path even if the preferred I/O path is restored. In this case,
bdev_nvme_set_preferred_path can be used to do manual failback.

We may be able to clear/fill I/O path cache more strictly but it will
be complicated and have bugs. This patch does the minimal change,
just skips an apparent case.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I78fe5faee6ff04e88ae3d7c6be6da1c20637c912
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12431
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-05-17 12:54:45 +00:00
Karol Latecki
2874d8ff8f scripts/nvmf_perf: reload ice driver after ADQ tests
Always reload ICE driver after tests with ADQ enabled.
This is to reset tc qdisc and filter settings and make
sure they don't affect future tests.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I70b94ef236741fe81870954f9901e02b9b5aadb9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12622
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-05-13 07:26:25 +00:00
wanghailiangx
1427379477 nvmf: remove support for deprecated RPC names
These were deprecated in 2019, it's time to remove
support for them now.

Change-Id: I3b75eea83bd7d700d20a6189e8fb6d1f066dc9b4
Signed-off-by: wanghailiangx <hailiangx.e.wang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12603
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-05-12 07:27:45 +00:00
Karol Latecki
5da326ee4d scripts/nvmf_perf: save SAR CPU util to different file
Use different file to save calculated CPU utlization
and do not modify original SAR output.

Change-Id: I415242a5610f1c2a43a038baafa8dcc4f9e8fae2
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12526
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-05-12 07:27:39 +00:00
Karol Latecki
0eec19d740 scripts/nvmf_perf: move sar file name into function
Move sar output file generation into measure_sar
function and pass a file name prefix instead.
More than one file will be created by the function
based on that prefix.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I671887d4d232821c0de19eb338150d30201be204
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12525
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Kamil Godzwon <kamilx.godzwon@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-05-12 07:27:39 +00:00
Karol Latecki
6acfe37999 scripts/nvmf_perf: modify SAR log messages
Re-phrase "waiting" message a bit and add another
message notifying that measurements are actually starting.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I55d8ed761bf45a590322cffc5b776a44774977dd
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12522
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-05-12 07:27:39 +00:00
wanghailiangx
d8d29ec0b1 log: remove support for deprecated RPC names
These were deprecated in 2019, it's time to remove
support for them now.

Change-Id: I32dd9960bc397244d8e3d0a384fc8b67e907bf68
Signed-off-by: wanghailiangx <hailiangx.e.wang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12601
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-05-12 07:26:35 +00:00
Monica Kenguva
91751e1e40 test/nvmf: Multipath testing with ANA states support
Signed-off-by: Monica Kenguva <monica.kenguva@intel.com>
Change-Id: Ide3af4fd027450eeac523e7078ba41ab0572ed80
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10197
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
2022-05-12 07:22:23 +00:00
luzhipeng
6735751844 scripts/rpc.py: remove redundant code
This call bdev_rbd_resize twice, so delete one.

Signed-off-by: luzhipeng <luzhipeng@cestc.cn>
Change-Id: I5fa07a3b66284436759983405acb32b511051c0c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12570
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
2022-05-11 09:41:27 +00:00
wanghailiangx
10ba934845 nbd: remove support for deprecated RPC names
These were deprecated in 2019, it's time to remove
support for them now.

Change-Id: I6931e80c836b568dec8989dad2a7be4e112c42b4
Signed-off-by: wanghailiangx <hailiangx.e.wang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12577
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-05-09 23:13:39 +00:00
Jim Harris
47bc58701c bdev: remove support for deprecated RPC names
These were deprecated in 2019, it's time to remove
support for them now.

Fix ocf test script that was still using the
deprecated get_bdevs RPC name - change it to
bdev_get_bdevs.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I7f8caedc250b80503671a0236694181613f63860
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12553
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2022-05-09 23:03:02 +00:00
Jim Harris
febe9002a7 iscsi, scsi: remove support for deprecated RPC names
These were deprecated in 2019, it's time to remove
support for them now.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I2c9918ed0296f644b0728c5106c47d93e3c7ec30
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12552
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2022-05-09 23:03:02 +00:00
Kamil Godzwon
9cdadbea09 scripts/pkgdep: add support for Rocky Linux
Add Rocky Linux as another supported OS
Adjust package dependencies

Signed-off-by: Kamil Godzwon <kamilx.godzwon@intel.com>
Change-Id: Ide4f5075a224e9eb37cb99ae5e138ebf91e4ad5b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11994
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Michal Berger <michallinuxstuff@gmail.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
2022-05-09 09:14:36 +00:00
Boris Glimcher
8ccb3c10c0 pkgdep/centos: Add openssl11-devel for centos7
By default centos7 uses openssl-devel version 1.0.2k
The newer version 1.1.1k is available with EPEL repo

Install and use newer version 1.1 for centos7

The new 1.1 version is needed for TLSv1.3 support.

Change-Id: I489f38dd519276741ab0bb2cf7bc2d368a16387f
Signed-off-by: Boris Glimcher <Boris.Glimcher@emc.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12511
Reviewed-by: Michal Berger <michallinuxstuff@gmail.com>
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-05-06 08:12:42 +00:00
Konrad Sztyber
f331ae167b bdev/nvme: add RPC returning information about discovery service
The RPC returns a list of active discovery service connections.  Each
discovery service is described by a name, its trid, and a list of
discovery service trids it refers to.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ifa4b9501dd353e7b4948ad830575a6c94dafd86b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12380
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-05-05 14:08:57 +00:00
Shuhei Matsumoto
8f9b977504 bdev/nvme: Add active/active policy for multipath mode
The NVMe bdev module supported active-passive policy for multipath mode
first. By this patch, the NVMe bdev module supports active-active policy
for multipath node next. Following the Linux kernel native NVMe multipath,
the NVMe bdev module supports round robin algorithm for active-active
policy.

The multipath policy, active-passive or active-active, is managed per
nvme_bdev. The multipath policy is copied to all corresponding
nvme_bdev_channels.

Different from active-passive, active-active caches even non_optimized
path to provide load balance across multiple paths.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: Ie18b24db60d3da1ce2f83725b6cd3079f628f95b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12001
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2022-05-05 07:11:24 +00:00
Shuhei Matsumoto
22b77a3c80 bdev/nvme: Set preferred I/O path in multipath mode
If we specify a preferred path manually for each NVMe bdev, we will
be able to realize a simple static load balancing and make the failover
more controllable in the multipath mode.

The idea is to move I/O path to the NVMe-oF controller to the head of
the list and then clear the I/O path cache for each NVMe bdev channel.
We can set the I/O path to the I/O path cache directly but it must be
conditional and make the code very complex. Hence, let find_io_path() do
that.

However, a NVMe bdev channel may be acquired after setting the preferred
path. To cover such case, sort the nvme_ns list of the NVMe bdev too.

This feature supports only multipath mode. The NVMe bdev module supports
failover mode too. However, to support the latter, the new RPC needs to
have trid as parameters and the code and the usage will be come very
complex. Add a note for such limitation.

To verify one by one exactly, add unit test.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: Ia51c74f530d6d7dc1f73d5b65f854967363e76b0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12262
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: <tanl12@chinatelecom.cn>
Reviewed-by: GangCao <gang.cao@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-05-05 07:11:24 +00:00
Richael Zhuang
9bff828f99 sock: introduce dynamic zerocopy according to data size
MSG_ZEROCOPY is not always effective as mentioned in
https://www.kernel.org/doc/html/v4.15/networking/msg_zerocopy.html.

Currently in spdk, once we enable sendmsg zerocopy, then all data
transferred through _sock_flush are sent with zerocopy, and vice
versa. Here dynamic zerocopy is introduced to allow data sent with
MSG_ZEROCOPY or not according to its size, which can be enabled by
setting "enable_dynamic_zerocopy" as true.

Test with 16 P4610 NVMe SSD, 2 initiators, target's and initiators'
configurations are the same as spdk report:
https://ci.spdk.io/download/performance-reports/SPDK_tcp_perf_report_2104.pdf

For posix socket, rw_percent=0(randwrite), it has 1.9%~8.3% performance boost
tested with target 1~40 cpu cores and qdepth=128,256,512. And it has no obvious
influence when read percentage is greater than 50%.

For uring socket, rw_percent=0(randwrite), it has 1.8%~7.9% performance boost
tested with target 1~40 cpu cores and qdepth=128,256,512. And it still has
1%~7% improvement when read percentage is greater than 50%.

The following is part of the detailed data.

posix:
qdepth=128
rw_percent      0             |           30
cpu  origin  thisPatch  opt   | origin  thisPatch opt
1	286.5	298.5	4.19%		 307	304.15	-0.93%
4	1042.5	1107	6.19%		1135.5	1136	0.04%
8	1952.5	2058	5.40%		2170.5	2170.5	0.00%
12	2658.5	2879	8.29%		3042	3046	0.13%
16	3247.5	3460.5	6.56%		3793.5	3775	-0.49%
24	4232.5	4459.5	5.36%		4614.5	4756.5	3.08%
32	4810	5095	5.93%		4488	4845	7.95%
40	5306.5	5435	2.42%		4427.5	4902	10.72%

qdepth=512
rw_percent      0             |           30
cpu  origin  thisPatch  opt   | origin  thisPatch opt
1    275	 287	4.36%		294.4	295.45	0.36%
4	 979	1041	6.33%		1073	1083.5	0.98%
8	1822.5	1914.5	5.05%		2030.5	2018.5	-0.59%
12	2441	2598.5	6.45%		2808.5	2779.5	-1.03%
16	2920.5	3109.5	6.47%		3455	3411.5	-1.26%
24	3709	3972.5	7.10%		4483.5	4502.5	0.42%
32	4225.5	4532.5	7.27%		4463.5	4733	6.04%
40	4790.5	4884.5	1.96%		4427	4904.5	10.79%

uring:
qdepth=128
rw_percent      0             |           30
cpu  origin  thisPatch  opt   | origin  thisPatch opt
1	270.5	287.5	6.28%		295.75	304.75	3.04%
4	1018.5	1089.5	6.97%		1119.5	1156.5	3.31%
8	1907	2055	7.76%		2127	2211.5	3.97%
12	2614	2801	7.15%		2982.5	3061.5	2.65%
16	3169.5	3420	7.90%		3654.5	3781.5	3.48%
24	4109.5	4414	7.41%		4691.5	4750.5	1.26%
32	4752.5	4908	3.27%		4494	4825.5	7.38%
40	5233.5	5327	1.79%		4374.5	4891	11.81%

qdepth=512
rw_percent      0             |           30
cpu  origin  thisPatch  opt   | origin  thisPatch opt
1	259.95	 276	6.17%		286.65	294.8	2.84%
4	955 	1021	6.91%		1070.5	1100	2.76%
8	1772	1903.5	7.42%		1992.5	2077.5	4.27%
12	2380.5	2543.5	6.85%		2752.5	2860	3.91%
16	2920.5	3099	6.11%		3391.5	3540	4.38%
24	3697	3912	5.82%		4401	4637	5.36%
32	4256.5	4454.5	4.65%		4516	4777	5.78%
40	4707	4968.5	5.56%		4400.5	4933	12.10%

Signed-off-by: Richael Zhuang <richael.zhuang@arm.com>
Change-Id: I730dcf89ed2bf3efe91586421a89045fc11c81f0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12210
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-04-28 07:29:28 +00:00
Jim Harris
6e2ba81ee7 check_format.sh: only use specific v3.1.0 shfmt version
Our CI explicitly uses v3.1.0 shfmt.  Other versions,
including newer versions, may suggest a lot of script
changes that are different than what v3.1.0 reports.
So change check_format.sh to *only* run shfmt
checking if the version is exactly v3.1.0.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I788961faaae7992cf1369abf1e54f4075bd2521c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12309
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-04-27 07:36:14 +00:00
Kamil Godzwon
ac7b9b7c66 pkgdep/centos: fix centos routines for installing the packages
Centos7: ndb package version update - version 3.14-2
of the nbd package is no longer available in
epel repository.
Updated for version 3.24-1.

Centos8: repository URL change - old ceph
repository has Centos version mismatch.

Centos8: no need to install NBD package. It is available
on the system by default.

Signed-off-by: Kamil Godzwon <kamilx.godzwon@intel.com>
Change-Id: Ifae0833822b270127b31fe95c14f6803f688bfc6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12156
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-04-26 07:47:27 +00:00
Shuhei Matsumoto
2a6a64485c bdev/nvme: Add bdev_nvme_get_io_paths RPC to monitor I/O path states
Add an new RPC bdev_nvme_get_io_paths to query all active I/O paths.

One io_path belongs to One nvme_bdev_channel.
Each nvme_bdev_channel is associated with one nvme_bdev.

If the RPC bdev_nvme_get_io_paths has a bdev name as a parameter
it can use spdk_for_each_channel() simply for the corresponding
nvme_bdev.

However, users will want to know I/O paths of all nvme_bdevs like
the RPC bdev_get_bdevs.

One io_path has one nvme_qpair. One nvme_qpair belongs to one
nvme_poll_group. By relying on these relationships, the RPC
bdev_nvme_get_io_paths traverses all nvme_poll_groups by using
spdk_for_each_channel() to g_bdev_nvme_ctrlrs.

The RPC bdev_nvme_get_io_paths has two modes, display all or
the specified NVMe bdev's active I/O paths.

The specified bdev name is used just for comparison and empty
array is returned if no matched io_path is found.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I4a0dbf3ef7aaa9a7b7345fc03dc493cc6d37bc99
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12146
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-04-22 09:44:57 +00:00
Andreas Economides
3b047a6162 nvmf/vfio-user: support shadow doorbells
As per the NVMe specification, a host can identify two areas of guest
memory: one of which is used for the host-written doorbells, and one of
which contains event indexes. The host writes to the shadow doorbell
area, but also writes to the controller's BAR0 doorbell area if the
corresponding event index is crossed by the update. This avoids many
mmio exits in interrupt mode, where BAR0 doorbells are not directly
mapped into the guest VM, with greatly improved performance.

This isn't a useful feature in BAR0 doorbells are mapped into the VM, so
we explicitly disable support in that case.

NB: the Windows NVMe driver doesn't yet support this feature.

Although the specification says that the admin queues should also engage
in this behaviour, in practice, no VM does, so have to include some
hacks to account for this.

Co-authored-by: John Levon <john.levon@nutanix.com>
Signed-off-by: John Levon <john.levon@nutanix.com>
Change-Id: I0646b234d31fbbf9a6b85572042c6cdaf8366659
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11492
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2022-04-21 08:12:29 +00:00
zhangduan
31db7b139b nvme_tcp: set transport_ack_timeout to ack_timeout
The value of ack_timeout is calculated according to
the formula 2^(transport_ack_timeout) msec.

Signed-off-by: zhangduan <zhangd28@chinatelecom.cn>
Change-Id: I5a938635d70693ddd405fa5907555bb745b4df0f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12215
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-04-20 08:21:42 +00:00
Jim Harris
d710397125 scripts/bpf: remove bdev_nvme_destroy_qpair probe
This probe was removed from the bdev nvme module during
some recent refactoring.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I31508fb67651693e1e9c4a656e2e7111c14a8415
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12002
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-04-20 08:20:02 +00:00
Konrad Sztyber
6e8e184bca scripts: fix PYTHONPATH references to scripts/ directory
After 7610bc38d, python modules are now located in the python/
directory.  That patch also updated PYTHONPATH assignment in most of the
scripts, but one of the perf script and some docs/messages were missed.

Change-Id: Iff3588526f2a382e599f5734d38fc32c15a7bf56
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12223
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-04-18 17:22:09 +00:00
Karol Latecki
20ec3ed66f scripts/vagrant: add missing Fedora 35 image name
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I910e1b9d23673fd5876bca60371051b4ac888ba4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12228
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
2022-04-13 14:41:49 +00:00
Karol Latecki
712c904a2f scripts/vagrant: change default Fedora 34 box
Use "generic" Fedora34 from Roboxes, as we did for
all other Fedora Vagrant boxes.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I208264c50a067b26c9ec5da2250f27881ba8431d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12227
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
2022-04-13 14:41:49 +00:00
Karol Latecki
a38452186c scripts/vagrant: Add Fedora 36 as an option
Change-Id: Ie25212a3cd4a523b8fc6c1bbee6e03b8f05a8583
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12226
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
2022-04-13 14:41:49 +00:00
Pawel Piatek
9842b62c63 scripts/vhost: fix pylint W0611,W0612
Fix pylint findings:
- W0612 unused-variable
- W0611 unused-import

Signed-off-by: Pawel Piatek <pawelx.piatek@intel.com>
Change-Id: Id5ab32c53c59ee62a34897c29ee816cce533c929
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12138
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-04-12 07:24:15 +00:00
Pawel Piatek
20955a4938 scripts/nvmf_perf: fix pylint R0201
Fix pylint findings:
- R0201 Method could be a function

Signed-off-by: Pawel Piatek <pawelx.piatek@intel.com>
Change-Id: I1b9141b9ab150e20fdcd63510b484c6be514efb5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12137
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-04-12 07:24:15 +00:00
Pawel Piatek
6d98816767 scripts/nvmf_perf: fix pylint W0107,W0612,R1725
Fix pylint findings:
- W0107 unnecessary-pass
- W0612 unused-variable
- R1725 super-with-arguments

Signed-off-by: Pawel Piatek <pawelx.piatek@intel.com>
Change-Id: Ib9f35173d89b00e6d164900dfd3645ed5b5754ed
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12136
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-04-12 07:24:15 +00:00
Michal Berger
9af7c30e8f scripts/setup: Skip devices which have any valid data present
This is done to make sure we don't miss more complex setups where
target devices are not mounted but still hold some valid data that
shouldn't be touched in any way.

Also, adjust function names so they clearly indicate what is being
checked.

Signed-off-by: Michal Berger <michallinuxstuff@gmail.com>
Change-Id: Ibb0f1f21de68009a2f8f1faf4595a07ae527da35
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11111
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
2022-04-06 07:21:17 +00:00
Michal Berger
bb4657c7c6 scripts/setup: Check if target block device is hold by other devices
Change-Id: Ie072e00b980095c5c794a189c0886e128b7377c9
Signed-off-by: Michal Berger <michallinuxstuff@gmail.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11109
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-04-06 07:21:17 +00:00
Konrad Sztyber
7610bc38dc scripts: move python modules to python directory
Up until now, importing an SPDK RPC python module was just a matter of
`import rpc`.  It's fine until there's another module called `rpc`
installed on the system, in which case it's impossible to import both of
them.  Therefore, to avoid this problem, all of the modules were moved
to a separate directory under the "spdk" namespace.

The decision to move to a location under a separate directory was
motivated by the fact that a directory called scripts/spdk would look
pretty confusing.  Moreover, it should make it also easier to package
these scripts as a python package.

Other than moving the packages, all of the imports were updated to
reflect these changes.  Files under python now use relative imports,
while those under scripts/ use the "spdk" namespace and have their
PYTHONPATH extended with python directory.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ib43dee73921d590a551dd83885e22870e72451cf
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9692
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-04-05 14:40:47 +00:00
yupeng
64eebbd132 bdev/raid: Add concat module
The concat module can combine multiple underlying bdevs to a single
bdev. It is a special raid level. You can add a new bdev to the end of
the concat bdev, then the concat bdev size is increased, and it won't
change the layout of the exist data. This is the major difference
between concat and raid0. If you add a new underling device to raid0,
the whole data layout will be changed. So the concat bdev is extentable.

Change-Id: Ibbeeaf0606ff79b595320c597a5605ab9e4e13c4
Signed-off-by: Peng Yu <yupeng0921@gmail.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11070
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-04-05 07:39:00 +00:00
Samir Raval
b0aba3fcd5 setup.sh: Speed up the VMD device unbind by running in parallel.
If a VMD device present, add message to communicate to the user to use next steps.
Allow for DRIVER_OVERRIDE="none" to just unbind the driver without binding it to any other.

Fixes #2423

Signed-off-by: Samir Raval <samir.raval@intel.com>
Change-Id: Ifef6ed50dd619ce7629eabf458edd54e6bb22fa4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12035
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-04-01 20:49:43 +00:00
Jim Harris
0bd7ace836 bdev/nvme: add wait_for_attach param to discovery RPC
Setting this optional parameter to true makes the
RPC completion wait until the attach for all
discovered NVM subsystems have completed.

This is especially useful for fio or bdevperf, to
ensure that all of the namespaces are actually
available before testing.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Icf04a122052f72e263a26b3c7582c81eac32a487
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12044
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-04-01 10:03:45 +00:00
Rafal Stefanowski
494b1ba8e6 ocf/rpc: Add RPC method to manage sequential cutoff
Introduce bdev_ocf_set_seqcutoff RPC method for
changing OCF sequential cutoff policy and parameters.

Signed-off-by: Rafal Stefanowski <rafal.stefanowski@intel.com>
Change-Id: I509644115402c00c4a026c1c37e887c8fc90289f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8459
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-04-01 08:31:32 +00:00
Kamil Godzwon
a06aaacbe7 scripts/vagrant: added support for Rocky Linux
Signed-off-by: Kamil Godzwon <kamilx.godzwon@intel.com>
Change-Id: I3b33167e5600047a3e478a7c380fbb08c90c3941
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12084
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
2022-03-31 09:35:49 +00:00
Boris Glimcher
637f1a101c scripts/rpc_http_proxy: Handled chunked data
'Content-Length' is not always available
specifically when using 'chunked' Transfer-Encoding

https://en.wikipedia.org/wiki/Chunked_transfer_encoding

This patch allows to handle chunked data transfer
and construct the request from peices.

The example of how to send chunked data:

> curl -k --user spdkuser:spdkpass -X POST -H "Content-Type: application/json" \
                                           -H "Transfer-Encoding: chunked" \
                                   -d '{"id": 1, "method": "bdev_get_bdevs"}' \
                                   http://127.0.0.1:9009/

vs

> curl -k --user spdkuser:spdkpass -X POST -H "Content-Type: application/json" \
                                   -d '{"id": 1, "method": "bdev_get_bdevs"}' \
                                   http://127.0.0.1:9009/

Signed-off-by: Boris Glimcher <Boris.Glimcher@emc.com>
Change-Id: I6cf565d6639aa31898585f005d75785c43204552
Signed-off-by: Boris Glimcher <Boris.Glimcher@emc.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12082
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-03-31 09:35:32 +00:00
Konrad Sztyber
fa649869b9 bdev: add timeout option to bdev_get_bdevs RPC
This opption allows the bdev_get_bdevs RPC to block until a bdev with
specified name appears.  It can be useful, when a bdev is created
asynchronously and the exact moment at which it appears is not known.
For instance, with a discovery service, a bdev is created when a
namespace on a remote NVMeoF target is added, but it's not possible to
specify when that happens exactly.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I6c1f974fba445376ca9d45aac2639202547410cc
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11960
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-03-30 08:02:08 +00:00
Jim Harris
13cffc5e76 bdev_nvme: add timeout parameters to start_discovery RPC
These parameters will be used for any controller created
by the discovery service.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I221b791f38b9c5797ba084c647a98b82c102a121
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11942
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
2022-03-28 17:10:04 +00:00
Rui Chang
dd17459701 nvmf/vfio-user: Add adaptive irq feature for vfio-user transport
In vfio-user transport, whenever one IO is completed, it will trigger
an interrupt to guest machine. This cost quite some overhead. This patch
adds an adaptive irq feature to reduce interrupt overhead and boost
performance.

Signed-off-by: Rui Chang <rui.chang@arm.com>
Change-Id: I585be072231a934fa2e4fdf2439405de95151381
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11840
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-03-25 08:18:59 +00:00
John Levon
70f3606bfc add USDT tracepoints for interrupt wakeups
With these we can write a simple bpftrace script to identify work being done,
and in particular what woke us up from sleeping.

Signed-off-by: John Levon <john.levon@nutanix.com>
Change-Id: I8997d847625ee4558092dbd753e6fc1b17beca92
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9424
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-03-21 11:07:05 +00:00
Ben Walker
f0bf4e75f5 idxd: Eliminate configs
SPDK has settled on what the optimal DSA configuration is, so let's
always use it.

Change-Id: I24b9b717709d553789285198b1aa391f4d7f0445
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11532
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2022-03-21 11:05:28 +00:00
Krzysztof Karas
ca8e6d2361 event: use implemented set/get methods in RPC calls
Make use of code implemented in previous patches in the series
to get and set dynamic scheduler values.

Modifiy app.py and rpc.py to accomodate new changes and allow
user to specify scheduler parameters in the RPC calls.

Change-Id: I6173aefbf1d774b91b80ee5bce67eea80a2ab23d
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11449
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-03-16 08:19:26 +00:00
Shuhei Matsumoto
0fba8dc8cb bdev/nvme: I/O error resiliency can be configured by global options
Add three options for I/O error resiliency to spdk_nvme_bdev_opts.
Then the RPC bdev_nvme_set_options can configure these.
These can be overridden if these are given by the RPC bdev_nvme_attach_controller.

Change-Id: If3ee23aeef8b7585fe0fb5ec4695df5866fc1e74
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11830
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-03-15 09:05:09 +00:00
Michal Berger
eb8655b2fe setup.sh: Make sure driver is known upon reset
In case collect_driver() returns immediately due to missing modalias
attr, an empty $driver will be passed to check_for_driver() causing
grep inside to fail. Further false-positive then leads to call to
linux_bind_driver() as it's not able to locate proper paths for
given driver.

To avoid this, make sure collect_driver() always attempts to fallback
to a driver setup.sh supports for a target device.

Signed-off-by: Michal Berger <michallinuxstuff@gmail.com>
Change-Id: Id5631a731910f60b63c6afb3a412575bb69d784a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11747
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-03-09 08:02:41 +00:00
Konrad Sztyber
f6dbaa0686 pkgdep: add python3-devel dependency
The following patch will need it to expand the %{python3_sitelib} macros
when building the RPMs.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I2782b228066fa0524d675c8d964217a7c9d11e36
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11656
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2022-03-08 07:54:23 +00:00
Jim Harris
06ef6757a9 rpc.py: add 'discovery' nqn shortcut for nvmf listener RPCs
Now that users need to explicitly add a listener for the discovery
subsystem, make that a bit easier when using rpc.py.  Instead of
having to type out nqn.2014-08.org.nvmexpress.discovery, allow
user to just specify 'discovery' as the NQN and rpc.py will
convert it to the discovery NQN before sending the RPC.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I4854d4f072f1758fdd6b37a4c3685e2a2d015caa
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11540
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
2022-02-28 11:06:16 +00:00
Karol Latecki
160af0934e scripts/nvmf_perf: add max_queue_depth param for transport layer
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I80e1fdb8b0a3b944ab6efdd21187cef07b685c29
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11271
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-02-17 17:08:49 +00:00
Michal Berger
7dc4e55849 scripts/vagrant: Don't define type while running create_nvme_img.sh
This was left out by the ea71df4f48.

Signed-off-by: Michal Berger <michallinuxstuff@gmail.com>
Change-Id: Ibd341eaa7e5a0b1a409a9efad081555728b7f880
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10941
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-02-15 14:48:03 +00:00
Jim Harris
5d2d253125 nvmf: remove deprecated conn_sched parameter
This parameter was ignored, and was a parameter to the
nvmf_set_config RPC.

For reference, this was deprecated in June 2020, commit
c37cf9fb.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I013f4d7cf874e7e26a8a1d299fdf9d8fa05da580
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11544
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2022-02-15 14:38:37 +00:00
Jim Harris
e8fdde5235 nvmf: remove deprecated max_qpairs_per_ctrlr parameter
This was a parameter on the nvmf_create_transport
RPC, and was replaced with max_io_qpairs_per_ctrlr to
reduce confusion on whether this number included the
admin queue or not.

nvmf_vhost test was using this deprecated parameter.
Change it to use -m (max_io_qpairs_per_ctrlr)
instead. '-p 4' would have been evaluated as 1 admin
queue + 3 I/O queues, but it's likely the intent
was for 4 I/O queues.  This is a perfect example of
why this parameter was deprecated.

For reference, this was deprecated in June 2020,
commit 1551197db.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I4364fc0a76c9993b376932b6eea243d7cefca9cd
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11543
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2022-02-15 14:38:37 +00:00
Jim Harris
70a6e3ac22 rpc.py: ignore lines starting with # on stdin
rpc.py users can pipe RPC calls through stdin, which
reduces overhead compared to calling rpc.py
separately for each RPC.

It is common to put these RPC calls in a file, and
then pipe that file to rpc.py.  To allow commenting
out RPC calls when doing debugging, have rpc.py
ignore any lines which begin with '#', allowing users
to comment out RPC calls (or even add comments
explaining the RPC calls) with a '#' character.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I8d9c6ac95dd5864c16e4d69ba80f81799068e808
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11506
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-02-14 15:29:39 +00:00
Karol Latecki
d4b8944881 test/nvmf: enable fio verify_backlog option
Using time_based tasks with do_verify=1 and rw=write
results in verification not being run. Basically
writes consume all run time and there's no time
left for verify:

fio: verification read phase will never start
because write phase uses all of runtime

Using verify_backlog we ask fio to verify after every
N blocks written.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I4c1571f4fad893e47cc5fea2ad6b4e2a7d8cc5ef
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11376
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-02-04 21:58:18 +00:00
Michael Haeuptle
7a0c901a4c bdev/nvme: RPCs for adding/removing error injections
Provides RPCs for the qpair error injection APIs to bdev_nvme.
These RPCs are useful in testing NVMeoF/NVMe behavior for various
error scenarios in production.

Signed-off-by: Michael Haeuptle <michael.haeuptle@hpe.com>
Change-Id: I0db7995d7a712d4f8a60e643d564faa6908c3a55
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10992
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-01-31 09:51:07 +00:00
Alexey Marchuk
2ccaf2acfa bdev/nvme: Add transport_ack_timeout to bdev_nvme_set_options RPC
It may take a long time to detect network transport error
when e.g. port is removed on remote target. This timeout
depends on 2 parameters - retry_count and ack_timeout.
bdev_nvme_set_options supports configuration of retry_count
but transport_ack_timeout is missed. Note: this parameter
is used by RDMA transport only.

Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Change-Id: I7c3090dc8e4078f64d444e2392a9e0a6ecdc31c0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11175
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: <tanl12@chinatelecom.cn>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-01-31 09:44:28 +00:00
Tomasz Zawadzki
7d97822f17 vagrant: preallocate nvme images by default
Added preallocation option for raw NVMe disk images.
It matches qemu-img option for raw images.

By default raw images are created with "off" option,
resulting in increased size of the disk image as it is used.

Using "falloc" option preallocates the full size of the disk,
but does not write zeroes to it, compared to "full" option.

This will make sure that in CI, a device used during test
is never exhaused due to lack of host space.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ie1c30909dc87552a6d6c385d0ad80905ce68ff9d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11243
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-01-27 19:36:38 +00:00
Tomasz Zawadzki
c733cd682f scripts/common: set special name to test partitions
SPDK test script can fail during any point of the
execution. To keep using the same devices
between test runs in CI, those devices have to
be cleaned between runs.

To select device for tests or to clean before tests,
the partition table and lack of mount points is used.
SPDK GPT partitions were always considered not in use.
Meanwhile Linux GPT were not, even if lacking a mount
point.

Some of the SPDK tests create Linux GPT partitions,
which might remain after a test run.

Linux GPT partitions created during tests now contain
"SPDK_TEST" as part of their partition name.
This allows marking such block devices as 'not in use',
to be cleaned and used in further SPDK test.

fixes #2345

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I0098776239da9b0c5dcc748625485284e4863525
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11220
Reviewed-by: Michal Berger <michallinuxstuff@gmail.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-01-26 19:07:43 +00:00
Michal Berger
61dadf29d1 scripts/spdk-gpt: Fix path lookup of the target device
Signed-off-by: Michal Berger <michallinuxstuff@gmail.com>
Change-Id: I5e1cdb75b8b4421b2a38fe315f75e11d8ea5de23
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11110
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-01-25 08:13:40 +00:00
shuochen0311
b635d19a26 aio: add aio bdev rescan feature
Signed-off-by: shuochen0311 <shuo.chen@databricks.com>
Change-Id: I7f2788640a56d1e1bc8b7b311622628e8a6be56e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11084
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
2022-01-25 08:13:28 +00:00
John Kariuki
b7e4940952 scripts/nvmf_perf: enable idxd in nvmf_tgt
Add configuration options to enable using IDXD
for data digest during NVMe-oF performance testing.

Change-Id: I54d443c35c67e0331e3d6ec4aecb577fc7f544c7
Signed-off-by: John Kariuki <John.K.Kariuki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10282
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2022-01-21 07:24:57 +00:00
Krzysztof Karas
30ea7ecc6f bdev/nvme: implement additional dtrace probes
Add more dtrace probes to help with identifying issues
in production.

Change-Id: I8fb621a15c5e33ae94d75b4fc31135e2635dcfce
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10561
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-01-20 21:12:56 +00:00
Maciej Wawryk
9cf27937df scripts/nvmf: Distribute all disk between all NICs
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I72fc8cb42f57d3272a08a6cefc58b2fa5eb26419
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11094
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
2022-01-20 15:46:17 +00:00
Karol Latecki
741b0839ef scripts/nvmf_perf: don't rely on __del__ for destroying objects
As described in Python documentation:
It is not guaranteed that __del__() methods are called for
objects that still exist when the interpreter exits.

Because of that class instance objects were not always destroyed
after tests.

Change-Id: I10f05a480cde37d274d6475efdf47d094cab7cb9
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11136
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-01-20 00:36:16 +00:00
Jim Harris
e4ef7b3910 setup.sh: fix incorrect help message
Reported-by: Chris Demia on Slack
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ib77cee03fbd9eaf600f2c0b74f5b6ceae9624600
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11108
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-01-19 08:55:46 +00:00
Shuhei Matsumoto
80e81273e2 bdev/nvme: Do not use ctrlr for I/O submission if reconnect failed repeatedly
If ctrlr_loss_timeout_sec is set to -1, reconnect is tried repeatedly
indefinitely, and I/Os continue to be queued.

This patch adds another option fast_io_fail_timeout_sec, a flag
fast_io_fail_timedout to nvme_ctrlr.

If the time fast_io_fail_timeout_sec passed after starting reset,
set fast_io_fail_timedout to true not to use the path for I/O submission.

fast_io_fail_timeout_sec is initialized to zero as same as
ctrlr_loss_timeout_sec and reconnect_delay_sec.

The name of the parameter follows the famous DM-multipath, its fast_io_fail_tmo.

Change-Id: Ib870cf8e2fd29300c47f1df69617776f4e67bd8c
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10301
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-01-17 14:25:15 +00:00
Shuhei Matsumoto
ae4e54fdc3 bdev/nvme: Retry reconnecting ctrlr after seconds if reset failed
Previously reconnect retry was not controlled and was repeated indefinitely.

This patch adds two options, ctrlr_loss_timeout_sec and reconnect_delay_sec,
to nvme_ctrlr and add reset_start_tsc, reconnect_is_delayed, and
reconnect_delay_timer to nvme_ctrlr to control reconnect retry.

Both of ctrlr_loss_timeout_sec and reconnect_delay_sec are initialized to
zero. This means reconnect is not throttled as we did before this patch.

A few more changes are added.

Change nvme_io_path_is_failed() to return false if reset is throttled
even if nvme_ctrlr is reseting or is to be reconnected.

spdk_nvme_ctrlr_reconnect_poll_async() may continue returning -EAGAIN
infinitely. To check out such exceptional case, use ctrlr_loss_timeout_sec.

Not only ctrlr reset but also non-multipath ctrlr failover is controlled.
So we need to include path failover into ctrlr reconnect.

When the active path is removed and switched to one of the alternative paths,
if ctrlr reconnect is scheduled, connecting to the alternative path is left
to the scheduled reconnect.

If reset or reconnect ctrlr is failed and the retry is scheduled,
switch the active path to one of alternative paths.

Restore unit test cases removed in the previous patches.

Change-Id: Idec636c4eced39eb47ff4ef6fde72d6fd9fe4f85
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10128
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
2022-01-17 14:25:15 +00:00
Tan Long
20c8a3b8db bdev/rbd: Add key_file to the rbd_register_cluster RPC
In project practice, config_file and key_file are often used to connect
to a rados cluster, config_file includes "mon_host" and other rados
configurations like "rbd_cache", and key_file includes the secret key
and the access authority to each pool for current user. This patch adds
key_file option, user can specify config_file and key_file or only config_param
to connect rados cluster. This will make it much more flexible for users with
his/her convenience.

Signed-off-by: Tan Long <tanl12@chinatelecom.cn>
Change-Id: I6b49aad70b578bdeb3ac8ea9ca0fcbd931582025
Signed-off-by: Tan Long <tanl12@chinatelecom.cn>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10485
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: GangCao <gang.cao@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-01-17 09:44:56 +00:00
Krzysztof Karas
aa499efdb6 trace_rpc.c: add support for enabling individual traces
Add support to enable individual traces through rpc commands
and modify jsonrpc.md to describe the changes.

Change-Id: I3664fc28f1c25a76eade4cff0a0ab1870172f8de
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10518
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-01-14 11:01:15 +00:00
Alexey Marchuk
41714ffa73 rpc: Description of mlx5_pci pmd in bdev_compress_set_pmd
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Change-Id: I85f86cb149e7241c4c0da754f43e4f9321b11bce
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11073
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
2022-01-14 08:36:10 +00:00
Krzysztof Karas
d11601e87f traces: add dtrace probes around qpair/controller/subsystem
Add dtrace probes aroung qpair/controller/subsystem management
to help with debugging issue #2055.

Change-Id: I0b981bffadee3fe4172ad6916c059bf357959dde
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10237
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2022-01-13 08:44:54 +00:00
Jim Harris
932ee64b8f bdev/nvme: add bdev_nvme_stop_discovery RPC
This RPC will stop the specified discovery service,
including detaching from any controllers that were
attached as part of that discovery service.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I9222876457fc45e1acde680a7bd1925917c22308
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10832
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
2022-01-12 08:20:23 +00:00
Jim Harris
b68f2eeb0b bdev_nvme: add bdev_nvme_start_discovery RPC
This patch adds the framework for a discovery
service in the bdev/nvme module.

Users can specify an IP/port of a discovery service.
The bdev/nvme module will connect to a discovery
controller, get the discovery log page, and then
register for AERs.  It will connect to each
subsystem specified in the initial log page.
AER completions will trigger fetching the log
page again, at which point new subsystems will
be connected to, or removed subsystems will be
detached.

This patch does the following:
* Adds the new start_discovery RPC
* Connects to the discovery controller
* Gets the discovery log page
* Registers for AERs
* Detach from discovery controllers at shutdown

Subsequent patches in this series will:
* Connect to subsystems listed in discovery log page
* Detach from subsystems that were listed in earlier
  discovery log pages but subsequently removed
* Add a stop_discovery RPC

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I54bfa896a48c5619676f156b5ea9f2d1f886c72f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10694
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
2022-01-10 15:23:39 +00:00
Konrad Sztyber
aa1d039836 nvmf: zero-copy enable flag in transport opts
It makes it possible for the user to specify whether a transport should
try to use zero-copy to execute requests when possible.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I40a92b0d7a6707f4c9292795f380846acb227200
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10780
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-01-06 18:53:42 +00:00
Jim Harris
63ff27f910 check_format.sh: handle spdk_ functions that are moved
If a function prefixed with spdk_ was moved between
files, the check_format.sh naming convention checks
would fail.  This is because it thinks the function
was added, but doesn't see it getting added to the
header file, since the header file wasn't touched
by the commit.

So resolve this by doing the defined/removed checks
on a per-library basis, rather than per-file.  The
checks already handled the case where functions
were moved within a file, and that will all work
the same now that we check on a per-lib basis.

Fixes issue #2307.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: If85a1e9c3cd349b701a10531726e814b60fba26d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10967
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: wanghailiang <hailiangx.e.wang@intel.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-01-06 16:05:53 +00:00
Jim Harris
76159db955 bpf: use map in nvmf_path.bt
Modify nvmf_path.bt to collect the (ip, port)
tuples in a map, and then print the map every
second. This generates a lot less output than
printing the path for every single I/O.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I8f655ed7a4558735f51ddf1169134fbf6950cc1c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10616
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-01-05 21:44:24 +00:00