Commit Graph

58 Commits

Author SHA1 Message Date
paul luse
efa33b8590 lib/accel: add RPC to enable override of opcode to engine
Docs explaining how to use the RPC are in the next patch in the
series.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I7dab8fdbeb90cdfde8b3e916ed6d19930ad36e66
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12848
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-08-10 07:25:29 +00:00
paul luse
44cbea402e lib/accel: Add new RPC to get valid engine info.
The RPC provides a list of initialized engine names along with
that engine's supported operations.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I59f9e5cb7aa51a6193f0bd2ec31e543a56c12f17
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13745
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2022-08-03 07:53:21 +00:00
paul luse
c6ecddcc1c lib/accel: add RPC to get list of OP codes per module
In prep for upcoming patch that will provide an RPC to override
and automatic assignment of an op code to an engine.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I17d4b962fb376a77f97ce051a513679d0fba698e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12829
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-08-03 07:53:21 +00:00
Artur Paszkiewicz
be90ea6e7b FTL: Add bdev_ftl_create and delete rpc definitions
Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Change-Id: I0837028fbe349e8df7f05fb3c9db1f4682f04679
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13301
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-08-02 19:00:42 +00:00
Milosz Linkiewicz
d5d6efd8a8 sma: nvmf/vfiouser device manager implementation
This patch utilize generic sma implementation by adding vfiouser
devices manager. It's allow to expose virtualized block devices to
QEMU instances or other arbitrary processes.

Max device capacity depend on available `pci-bridge`
```yaml
devices:
  - name: 'vfiouser'
    params:
      buses:
        - name: 'pci.spdk.0'
          count: 32
        - name: 'pci.spdk.1'
          count: 32
      qmp_addr: 127.0.0.1
      qmp_port: 9090
```

Signed-off-by: Milosz Linkiewicz <milosz.linkiewicz@intel.com>
Signed-off-by: Sebastian Brzezinka <sebastian.brzezinka@intel.com>
Change-Id: I5ab43f4b877c371fa16a4daf4212ac2686991bd4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13004
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-08-01 08:45:58 +00:00
Boris Glimcher
806744b7c8 sock: Add ktls and tls_version to spdk_sock_impl_opts
Since `sock_impl_opts` was added to `sock_opts`
Can remove `ktls` and `tls_version` from spdk_sock_opts

Example:
  rpc.py sock_impl_set_options -i ssl --enable-ktls
  rpc.py sock_impl_set_options -i ssl --disable-ktls
  rpc.py sock_impl_set_options -i ssl --tls-version=12

  ./build/examples/perf --enable-ktls
  ./build/examples/perf --disable-ktls
  ./build/examples/perf --tls-version=12

Check kTLS statistics here: /proc/net/tls_stat

Change-Id: Icf7ee822bad92fda149710be77feb77fc8d4f163
Signed-off-by: Boris Glimcher <Boris.Glimcher@emc.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13510
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-07-22 06:41:39 +00:00
Krishna Kanth Reddy
6f338d4bf3 bdev : xNVMe BDEV module implementation
This implementation of xNVMe BDEV module supports the char-device / ioctl-over-uring,
along with the "regular" io_uring, libaio, POSIX aio, emulated aio (via threadpools) etc.

Code changes done :
a. Addition of xNVMe submodule to SPDK
b. Modification of RPC scripts to Create / Delete xNVMe BDEVs
c. Implementation of xNVMe BDEV module

Signed-off-by: Krishna Kanth Reddy <krish.reddy@samsung.com>
Change-Id: If814ca1c784124df429d283015a6570068b44f87
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11161
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-07-15 12:52:13 +00:00
Yifan Bian
f656eed6e0 bdev/rbd: add warning message for bdev_rbd_create without specifying -c
This patch will print warning message to help customer use
rpc command bdev_rbd_create correctly.

Fix issue #2189

Change-Id: I734516bef4c5471249dda6463d81be670d975566
Signed-off-by: Yifan Bian <yifan.bian@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13555
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-07-07 07:02:26 +00:00
Jaroslaw Chachulski
642f8b3da5 scripts/rpc: remove default values from bdev functions
Removing default options from parser command to prevent overwrite by
argparse the defaults defined in the .c files.

Signed-off-by: Jaroslaw Chachulski <jaroslawx.chachulski@intel.com>
Change-Id: I39026e0866f2a9aecda6588bb3f91ed87676ddd1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13461
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-07-04 07:23:26 +00:00
Jaroslaw Chachulski
cd79f1a210 scripts/rpc: remove default value from bdev_ocf_create
Removing default options from parser command to prevent overwrite by
argparse the defaults defined in the .c files.

Signed-off-by: Jaroslaw Chachulski <jaroslawx.chachulski@intel.com>
Change-Id: I9c1ba26e2b867b91fd5f7cd91edbff1844f08c79
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12979
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2022-07-04 07:23:26 +00:00
yupeng
4c59c6ac53 blobstore: add RPC for growing blobstore
Add num_md_pages_per_cluster_ratio parameter to bdev_lvol_create_lvstore.
Add bdev_lvol_grow_lvstore rpc.

Signed-off-by: Peng Yu <yupeng0921@gmail.com>
Change-Id: I43a129efbc1a08f9d0df027e80f63ab1393163a2
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9734
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-06-28 17:55:43 +00:00
Jim Harris
488570ebd4 Replace most BSD 3-clause license text with SPDX identifier.
Many open source projects have moved to using SPDX identifiers
to specify license information, reducing the amount of
boilerplate code in every source file.  This patch replaces
the bulk of SPDK .c, .cpp and Makefiles with the BSD-3-Clause
identifier.

Almost all of these files share the exact same license text,
and this patch only modifies the files that contain the
most common license text.  There can be slight variations
because the third clause contains company names - most say
"Intel Corporation", but there are instances for Nvidia,
Samsung, Eideticom and even "the copyright holder".

Used a bash script to automate replacement of the license text
with SPDX identifier which is checked into scripts/spdx.sh.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iaa88ab5e92ea471691dc298cfe41ebfb5d169780
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12904
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: <qun.wan@intel.com>
2022-06-09 07:35:12 +00:00
Jim Harris
8d515e0243 python: remove unneeded deprecated_alias imports
We removed all of the existing deprecated aliases
recently, but didn't remove the associated imports.

Found by lgtm.com.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I2b8526254da26ad4a9b34eb67b155dacfe9a5e57
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12862
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-06-03 16:06:33 +00:00
Konrad Sztyber
5ab0f37fe6 rpc: pass params as **kwargs in vhost_create_blk_controller
This will make it possible for out-of-tree virtio-blk transports to pass
transport-specific options when creating controllers.

Change-Id: I862f0f8096aadb5e47bbe4c1c418aaf06691e1ac
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12818
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-05-26 17:22:30 +00:00
Konrad Sztyber
e661c8e824 rpc: pass params as **kwargs in virtio_blk_create_transport
This will make it possible for out-of-tree virtio-blk transports to pass
their own, transport-specific, set of options.

Change-Id: Ic37b7fb09ca74a744c006d032638d02cc3350471
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12816
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-05-26 17:22:30 +00:00
Tomasz Zawadzki
d39963a9da vhost: remove opts argument from virtio_blk_create_transport
'opts' was a left over from debugging, it should not be present
in the RPC.

virtio-blk transports are supposed to extend the RPC with custom
JSON parameters by existing commands.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ie515db192bd69baec74f7aedb93f29a1d98a0615
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12811
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-05-26 08:19:54 +00:00
wanghailiangx
31513614a7 some remaining rpc: remove support for deprecated RPC names
These were deprecated in 2019, it's time to remove
support for them now.

Change-Id: I7d3804a84851753992af4a3a37b60dc6de0d22cb
Signed-off-by: wanghailiangx <hailiangx.e.wang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12780
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-05-24 07:17:42 +00:00
wanghailiangx
f552937ef4 trace module: remove support for deprecated RPC names
These were deprecated in 2019, it's time to remove
support for them now.

Change-Id: Ie50c7421f991ad0474edba0e0f339180f7afee00
Signed-off-by: wanghailiangx <hailiangx.e.wang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12778
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2022-05-24 07:16:58 +00:00
wanghailiangx
000ee408e7 app module: remove support for deprecated RPC names
These were deprecated in 2019, it's time to remove
support for them now.

Change-Id: Ia09368e426a83274d9c7fc90ed8b0391f4d0b67c
Signed-off-by: wanghailiangx <hailiangx.e.wang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12774
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-05-23 18:58:46 +00:00
Tomasz Zawadzki
b727e804d6 vhost: add virtio_blk abstraction
This patch adds virtio_blk abstraction for custom transports,
with the 'vhost_user_blk' first one being used.

Added spdk_virtio_blk_transport_ops describing the nessecary
callbacks to be implemented by each transport.
Please use SPDK_VIRTIO_BLK_TRANSPORT_REGISTER to register the transport.
Transports can use virtio_blk_process_request() to process the
incoming I/O from their queues.

virtio_blk_create_transport RPC was added to create one of the
registered transports, possibly with custom JSON arguments.

Added 'transport' argument to vhost_create_blk_controller RPC,
to specify which transport should create the controller.
By default the vhost_user_blk transport is used.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ic9d93a6e0f483796eb56b7174a678e41a6ea4808
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9540
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-05-23 17:31:16 +00:00
wanghailiangx
81d3cc1b5a subsystem module: remove support for deprecated RPC names
These were deprecated in 2019, it's time to remove
support for them now.

Change-Id: I56dbaef56ff793e48441219e07dc6b02dda0b470
Signed-off-by: wanghailiangx <hailiangx.e.wang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12777
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-05-23 16:16:55 +00:00
wanghailiangx
465f99e9ff accel module: remove support for deprecated RPC names
These were deprecated in 2019, it's time to remove
support for them now.

Change-Id: I5a715e9b9e991c6febec5e505384728281eee8b7
Signed-off-by: wanghailiangx <hailiangx.e.wang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12773
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-05-23 16:14:41 +00:00
wanghailiangx
23d832a04c vhost: remove support for deprecated RPC names
These were deprecated in 2019, it's time to remove
support for them now.

Change-Id: I33a497fb134320f13606b66ad55fc7b068d011d9
Signed-off-by: wanghailiangx <hailiangx.e.wang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12716
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: GangCao <gang.cao@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-05-23 16:13:42 +00:00
wanghailiangx
405be3b794 notify module: remove support for deprecated RPC names
These were deprecated in 2019, it's time to remove
support for them now.

Change-Id: I477da05a42ca607fbad4d178aa541726197d7c83
Signed-off-by: wanghailiangx <hailiangx.e.wang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12775
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-05-23 16:13:30 +00:00
wanghailiangx
7aa92ad513 bdev module: remove support for deprecated RPC names
These were deprecated in 2019, it's time to remove
support for them now.

Change-Id: I9e203a52877802127df8144e68090d7975f9d200
Signed-off-by: wanghailiangx <hailiangx.e.wang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12772
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-05-23 16:13:00 +00:00
paul luse
b483811ff1 modules/accel/iaa: add IAA accel_fw module
And associated RPC to enable.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I06785bcd8b8957293ad41d13bab556fe62f29fd5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12765
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-05-23 16:04:57 +00:00
wanghailiangx
3ac967baa6 bdev_iscsi: remove support for deprecated RPC names
These were deprecated in 2019, it's time to remove
support for them now.

Change-Id: I25aea510648a55d751db3740b36fb9924d1f52ed
Signed-off-by: wanghailiangx <hailiangx.e.wang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12747
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2022-05-23 07:02:37 +00:00
paul luse
ffef30ae0d modules/accel_dsa: update IDXD references to DSA where it makes sense
IDXD has always been used everywhere but technically it stands for
the driver, not the HW (Intel Data Streaming Accelerator Driver)
where the X comes from "Streaming Accelerator" somehow.  Anyway, the
underlying hardware is just DSA.  It doesn't matter much now but
upcoming patches will add support for a new HW accelerator called
the Intel In-Memory Analytics Accelerator which we'll call IAA and
it will use the same (mostly) device driver (IDXD) as DSA.  So, calling
the HW what it is will lessen confusion when adding IAA support.

This patch just does renaming for the accel_fw module and associated
files (RPC, etc).

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Ib3b1f982cc60359ecfea5dbcbeeb33e4d69aee6a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11984
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-05-23 07:02:21 +00:00
Tomasz Zawadzki
e0516095fc event/vhost: separate vhost subsystem to scsi and blk
Separate out SCSI and BLK vhost subsystems to later add
virtio_blk transport abstraction.

This allows for further changes to the vhost_blk, not
affecting vhost_scsi.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Id1ecfeafeb936809a479a43c321e13f75cb3d5ad
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9539
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2022-05-20 09:20:07 +00:00
Sebastian Brzezinka
56657ad058 sma: vhost-user sma implementation
This patch utilize generic sma implementation by adding vhost-user
devices manager. It's allow to expose virtualized block devices to
QEMU instances or other arbitrary processes.

Max device capacity depend on available `pci-bridge`
```
devices:
  - name: 'vhost_blk'
    params:
      buses:
        - name: 'pci.spdk.0'
          count: 32
        - name: 'pci.spdk.1'
          count: 32
      qmp_addr: 127.0.0.1
      qmp_port: 9090
```

To attach `pci-bridge` at boot time we need to run qemu with this option
```
device pci-bridge,chassis_nr=1,id=pci.spdk.0
device pci-bridge,chassis_nr=2,id=pci.spdk.1
``

Change-Id: Idbe841b12cf59975ff0e6717f8dc905d38379357
Signed-off-by: Sebastian Brzezinka <sebastian.brzezinka@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12207
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Artek Koltun <artsiom.koltun@intel.com>
2022-05-20 09:16:54 +00:00
Milosz Linkiewicz
16cc2ade5d sma: QMP base definitions created
This patch adds python scripts for basic handling of QEMU Machine
Protocol connection for use with vfio_user. Basic low-level connection
and error handling added as extended socket class.
QMPClient: Wrapper class for handling connection with QMP was added.

Protected class method _receive relies on the fact, that QMP always
sends command 'in one line' - each one is always terminated with CRLF.
https://github.com/qemu/qemu/blob/master/docs/interop/qmp-spec.txt

Usage example, command line calls:
1) Without event parameter:
```json
{
    "request": {
        "execute": "device-list-properties",
        "arguments": {
            "typename": "vfiouser-0-1-1"
        }
    }
}
```
2) With event parameter specified. Specifying 'event'
parameter will set script to block wait for occurrence
of such one after a valid execution of specified request:
```json
{
    "event": {
        "event": "DEVICE_DELETED",
        "data": {
            "device": "vfiouser-0-1-1"
        }
    },
    "request": {
        "execute": "device_del",
        "arguments": {
            "id": "vfiouser-0-1-1"
        }
    }
}
```

Signed-off-by: mionsz <milosz.linkiewicz@intel.com>
Change-Id: I7a548f61d31097ad425994b2ea4bac8bc6800b5b
Signed-off-by: Sebastian Brzezinka <sebastian.brzezinka@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11106
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-05-20 09:16:54 +00:00
Konrad Sztyber
943088499b sma: add volume cleanup thread
Since SMA keeps track of attached volumes and a volume might get
disconnected at any time (in which case they're also removed from the
host side), a mechanism is needed to monitor if any of the volumes are
no longer accessible.

This patch implements that mechanism by adding a new thread running in
the background that will periodically (60s by default) send a
bdev_get_bdevs RPC and check that all previously attached volumes are
available.  If any of them are not, it'll remove it and stop the
associated discovery services (if their refcount goes down to zero).
The period can be changed through the `volume_cleanup_period` variable
in the config file.

One important thing to note is that we assume that any intermittent
connection issues are handled internally by the SPDK application and a
bdev is only removed after all reconnection attempts are performed.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I5b9e63698879527d9f79156a0eda1c8bc5e66def
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12699
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-05-20 09:16:54 +00:00
Konrad Sztyber
0c2b10f250 sma: initial support for volume connection via discovery
This patch adds support for connecting volumes via discovery service.
The user specifies a volume UUID/GUID and a list of discovery endpoints,
which are then used to start the discovery service on and attach the
volume to a device.

SMA will keep track of the attached volumes and will also refcount the
connections to discovery services.  So if two volumes are attached using
the same discovery endpoint, it'll be disconnected only after both of
them are detached.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ie8ea50a2a784cf0db8a5953234c6bb2b68685d7c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12413
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-05-20 09:16:54 +00:00
Konrad Sztyber
20e3fb3687 sma: pass config to StorageManagementAgent's __init__
This will make it easier to control various options through config file.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I19ab9c020d536521f5985227bc43965bd04d2707
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12724
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-05-20 09:16:54 +00:00
Konrad Sztyber
4ee3d468c7 sma: handle SIGINT and SIGTERM signals
If the app is killed with either one of these signals, it'll shutdown
gracefully and exit with zero.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I630f9e34c3ccb382c8e1b53d0f589f3ec4d1483b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11727
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-05-20 09:16:54 +00:00
wanghailiangx
c77f17a53e bdev_malloc and bdev_null : remove support for deprecated RPC names
These were deprecated in 2019, it's time to remove
support for them now.

Change-Id: Ic80ce74344b24814dad792cfff6a4791d0430527
Signed-off-by: wanghailiangx <hailiangx.e.wang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12741
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-05-19 13:07:41 +00:00
Konrad Sztyber
c802dbe9fe sma: add device protocol name
A device can now be identified by a protocol and a name.  This allows to
have multiple types of devices providing the same protocol.  The
selection of which device manager to use for specific protocol will be
added in following patches.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ib4a6c323ba6a2a5d36ff958ddc40631fd9329cb1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11683
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-05-18 07:24:06 +00:00
Konrad Sztyber
e5e43f9615 sma: add golang package definitions
It makes it possible to generate golang code from the SMA's protobuf
definitions.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I6f03bc2b602f93ea608597e863cb9e016b68f12e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10279
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-05-18 07:24:06 +00:00
Konrad Sztyber
ae60344eb7 sma: volume attach/detach
This patch implements the Volume(Attach|Detach) methods.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I639f1e7b6d4d5a3e52795f9c8b1ae890407e2375
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10277
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-05-18 07:24:06 +00:00
Konrad Sztyber
2ea8747935 sma: device deletion
This patch implements the generic DeleteDevice method as well as its
NVMe/TCP implementation.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ie7aea6b12c399aec242746e5fae108623a3a161a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10274
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-05-18 07:24:06 +00:00
Konrad Sztyber
509241cef3 configure: add switch for enabling SMA
The SMA requires some extra python packages during the build to generate
the protobuf interface, so this option will select whether it's enabled
or not.  Additionally, the dependencies are added to `pkgdep.sh` in this
patch.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ib0d75c86d4d8575ec1245b219b4e96edb773b366
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10974
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2022-05-18 07:24:06 +00:00
Konrad Sztyber
048fb36ac3 sma: initial Storage Management Agent structures
Storage Management Agent is a gRPC server that provides an abstraction
layer above the SPDK RPC interface.  The interface aims to provide a set
of methods for managing various protocols (e.g. NVMe, virtio-blk) while
hiding the details of a particular transport.

The external API is defined by `lib/python/spdk/sma/proto/sma.proto`
protobuf file.  It defines the generic gRPC service methods and their
requests/responses.  Device-specific messages are defined in their own
files.  This patch also defines messages for creating NVMe and NVMe/TCP
devices.

This patch implements a gRPC service that delegates the work to a
specific device type.  A DeviceManager is a class that implements some
of the methods defined by the service for a given type of devices (e.g.
NVMe, virtio-blk, NVMe/TCP, etc.).  For now, only the RPC for creating a
device is implemented, others are added in subsequent patches.

The series implements the generic calls as well as their NVMe/TCP
implementation.  Support for other devce types could be easily added by
creating a new device manager and defining its protobuf parameter
definition.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I17cde3b31d3514878f1027cfcd112b48848f6123
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10273
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-05-18 07:24:06 +00:00
Konrad Sztyber
e5f9e82291 bdev/nvme: add timeout option to start_discovery
It's now possible to specify a time to wait until a connection to the
discovery controller and the NVM controllers it exposes is made.

Whenever that time is exceeded, a callback is immediately executed.
However, depending on the stage of the discovery process, we might need
to wait a while before actually stopping it (e.g. because a controller
attach is in progress).  That means that a discovery service might be
visible for a while after it timed out.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I2d01837b581e0fa24c8e777730d88d990c94b1d8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12684
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-05-18 07:24:06 +00:00
Shuhei Matsumoto
00d46b80b2 bdev/nvme: Disable automatic failback in multipath mode
By default, failback to the preferred I/O path is done automatically
if it is restored. Some users may want to keep using the backup I/O
path even if the preferred I/O path is restored. In this case,
bdev_nvme_set_preferred_path can be used to do manual failback.

We may be able to clear/fill I/O path cache more strictly but it will
be complicated and have bugs. This patch does the minimal change,
just skips an apparent case.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I78fe5faee6ff04e88ae3d7c6be6da1c20637c912
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12431
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-05-17 12:54:45 +00:00
wanghailiangx
1427379477 nvmf: remove support for deprecated RPC names
These were deprecated in 2019, it's time to remove
support for them now.

Change-Id: I3b75eea83bd7d700d20a6189e8fb6d1f066dc9b4
Signed-off-by: wanghailiangx <hailiangx.e.wang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12603
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-05-12 07:27:45 +00:00
wanghailiangx
d8d29ec0b1 log: remove support for deprecated RPC names
These were deprecated in 2019, it's time to remove
support for them now.

Change-Id: I32dd9960bc397244d8e3d0a384fc8b67e907bf68
Signed-off-by: wanghailiangx <hailiangx.e.wang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12601
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-05-12 07:26:35 +00:00
wanghailiangx
10ba934845 nbd: remove support for deprecated RPC names
These were deprecated in 2019, it's time to remove
support for them now.

Change-Id: I6931e80c836b568dec8989dad2a7be4e112c42b4
Signed-off-by: wanghailiangx <hailiangx.e.wang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12577
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-05-09 23:13:39 +00:00
Jim Harris
47bc58701c bdev: remove support for deprecated RPC names
These were deprecated in 2019, it's time to remove
support for them now.

Fix ocf test script that was still using the
deprecated get_bdevs RPC name - change it to
bdev_get_bdevs.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I7f8caedc250b80503671a0236694181613f63860
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12553
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2022-05-09 23:03:02 +00:00
Jim Harris
febe9002a7 iscsi, scsi: remove support for deprecated RPC names
These were deprecated in 2019, it's time to remove
support for them now.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I2c9918ed0296f644b0728c5106c47d93e3c7ec30
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12552
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2022-05-09 23:03:02 +00:00
Konrad Sztyber
f331ae167b bdev/nvme: add RPC returning information about discovery service
The RPC returns a list of active discovery service connections.  Each
discovery service is described by a name, its trid, and a list of
discovery service trids it refers to.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ifa4b9501dd353e7b4948ad830575a6c94dafd86b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12380
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-05-05 14:08:57 +00:00