Commit Graph

919 Commits

Author SHA1 Message Date
Kamil Godzwon
3f922a68f3 doc/distributions: update doc according to the recent changes
Added Rocky Linux 8 as a new supported OS (CI)

Fedora 35 has been deployed to all VM-hosts and used instead
of Fedora 34 as a default image.

Signed-off-by: Kamil Godzwon <kamilx.godzwon@intel.com>
Change-Id: I91926b442b188e9bf7cee32e5b957862c5ca878f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13563
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
2022-07-22 06:44:32 +00:00
Boris Glimcher
806744b7c8 sock: Add ktls and tls_version to spdk_sock_impl_opts
Since `sock_impl_opts` was added to `sock_opts`
Can remove `ktls` and `tls_version` from spdk_sock_opts

Example:
  rpc.py sock_impl_set_options -i ssl --enable-ktls
  rpc.py sock_impl_set_options -i ssl --disable-ktls
  rpc.py sock_impl_set_options -i ssl --tls-version=12

  ./build/examples/perf --enable-ktls
  ./build/examples/perf --disable-ktls
  ./build/examples/perf --tls-version=12

Check kTLS statistics here: /proc/net/tls_stat

Change-Id: Icf7ee822bad92fda149710be77feb77fc8d4f163
Signed-off-by: Boris Glimcher <Boris.Glimcher@emc.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13510
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-07-22 06:41:39 +00:00
Alexey Marchuk
36589e1e44 bdev_nvme_rpc: Update bdev_nvme_get_io_paths RPC
This RPC prints only cntlid, to identify a controller
it is needed to issue bdev_get_bdevs and find more info
using cntlid. Printing the controller's trid helps to
identify the controller faster.

Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: I97325b822528ef9e71afbe2ff1c30b3bce2ae203
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13655
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2022-07-19 08:05:07 +00:00
Krishna Kanth Reddy
6f338d4bf3 bdev : xNVMe BDEV module implementation
This implementation of xNVMe BDEV module supports the char-device / ioctl-over-uring,
along with the "regular" io_uring, libaio, POSIX aio, emulated aio (via threadpools) etc.

Code changes done :
a. Addition of xNVMe submodule to SPDK
b. Modification of RPC scripts to Create / Delete xNVMe BDEVs
c. Implementation of xNVMe BDEV module

Signed-off-by: Krishna Kanth Reddy <krish.reddy@samsung.com>
Change-Id: If814ca1c784124df429d283015a6570068b44f87
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11161
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-07-15 12:52:13 +00:00
Karol Latecki
474da9ea4f doc: SPDK NVMe Bdev 22.05 Performance Report
Add doc section with URL to 22.05 SPDK NVMe Bdev report.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: If022b4df2b44b3e71a3d1db40b76ddb3eaa4f1ad
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13617
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Michal Berger <michallinuxstuff@gmail.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-07-12 07:26:35 +00:00
yupeng
4c59c6ac53 blobstore: add RPC for growing blobstore
Add num_md_pages_per_cluster_ratio parameter to bdev_lvol_create_lvstore.
Add bdev_lvol_grow_lvstore rpc.

Signed-off-by: Peng Yu <yupeng0921@gmail.com>
Change-Id: I43a129efbc1a08f9d0df027e80f63ab1393163a2
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9734
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-06-28 17:55:43 +00:00
yupeng
1f0b8df7b0 blobstore: implement spdk_bs_grow and bdev_lvol_grow_lvstore RPC
The bdev_lvol_grow_lvstore will grow the lvstore size if the undering
bdev size is increased. It invokes spdk_bs_grow internally. The
spdk_bs_grow will extend the used_clusters bitmap. If there is no
enough space resereved for the used_clusters bitmap, the api will
fail. The reserved space was calculated according to the num_md_pages
at blobstore creating time.

Signed-off-by: Peng Yu <yupeng0921@gmail.com>
Change-Id: If6e8c0794dbe4eaa7042acf5031de58138ce7bca
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9730
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-06-28 17:55:43 +00:00
Jim Harris
13fa72c6d5 doc: cleanup bdevperf.py
- Simply use 'bdevperf' as the page title.  This matches
  spdkcli and spdk_top.
- a few additional miscellaneous changes

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ied4d30ab8c2dd5dfdaf0ae2cb22459be63cdd24c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13058
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2022-06-20 09:56:52 +00:00
Jim Harris
26184ad62a doc: cleanup spdkcli.py
- spdkcli.py is no longer be considered experimental.
- consistently refer to it as 'spdkcli'
- a few other miscellaneous changes

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I1f5c75961e4c1741d0573f2041b4b023ee627357
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13057
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2022-06-20 09:56:52 +00:00
Richael Zhuang
3a82f5fa22 doc: update sock documentation
add zerocopy_threshold related information in jsonrpc.md

Change-Id: I9fbea29f39c532af41b33da81420c0c6d3200031
Signed-off-by: Richael Zhuang <richael.zhuang@arm.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12585
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-05-31 07:11:43 +00:00
paul luse
e65ab145ec doc/idxd: update idxd docs regarding configs
A while back config changes were made and the doc wasn't updated.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Id6046a97dc623a9af00de115995c928fc9fbe240
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12759
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-05-23 19:51:23 +00:00
Tomasz Zawadzki
b727e804d6 vhost: add virtio_blk abstraction
This patch adds virtio_blk abstraction for custom transports,
with the 'vhost_user_blk' first one being used.

Added spdk_virtio_blk_transport_ops describing the nessecary
callbacks to be implemented by each transport.
Please use SPDK_VIRTIO_BLK_TRANSPORT_REGISTER to register the transport.
Transports can use virtio_blk_process_request() to process the
incoming I/O from their queues.

virtio_blk_create_transport RPC was added to create one of the
registered transports, possibly with custom JSON arguments.

Added 'transport' argument to vhost_create_blk_controller RPC,
to specify which transport should create the controller.
By default the vhost_user_blk transport is used.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ic9d93a6e0f483796eb56b7174a678e41a6ea4808
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9540
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-05-23 17:31:16 +00:00
wanghailiangx
7aa92ad513 bdev module: remove support for deprecated RPC names
These were deprecated in 2019, it's time to remove
support for them now.

Change-Id: I9e203a52877802127df8144e68090d7975f9d200
Signed-off-by: wanghailiangx <hailiangx.e.wang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12772
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-05-23 16:13:00 +00:00
paul luse
b483811ff1 modules/accel/iaa: add IAA accel_fw module
And associated RPC to enable.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I06785bcd8b8957293ad41d13bab556fe62f29fd5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12765
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-05-23 16:04:57 +00:00
paul luse
76fae14976 lib/idxd: update names from IDXD->DSA where it makes sense
In prep for upcoming addition of IAA.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I47c5880aac37da9a38d6af6e52a51cefbfec91b9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12762
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-05-23 07:02:21 +00:00
paul luse
ffef30ae0d modules/accel_dsa: update IDXD references to DSA where it makes sense
IDXD has always been used everywhere but technically it stands for
the driver, not the HW (Intel Data Streaming Accelerator Driver)
where the X comes from "Streaming Accelerator" somehow.  Anyway, the
underlying hardware is just DSA.  It doesn't matter much now but
upcoming patches will add support for a new HW accelerator called
the Intel In-Memory Analytics Accelerator which we'll call IAA and
it will use the same (mostly) device driver (IDXD) as DSA.  So, calling
the HW what it is will lessen confusion when adding IAA support.

This patch just does renaming for the accel_fw module and associated
files (RPC, etc).

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Ib3b1f982cc60359ecfea5dbcbeeb33e4d69aee6a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11984
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-05-23 07:02:21 +00:00
paul luse
59af87d4b0 doc/accel_fw: add comment on use of DSA kernel mode
Comment about device claiming was missing from docs.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I6f5fe07b8fee9678a2ef588b295ce9ebdcdd1037
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12702
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-05-20 09:19:41 +00:00
Konrad Sztyber
79a3ac8863 doc: describe storage management agent
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ibe602571413531f151136a68ad4bdcd0a6be32e9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12117
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-05-20 09:16:54 +00:00
Konrad Sztyber
e5f9e82291 bdev/nvme: add timeout option to start_discovery
It's now possible to specify a time to wait until a connection to the
discovery controller and the NVM controllers it exposes is made.

Whenever that time is exceeded, a callback is immediately executed.
However, depending on the stage of the discovery process, we might need
to wait a while before actually stopping it (e.g. because a controller
attach is in progress).  That means that a discovery service might be
visible for a while after it timed out.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I2d01837b581e0fa24c8e777730d88d990c94b1d8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12684
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-05-18 07:24:06 +00:00
Shuhei Matsumoto
00d46b80b2 bdev/nvme: Disable automatic failback in multipath mode
By default, failback to the preferred I/O path is done automatically
if it is restored. Some users may want to keep using the backup I/O
path even if the preferred I/O path is restored. In this case,
bdev_nvme_set_preferred_path can be used to do manual failback.

We may be able to clear/fill I/O path cache more strictly but it will
be complicated and have bugs. This patch does the minimal change,
just skips an apparent case.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I78fe5faee6ff04e88ae3d7c6be6da1c20637c912
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12431
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-05-17 12:54:45 +00:00
Shuhei Matsumoto
b0ab552452 doc: Multipath document
Add an independent document for NVMe multipath. Then link it from
user_guide.md and bdev.md.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I6035a3c73727b4140d2a78fb10a1e6f94c0bdaa3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12297
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: GangCao <gang.cao@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-05-17 12:54:45 +00:00
Konrad Sztyber
f331ae167b bdev/nvme: add RPC returning information about discovery service
The RPC returns a list of active discovery service connections.  Each
discovery service is described by a name, its trid, and a list of
discovery service trids it refers to.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ifa4b9501dd353e7b4948ad830575a6c94dafd86b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12380
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-05-05 14:08:57 +00:00
paul luse
d71a91bb74 docs: update accel framework docs
Add info about engine assignment, remove content that is no longer
accurate.  Updated some other text for clarifications.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I7dcbb754a58b2bfd61d59dd383bf679e2ea4247d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12116
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-05-05 07:11:32 +00:00
Shuhei Matsumoto
8f9b977504 bdev/nvme: Add active/active policy for multipath mode
The NVMe bdev module supported active-passive policy for multipath mode
first. By this patch, the NVMe bdev module supports active-active policy
for multipath node next. Following the Linux kernel native NVMe multipath,
the NVMe bdev module supports round robin algorithm for active-active
policy.

The multipath policy, active-passive or active-active, is managed per
nvme_bdev. The multipath policy is copied to all corresponding
nvme_bdev_channels.

Different from active-passive, active-active caches even non_optimized
path to provide load balance across multiple paths.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: Ie18b24db60d3da1ce2f83725b6cd3079f628f95b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12001
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2022-05-05 07:11:24 +00:00
Shuhei Matsumoto
22b77a3c80 bdev/nvme: Set preferred I/O path in multipath mode
If we specify a preferred path manually for each NVMe bdev, we will
be able to realize a simple static load balancing and make the failover
more controllable in the multipath mode.

The idea is to move I/O path to the NVMe-oF controller to the head of
the list and then clear the I/O path cache for each NVMe bdev channel.
We can set the I/O path to the I/O path cache directly but it must be
conditional and make the code very complex. Hence, let find_io_path() do
that.

However, a NVMe bdev channel may be acquired after setting the preferred
path. To cover such case, sort the nvme_ns list of the NVMe bdev too.

This feature supports only multipath mode. The NVMe bdev module supports
failover mode too. However, to support the latter, the new RPC needs to
have trid as parameters and the code and the usage will be come very
complex. Add a note for such limitation.

To verify one by one exactly, add unit test.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: Ia51c74f530d6d7dc1f73d5b65f854967363e76b0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12262
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: <tanl12@chinatelecom.cn>
Reviewed-by: GangCao <gang.cao@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-05-05 07:11:24 +00:00
Kamil Godzwon
164ab112ab doc/doxyfile: remove timestamps generation
Doc files should be updated only if the file
content (actual documentation) will change.

Doxygen adds timestamps to the footer
of every doc file in spdk/doc directory.
Every day, when the date is changed,
all pages get a new date and it triggers
automatic commits push for all >500 doc pages
with a new date in the footer.

This change removes automatically generated
timestamps for each spdk/doc file footer.
In fact, these timestamps are not visible
directly on the page running in the browser anyway.

Signed-off-by: Kamil Godzwon <kamilx.godzwon@intel.com>
Change-Id: I6039a250613f109e255fd61a55ba9e3b197e7b10
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12326
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-04-29 07:28:36 +00:00
Shuhei Matsumoto
2a6a64485c bdev/nvme: Add bdev_nvme_get_io_paths RPC to monitor I/O path states
Add an new RPC bdev_nvme_get_io_paths to query all active I/O paths.

One io_path belongs to One nvme_bdev_channel.
Each nvme_bdev_channel is associated with one nvme_bdev.

If the RPC bdev_nvme_get_io_paths has a bdev name as a parameter
it can use spdk_for_each_channel() simply for the corresponding
nvme_bdev.

However, users will want to know I/O paths of all nvme_bdevs like
the RPC bdev_get_bdevs.

One io_path has one nvme_qpair. One nvme_qpair belongs to one
nvme_poll_group. By relying on these relationships, the RPC
bdev_nvme_get_io_paths traverses all nvme_poll_groups by using
spdk_for_each_channel() to g_bdev_nvme_ctrlrs.

The RPC bdev_nvme_get_io_paths has two modes, display all or
the specified NVMe bdev's active I/O paths.

The specified bdev name is used just for comparison and empty
array is returned if no matched io_path is found.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I4a0dbf3ef7aaa9a7b7345fc03dc493cc6d37bc99
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12146
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-04-22 09:44:57 +00:00
Shuhei Matsumoto
2730f5cac0 bdev/nvme: Add cntlid to bdev_get_bdevs and bdev_nvme_get_controllers RPCs
NVMe bdev name already includes the name of the NVMe bdev controller and
the NSID. CNTLID will be a good ID to identify a namespace from a NVMe
bdev when multipath is configured. However, the query RPCs,
bdev_get_bdevs and bdev_nvme_get_controllers had not returned such
information.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I2f2e355ff13f69ced616be803a3152c838cdc980
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12276
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
2022-04-22 09:44:57 +00:00
Andreas Economides
3b047a6162 nvmf/vfio-user: support shadow doorbells
As per the NVMe specification, a host can identify two areas of guest
memory: one of which is used for the host-written doorbells, and one of
which contains event indexes. The host writes to the shadow doorbell
area, but also writes to the controller's BAR0 doorbell area if the
corresponding event index is crossed by the update. This avoids many
mmio exits in interrupt mode, where BAR0 doorbells are not directly
mapped into the guest VM, with greatly improved performance.

This isn't a useful feature in BAR0 doorbells are mapped into the VM, so
we explicitly disable support in that case.

NB: the Windows NVMe driver doesn't yet support this feature.

Although the specification says that the admin queues should also engage
in this behaviour, in practice, no VM does, so have to include some
hacks to account for this.

Co-authored-by: John Levon <john.levon@nutanix.com>
Signed-off-by: John Levon <john.levon@nutanix.com>
Change-Id: I0646b234d31fbbf9a6b85572042c6cdaf8366659
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11492
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2022-04-21 08:12:29 +00:00
zhangduan
31db7b139b nvme_tcp: set transport_ack_timeout to ack_timeout
The value of ack_timeout is calculated according to
the formula 2^(transport_ack_timeout) msec.

Signed-off-by: zhangduan <zhangd28@chinatelecom.cn>
Change-Id: I5a938635d70693ddd405fa5907555bb745b4df0f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12215
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-04-20 08:21:42 +00:00
Karol Latecki
247390e81a doc: add SPDK 22.01 Bdev PCIe Gen4 Performance Report link
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I0f68a8b95e0220eed588fc4e5b408e21e655541b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12197
Reviewed-by: Kamil Godzwon <kamilx.godzwon@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Michal Berger <michallinuxstuff@gmail.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
2022-04-08 06:47:39 +00:00
Konrad Sztyber
7610bc38dc scripts: move python modules to python directory
Up until now, importing an SPDK RPC python module was just a matter of
`import rpc`.  It's fine until there's another module called `rpc`
installed on the system, in which case it's impossible to import both of
them.  Therefore, to avoid this problem, all of the modules were moved
to a separate directory under the "spdk" namespace.

The decision to move to a location under a separate directory was
motivated by the fact that a directory called scripts/spdk would look
pretty confusing.  Moreover, it should make it also easier to package
these scripts as a python package.

Other than moving the packages, all of the imports were updated to
reflect these changes.  Files under python now use relative imports,
while those under scripts/ use the "spdk" namespace and have their
PYTHONPATH extended with python directory.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ib43dee73921d590a551dd83885e22870e72451cf
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9692
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-04-05 14:40:47 +00:00
Shuhei Matsumoto
428b17a0a8 bdev: Add spdk_for_each_bdev/bdev_leaf for clean up and further improvements
To execute a callback function for each registered bdev or unclaimed
bdev, add new public APIs, spdk_for_each_bdev() and
spdk_for_each_bdev_leaf().

These functions are safe for race conditions by opening before and
closing after executing the provided callback function.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I59b702ffec7b4fc5e9779de5a3a75d44922b829b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12088
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-04-05 07:30:47 +00:00
Jim Harris
0bd7ace836 bdev/nvme: add wait_for_attach param to discovery RPC
Setting this optional parameter to true makes the
RPC completion wait until the attach for all
discovered NVM subsystems have completed.

This is especially useful for fio or bdevperf, to
ensure that all of the namespaces are actually
available before testing.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Icf04a122052f72e263a26b3c7582c81eac32a487
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12044
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-04-01 10:03:45 +00:00
Rafal Stefanowski
494b1ba8e6 ocf/rpc: Add RPC method to manage sequential cutoff
Introduce bdev_ocf_set_seqcutoff RPC method for
changing OCF sequential cutoff policy and parameters.

Signed-off-by: Rafal Stefanowski <rafal.stefanowski@intel.com>
Change-Id: I509644115402c00c4a026c1c37e887c8fc90289f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8459
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-04-01 08:31:32 +00:00
Konrad Sztyber
fa649869b9 bdev: add timeout option to bdev_get_bdevs RPC
This opption allows the bdev_get_bdevs RPC to block until a bdev with
specified name appears.  It can be useful, when a bdev is created
asynchronously and the exact moment at which it appears is not known.
For instance, with a discovery service, a bdev is created when a
namespace on a remote NVMeoF target is added, but it's not possible to
specify when that happens exactly.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I6c1f974fba445376ca9d45aac2639202547410cc
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11960
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-03-30 08:02:08 +00:00
Jim Harris
13cffc5e76 bdev_nvme: add timeout parameters to start_discovery RPC
These parameters will be used for any controller created
by the discovery service.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I221b791f38b9c5797ba084c647a98b82c102a121
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11942
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
2022-03-28 17:10:04 +00:00
Rui Chang
dd17459701 nvmf/vfio-user: Add adaptive irq feature for vfio-user transport
In vfio-user transport, whenever one IO is completed, it will trigger
an interrupt to guest machine. This cost quite some overhead. This patch
adds an adaptive irq feature to reduce interrupt overhead and boost
performance.

Signed-off-by: Rui Chang <rui.chang@arm.com>
Change-Id: I585be072231a934fa2e4fdf2439405de95151381
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11840
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-03-25 08:18:59 +00:00
Yuriy Umanets
1434e255ef bdev/crypto: Critical fix about using binary keys
- Added hexlify() and unhexlify() for key and key2. This is required
  for keys that contain zero and non-ascii characters. Since binary
  keys may contain zero character, strlen(key) cannot be used and
  key_size and key2_size are used instead. Non-asci chars are not
  allowed in json and using hexlified keys fixes this issue as well.
- Updated documentation to clearly state that hexlified keys must
  be used.
- Updated test scripts.

Signed-off-by: Yuriy Umanets <yumanets@nvidia.com>
Change-Id: I3fce7839f7eaa67d0307071eba80b4cea472d731
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11891
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2022-03-24 09:21:35 +00:00
Yuriy Umanets
45f24aebe7 bdev/crypto: MLX5 AES_XTS general support
- General MLX5 crypto support.
- Unit-tests MLX5 crypto support.
- Documentation update to list the MLX5 driver as supported,
  enumerate the cipher algorithms and provide some configuration
  hints.

Signed-off-by: Yuriy Umanets <yumanets@nvidia.com>
Change-Id: I0da1f49f4acd068d75a4d8633f84fe626d774431
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11630
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
2022-03-24 09:21:35 +00:00
Ben Walker
f0bf4e75f5 idxd: Eliminate configs
SPDK has settled on what the optimal DSA configuration is, so let's
always use it.

Change-Id: I24b9b717709d553789285198b1aa391f4d7f0445
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11532
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2022-03-21 11:05:28 +00:00
Krzysztof Karas
835494b53a docs: update scheduler documentation
Updated scheduler documentation with information
about thread and core workloads, scheduling process
and dynamic scheduler parameters.

Added descriptions of new dynamic scheduler parameters
in jsonrpc.md.

Change-Id: I567efe21c425c13c16235d7976aad6ae961381bb
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11886
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-03-17 17:41:51 +00:00
Shuhei Matsumoto
0fba8dc8cb bdev/nvme: I/O error resiliency can be configured by global options
Add three options for I/O error resiliency to spdk_nvme_bdev_opts.
Then the RPC bdev_nvme_set_options can configure these.
These can be overridden if these are given by the RPC bdev_nvme_attach_controller.

Change-Id: If3ee23aeef8b7585fe0fb5ec4695df5866fc1e74
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11830
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-03-15 09:05:09 +00:00
John Levon
ba4ffda671 lib/nvme: correct typo in transport stats
"doorbell" not "doobell"

Signed-off-by: John Levon <john.levon@nutanix.com>
Change-Id: I9261559576e72a09b63fbc984ae0ec2a2572eb2c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11841
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-03-09 08:03:42 +00:00
Liu Xiaodong
03b75034ae doc/container: display spdk-docker-suite in doc
Link spdk-docker-suite to get SPDK users realized that
there is already a good docker usage example.

Change-Id: Ia154061d894b77ba544d83aaffbb94c998f54bde
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11851
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-03-09 08:01:38 +00:00
Liu Xiaodong
a4c5411bdf doc/container: put containerizing SPDK to front
Recently, received questions around containerizing SPDK app
or app built on top of SPDK. Containerizing SPDK is more
useful for general SPDK users.
So move it to front in order to avoid SPDK users get confused
or stuck on the Kata one.

Change-Id: I3dc510d0a7bafa63be4559810990d28c5f9d7de5
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11850
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-03-09 08:01:38 +00:00
Karol Latecki
fa3ea8aad6 doc: add SPDK NVME-oF RDMA 22.01 Performance Report link
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I8e498d554f571937a839beb54e0b444cffdc8ec6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11743
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2022-02-28 11:07:51 +00:00
Karol Latecki
5489035988 doc: add SPDK Bdev 22.01 Performance Report link
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I0c7b9b373d429b93b0899324b263aaa9ed41e3b3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11741
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2022-02-28 11:07:51 +00:00
Alexey Marchuk
adde7ea533 doc/nvmf: Update nvmf_create_transport example
Some users refer to this document as an example of
"gold" RDMA configuration. But some parameters of
RDMA transport are not optimal for good performance,
e.g. `-c 0` disables in-capsule data, that reduces
IOPS of write operations with small payload; `-m 4`
allows to have only 4 qpairs per controller, that
doesn't allow to have more than 4 IO qpairs.
This patch sets several parameters which are default
for RDMA.

Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reported-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: I118bb032eaf7f38e981d06ee312b36c7f0c8322a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11454
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-02-28 11:06:32 +00:00
Karol Latecki
8c32f0d32f doc: add SPDK Vhost 22.01 performance report link
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: Ib594ecd991d25717413a66ed033aa9f7c183ffb3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11680
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-02-23 10:03:44 +00:00