Commit Graph

1113 Commits

Author SHA1 Message Date
paul luse
37b68d7287 accel: cleanup by getting rid of capabilties enum
In support of upcoming patches and to greatly simplify things,
the capabilites enum which held bit positions for each opcode
has been removed.  Only the opcodes enum remains and thus only
opcodes are used throughout.  For the capabiltiies bitmap a helper
function is added to convert from opcode to bit position.  Right
now it is used in the IO path but in upcoming patches that goes away
and the conversion is only done at init time.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Ic4ad15b9f24ad3675a7bba4831f4e81de9b7bc70
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11949
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-04-14 08:32:50 +00:00
Konrad Sztyber
7610bc38dc scripts: move python modules to python directory
Up until now, importing an SPDK RPC python module was just a matter of
`import rpc`.  It's fine until there's another module called `rpc`
installed on the system, in which case it's impossible to import both of
them.  Therefore, to avoid this problem, all of the modules were moved
to a separate directory under the "spdk" namespace.

The decision to move to a location under a separate directory was
motivated by the fact that a directory called scripts/spdk would look
pretty confusing.  Moreover, it should make it also easier to package
these scripts as a python package.

Other than moving the packages, all of the imports were updated to
reflect these changes.  Files under python now use relative imports,
while those under scripts/ use the "spdk" namespace and have their
PYTHONPATH extended with python directory.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ib43dee73921d590a551dd83885e22870e72451cf
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9692
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-04-05 14:40:47 +00:00
Shuhei Matsumoto
1906b0753e blobcli: Use spdk_for_each_bdev() for bdev list traversal
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: Ib7ea70c7046241dd6b09f44cb9a3e7ce8496d5c9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12140
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2022-04-05 11:48:31 +00:00
Evgeniy Kochetov
a2d4ddb3b1 nvme: Prioritize user provided trstring for transport lookup
This patch fixes the issue with custom nvme transport. It is possible
to register custom nvme transport with arbitrary name but it is not
usable because 'spdk_nvme_trid_populate_transport' call in probe
function will always set trstring to 'CUSTOM' and transport lookup
will fail.

Signed-off-by: Evgeniy Kochetov <evgeniik@nvidia.com>
Change-Id: I83fd24dd8732ac0a21e22435e0acff20ab0e7521
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9557
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-03-31 10:31:20 +00:00
paul luse
e68aebd50b lib/accel: remove public API for getting capabilities
First in a series of patches that will enable multiple engines
to exist at once and choose the best one based on their priorities
and capabilites, the public API will no longer be needed.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Ia87b83aa2263745a94a822a160b6e97bb2e0dc19
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11948
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-03-31 09:36:25 +00:00
Changpeng Liu
ed9f26f9c0 examples/nvme/perf: check block size after IO flags
When PRACT=0, the perf tool will skip tests for extended LBA format,
fix it now.

Change-Id: I12e6bc10bb16148a02a30b5d20b540166d45c242
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12053
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
2022-03-29 07:31:23 +00:00
Yuriy Umanets
f3950cf457 fio/bdev_plugin: Added env_context to fio config
FIO config can have "env_context" field now. It allows passing to DPDK
the start-up options essential for mlx5 crypto/reduce support. This
enables using FIO plugin for testing mlx5 crypto/compress vbdev PMDs.

For example:
env_context=--allow=0000:01:00.0,class=crypto,wcs_file=/path/credentials.txt

Signed-off-by: Yuriy Umanets <yumanets@nvidia.com>
Change-Id: I8b862a431fa5b838e337a64e736d4f6700f83599
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11633
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2022-03-24 09:21:35 +00:00
Ben Walker
f0bf4e75f5 idxd: Eliminate configs
SPDK has settled on what the optimal DSA configuration is, so let's
always use it.

Change-Id: I24b9b717709d553789285198b1aa391f4d7f0445
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11532
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2022-03-21 11:05:28 +00:00
Ben Walker
6fb6716d45 fio_plugin/bdev: Support spdk_threads exiting at run time
Now that the plugin supports SPDK-created spdk_threads, it needs to
handle those threads being removed at run time.

Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: I8dee09bf95d1c8fe80f5793ee5476d3f3e1384fd
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11867
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
2022-03-15 09:04:57 +00:00
Ben Walker
0d845de79d fio_plugin/bdev: Assign spdk_threads created by SPDK modules to the init
thread

This allows SPDK modules to spawn spdk_threads and they'll get correctly
polled.

Change-Id: Ie5cadc130aeaa475eb44c0ea55b9a25dfeb29552
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11866
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
2022-03-15 09:04:57 +00:00
Ben Walker
c4f8bff2b3 fio_plugin/bdev: Use thread context space for spdk_fio_thread
Instead of allocating this separately, use the thread context area.

Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: I2d5f4d223b15ff663e57f52237b3cc4dab43c291
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11865
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
2022-03-15 09:04:57 +00:00
John Levon
c4f7ddd2c7 lib/nvme: report shadow doorbell update stats
Currently shadow doorbell updates are not counted; add statistics for
those, and rename the other statistic for clarity.

Signed-off-by: John Levon <john.levon@nutanix.com>
Change-Id: I211a77902e38265c99b15862034c6d022dc582a0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11844
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-03-10 09:49:25 +00:00
John Levon
ba4ffda671 lib/nvme: correct typo in transport stats
"doorbell" not "doobell"

Signed-off-by: John Levon <john.levon@nutanix.com>
Change-Id: I9261559576e72a09b63fbc984ae0ec2a2572eb2c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11841
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-03-09 08:03:42 +00:00
Alexis Lescouet
a71cd5214b event: Add a user option to change the size of spdk_msg_mempool
The spdk_msg_mempool structure has a fixed size, which is not flexible
enough. The size of the memory allocation can now be changed in the
options given to the spdk_app_start function.

Signed-off-by: Alexis Lescouet <alexis.lescouet@nutanix.com>
Change-Id: I1d6524ab8cf23f69f553aedb0f5b0cdc9dde374b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11635
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-03-09 08:00:28 +00:00
paul luse
12c40f05e2 accel: plumb accel flags through operations that need them
This patch is just plumbing the flags param. Use of it for PMEM
will come in upcoming patches.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I620df072aaad3f8062a0312bbea3da1bc3f911b9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9281
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: GangCao <gang.cao@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-03-04 21:56:54 +00:00
paul luse
7d63e716eb idxd: Add flags parameter to all low level API for prep/submission
Previously required flags were hardcoded in the low level library.
By having the user pass them in there is more flexbility and control.
This was driven by the need to add a new flag for pmem durability,
coming in a future patch in this series.

There is no change in functionality with this patch, just movement
of where flags are set and by whom and the plumbing of 'flags'..

Also note that some flags in scenarios that we know are required are
still set by the library.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I194278f9e3cec0886628585cf84bcc2eae635e0a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9449
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: GangCao <gang.cao@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-03-04 21:56:54 +00:00
Jim Harris
f45c98dd29 blobcli: use spdk_thread_send_msg when dumping blob
For thin provisioned blobs, an IO read to an unallocated
cluster completes inline, meaning that if the read
completion function issues another read, we can quickly
blow the stack.  So use spdk_thread_send_msg() to
issue the next read operation.

Fixes #2405.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I10a40b83bfc25fa6bbd8bc93b6fea36ac8ee83c6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11784
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-03-02 08:29:01 +00:00
Changpeng Liu
b0f02769f9 nvme/fio_plugin: call spdk_env_fini() when exiting the process
spdk_fio_cleanup() is called per-thread basis, and spdk_env_fini()
is called multiple times when running with `numjobs` greater than 1.

Fix issue #2366.

Change-Id: I39ec379d71eeba1ca14ba42a93b819220ee1f307
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11443
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: GangCao <gang.cao@intel.com>
2022-02-23 10:05:25 +00:00
Evgeniy Kochetov
b3ca1c70ee nvme/perf: Add option to set NVMe IO queue size
Some targets advertise large Maximum Queue Entries Supported (MQES)
value but fail to create long queues due to some other limitations,
e.g. lack of some resource. New '--io-queue-size' option allows to
workaround such issues. It may also be useful when researching queue
size impact on performance.

Signed-off-by: Evgeniy Kochetov <evgeniik@nvidia.com>
Change-Id: Ie13b966070fbe5d8bb75cee59ae171bd25eafa63
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9090
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-02-15 18:31:52 +00:00
Mike Gerdts
ab2aad8553 blobcli: include bdev name in error message
When multiple bdevs are accessible by blobcli, it may be helpful to know
which bdev is giving the "Unsupported bdev event" error message.

Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: Ic55eacad13336366de3d6eac8bde932decbb2732
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11255
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-02-03 20:07:58 +00:00
Mike Gerdts
3453391baf blobcli: support for deleting blobs
Add a new option, -w for "whack", to delete a blob.

Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: Ic7650d9cb6d2aa1cdfa020ad6389c497a481409c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11254
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-02-03 20:07:58 +00:00
Mike Gerdts
ff4fd58b95 blobcli: add forced recovery via blobcli
In the event that the blobstore superblock is marked clean but there are
indications that it wasn't, `blobcli -R` may be used to force blobstore
recovery.

Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: Ibc631b6dc0576f81fa1e0b71bd0c18244823e4c4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11353
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2022-02-03 20:07:58 +00:00
Mike Gerdts
d14c3bfb8b blobcli: list help options in consistent order
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: I38375670125129d0b728077eb4f7a11b23a9f3bb
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11250
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-02-01 20:05:13 +00:00
Mike Gerdts
643385ea0a blobcli: uuid xattrs should be printed as strings
At the time of commit d1d22046df, xattr UUIDs were stored as
strings, not as `struct spdk_uuid`. That continues to be true today.
As such, bs_dump_xattr should be treating UUID XATTR values as strings
rather than as `struct spdk_uuid`.

This fix does verify that the UUID string is a well-formed UUID before
printing it. If it is not well-formed, "? Invalid UUID" is printed
along with the raw bytes.

Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: I5b574c1c2c4b4802aae3ba23d32ef58fb6fa7586
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11249
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-02-01 20:05:13 +00:00
Mike Gerdts
8389b65ad4 blobcli: do not read from an zero-length blob
Not all blobs have data: an lvstore's super blob is a common
example. Perform a size check prior to trying to dump the blob's
contents.

Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: Ic5bc1b11cdda72a36c6a3f4b4de0fd6f1a2948c6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11246
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-02-01 20:05:13 +00:00
Mike Gerdts
76a577b082 blob: blobcli should use hex for blob IDs
Blob IDs are sequentially assigned starting at 0x100000000.
When debugging with a small number of blob IDs, it is much
more intuitive to see blob ID 0x100000000 rather than blob
ID 4294967296. If blob IDs are displayed in hex, the things
that parse commands should also accept hex to facilitate
copy and paste.

Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: Ic71eaaf1987609b4f705d372ced4240650b12684
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11245
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
2022-02-01 20:05:13 +00:00
Ben Walker
0267f8a977 examples/idxd: Deal with queued tasks during drain
If a task is on the resubmit list, make sure to clear it out during the
drain phase or the queue depth will never go to 0.

Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: I859a03d76865d404ce43e38cfb34c27ca436e537
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11208
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-01-26 08:12:44 +00:00
Ben Walker
c1ec924166 examples/idxd: Correctly handle -EBUSY
Change-Id: I6e794c2355c845959fccc4f04ba588b8162be9bc
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11060
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2022-01-20 07:54:55 +00:00
Ben Walker
4b426c9745 examples/idxd: Remove support for batching
The batching API will be removed from idxd shortly.

Change-Id: I04cf61112f7831a9fb0fefc269706495761d0889
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11032
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2022-01-19 08:49:25 +00:00
Shuhei Matsumoto
1ac578d42c example/nvme_perf: Remove spdk_nvme_poll_group_remove() calls
spdk_nvme_ctrlr_free_io_qpair() calls spdk_nvme_poll_group_remove()
internally.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I0085eb74c1aa2e5a318a980b2916fa20693c8d12
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10662
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2022-01-19 08:44:09 +00:00
Evgeniy Kochetov
a69541fc31 examples/nvme_perf: Fix result calculation with warmup
Signed-off-by: Evgeniy Kochetov <evgeniik@nvidia.com>
Change-Id: I67868da84d61f82a77fc2350dbc530e73ace2d91
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11103
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
2022-01-17 14:26:11 +00:00
Ben Walker
e2efeef080 idxd: Add support for vectored crc32c generation
This uses a batch with the fence flag for now. There are several other
implementation options that will be explored in the future.

Change-Id: I4f344d671400508de05f80b026d42f775c5b9588
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10289
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-01-12 08:20:39 +00:00
Ben Walker
fa6ac87778 idxd: Add support for vectored fill operations
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: I0d58320a03ee82169e83be6449ba52c9d2ee3a55
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10288
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2022-01-12 08:20:39 +00:00
Ben Walker
f11869c44d idxd: Add support for vectored compare operations
Compare two scattered memory regions

Change-Id: I6ce5c9e7bc1ee1ef0e9173c00e86628d43a1e41f
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10287
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2022-01-12 08:20:39 +00:00
Ben Walker
fe70548070 idxd: Add support for vectored copy operations
Change-Id: Icb650129488b3cea76cf9082c02667f5b13b5ab4
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10286
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2022-01-12 08:20:39 +00:00
Monica Kenguva
e1c20e3755 examples/idxd: Add batch functionality to idxd perf tool
Signed-off-by: Monica Kenguva <monica.kenguva@intel.com>
Change-Id: Ib4e1c4d68092316b261cb84f0a1918a7e733dddf
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9735
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-01-12 08:20:39 +00:00
Changpeng Liu
9ec9c8b33d examples/nvme: add missed spdk_env_fini() when exiting
Althrough `rte_eal_cleanup` will be called in the destructor
function when exiting the application, it's OK not to call
`spdk_env_fini`, we still add this function call in case
there are cleanup function need to call in future.

Change-Id: I0a3eca44c5ed1d62059796ea8f1977fb19356939
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10904
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
2022-01-11 14:19:05 +00:00
Changpeng Liu
f2aa5435e9 examples/vmd: add missed spdk_env_fini() call when exiting
Change-Id: If80008cac2a291786984ba3ea06196ef7310271e
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10909
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2022-01-10 18:36:14 +00:00
Changpeng Liu
abcc84acd8 examples/ioat: add missed spdk_env_fini() call when exiting
Change-Id: I993d41dbec264fef140c83dcd585231c8d958a97
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10908
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2022-01-10 18:36:14 +00:00
Changpeng Liu
569e2eeda1 examples/idxd: add missed spdk_env_fini() call when exiting
Change-Id: I8993fa796722d65e899218dc9bc4da95c1538e81
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10907
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2022-01-10 18:36:14 +00:00
Changpeng Liu
50ced20e1c examples/bdev: add missed spdk_env_fini() call when exiting
Change-Id: Icee84d574a9519f6746f899741b553100f357ab0
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10906
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2022-01-10 18:36:14 +00:00
Changpeng Liu
974e069878 examples/nvmf: add missed spdk_env_fini() when exiting
Change-Id: Ia6d1b05602541e2884394a7cb242480fcc33eea9
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10905
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2022-01-10 18:36:14 +00:00
wwlliangliang
0f5a873e6e examples/accel/perf: fix partial-free error before _init_thread
We encountered a coredump in running accel_perf with only one
ioat/idxd device enabled:

accel_perf: accel_engine.c:973: accel_engine_create_cb:
Assertion `accel_ch->engine_ch != NULL' failed.
Aborted (core dumped)

In identify_accel_engine_usage, we try to get an accel channel,
read capabilities of it, and free it. However, the hw destory_cb
is not yet called before running _init_thread because the hw destroy_cb
is delayed by accel_engine_destroy_cb, which is already a delayed
event itself. So, we simply check the first available channel's
capabilities in _init_thread to avoid such partial-free error.

Signed-off-by: Wenliang Wang <wangwenliang.1995@bytedance.com>
Change-Id: I96ae0408568ed6cd2cb9b74fde406a821943616e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10426
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2022-01-05 09:23:12 +00:00
Konrad Sztyber
2e4466d0f1 examples/abort: fail when no controllers are found
Otherwise, the test using this application will succeed, even if no test
is actually executed.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ib2235795ea51985d418c2be6904a68db15aa593a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10772
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-12-23 08:41:41 +00:00
Jim Harris
7b284f6201 examples: add discovery_aer
This can be used to connect to a discovery controller
to get and print discovery logs whenever a discovery
AER completion is received.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ieeb8630633e1be3031074caff6196d4124c5fcee
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10548
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-12-20 18:12:41 +00:00
Jim Harris
843c387a1f nvme: add spdk_nvme_ctrlr_get_discovery_log_page API
This API is a helper for getting the full discovery
log page from a discovery controller.  It will read the
log page header to get the total number of entries,
allocate a buffer for all of the entries, and then
issue a series of get_log_page commands to read each
4KiB worth of entries.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I02666ef5adcb9fc8825a221655811ace708f97b8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10564
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-12-20 18:12:41 +00:00
Jim Harris
d07dc4d361 nvme/fio: free io_qpairs in close callback
If user uses the 'loops' parameter, then it will
call the open+close callbacks for each loop, but only
cleanup at the end of the entire run. Since we alloc
the io_qpair in open, but don't free them until
cleanup, we end up running out of io qpairs at some
point if we run too many loops.

So solution is to free the io qpairs in the close
callback.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ibac864836f94994cdd24a7886894778268b14f73
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10674
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-12-15 16:45:56 +00:00
Jacek Kalwas
43022da379 nvmf: remove accept poller from generic layer
Not every transport requires accept poller - transport specific
layer can have its own policy and way of handling new connection.
APIs to notify generic layer are already in place
 - spdk_nvmf_poll_group_add
 - spdk_nvmf_tgt_new_qpair

Having accept poller removed should simplify interrupt mode impl
in transport specific layer.

Fixes issue #1876

Change-Id: Ia6cac0c2da67a298e88956734c50fb6e6b7521f1
Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7268
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-12-14 13:18:33 +00:00
Krzysztof Karas
bace05499a logs: create a log level flag in abort and hotplug applications
Bloated log files appear during some tests
(nvmf_phy_autotest, during nvmf_abort test fot example),
what makes parsing them inconvenient. This change aims
to disable logs that cause this issue by adding a new flag
for SPDK target and disabling notice level logging completely
(using --silence-noticelog proved insufficient).

Fixes: #2149

Change-Id: Ibbad92d87d90fe73c23d6027e0ff8ec49b0393c2
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10311
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-12-14 09:12:01 +00:00
Krzysztof Karas
eb48f76d61 hotplug.c: add a second switch to parse_args()
Add another level of option parsing to allow usage
of string values.
Currently all values are converted to long
(or have to be checked before the switch),
which results in errors upon adding a string as a value.

This is going to be used in the next patch in the series.

Change-Id: Ib874d74b015eee825b5135ef3d49b9cb1c72029b
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10471
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-12-14 09:12:01 +00:00
wanghailiangx
df5ad409c2 examples/blobcli: remove a redundant error prompt
The details are as follows:
./build/examples/blobcli -h
Error: -b option is required.
(When we input '-h', this is a redundant error prompt)
......
Commands include:
        -b bdev - name of the block device to use (example: Nvme0n1)
        -d <blobid> filename - dump contents of a blob to a file
        -D - dump metadata contents of an existing blobstore
        -f <blobid> value - fill a blob with a decimal value
        -h - this help screen
        -i - initialize a blobstore
......

Change-Id: Ieccbcc56b02b10504170fb2e60090c25dc0d4b53
Signed-off-by: wanghailiangx <hailiangx.e.wang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10581
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2021-12-14 09:09:31 +00:00
Jim Harris
e123181d71 lsvmd: use spdk_pci_for_each_device API
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I1ff5bea00f4b08bf879a3c726689b74f616c1c70
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10658
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2021-12-14 09:08:59 +00:00
Jim Harris
63b41c7237 vmd_led: use spdk_pci_for_each_device API
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I8046242f05e4ffb676570e65e2217cf593add2e6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10657
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2021-12-14 09:08:59 +00:00
paul luse
998b5d664e examples/accel_perf: remove batching
It's being added to the low level library perf tool which makes
more sense as the accel_fw provides queueing thus allowing any
queue depth.  When using the low level lib you are limited to
a small number reported by HW unless you use batching then you
can get quite a bit more.

Future patches will remove batch from the accel_fw, this is just
the accel_perf tool.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I3775b9ecdf353f730c7f3e2c06bb45a8873e61f8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9845
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-12-08 16:35:40 +00:00
Mao Jiang
1f085e8e23 example/nvme/hello_world: add support for vfio-user transport
Change-Id: I280fc2cd84f577a71dda00dbc0f72f01fead6fba
Signed-off-by: Mao Jiang <maox.jiang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10182
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-12-07 08:22:13 +00:00
Changpeng Liu
4518e4c34e examples/arbitration: fix wrong usage of arbitration for vfio-user transport
This fixes commit 74dcf4aa "example/nvme/arbitration: add vfio-user transport support".
For vfio-user transport, we should use static DPDK memory model for
sharing memory between client and target.  Also enable log option here.

Change-Id: Iea1b28cbf234f5fc935c54899023bdbf1733a671
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10510
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-12-03 00:46:52 +00:00
Mao Jiang
74dcf4aa58 example/nvme/arbitration: add vfio-user transport support
Change-Id: I4aff5ec209de46e4e5e279cd84ce4968df568a60
Signed-off-by: Mao Jiang <maox.jiang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10464
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-12-01 09:19:28 +00:00
Krzysztof Karas
ac1abb5be8 hotplug: add check for uio_pci_generic driver
Add a check for this driver and enable iova=pa mode when it is
detected.
Add an option for the hotplug application to force iova mode.

This is to avoid:

EAL:   Expecting 'PA' IOVA mode but current mode is 'VA', not initializing
EAL: Requested device 0000:86:00.0 cannot be used

while using hw_hotplug test.

Change-Id: I7ff819c04b1e567b5ef88fc8f551ecec901806c8
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9774
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-12-01 09:16:29 +00:00
Josh Soref
b7876f9a63 spelling: examples
Part of #2256

* asymmetric
* bookkeeping
* current
* errors
* execution
* initialize
* initing
* management
* occurred
* operations
* prepping
* purposes
* similar
* syncing
* terminate

Change-Id: I7336d98c1327020ab83c9e01e6bf1e4f5f7b5d7d
Signed-off-by: Josh Soref <jsoref@gmail.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10403
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-12-01 09:15:03 +00:00
Niklas Cassel
66f7c55ed5 bdev_fio: use the new bdev_get_zone_id() helper
Use the new bdev_get_zone_id() helper when calculating the zslba.

Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Change-Id: I370a9ea3f82368e7b1a764bf221378797defe6b0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10181
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-11-11 23:23:35 +00:00
Ben Walker
7ffc8833f6 examples/nvme/hello_world: Iterate only active namespaces
Avoid registering inactive namespaces in the example. They can't be used
for I/O.

Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: I404ca48666b566fd3248a3f56b25ea6509fa8d38
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10099
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-11-10 19:36:27 +00:00
paul Luse
fa9e703f4f examples: update pollers to use the expected enum values
Some were still using 0, -1 and/or count values.

Signed-off-by: paul Luse <paul.e.luse@intel.com>
Change-Id: Ic161b5436c55f229f1f1a557d29e395bf8d4f660
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8584
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-11-03 23:12:25 +00:00
Alexey Marchuk
31a1b193a2 perf: Add check that io_size is a multiple of ns sector size
It is possible to specify io_size which is not a multiple
of sector size and in that case real IO will have size
of g_io_size_bytes/sector_size. It is integer division, so
IO will be less than requested by the user. At the same
time performance statistics are reported using io_size
specified by the user (which is bigger than real IO size)
so we have wrong statistics (e.g. we can see BW higher
than NIC line rate).
To avoid this confusing situation, add a check that
user's io_size can be evenly divided by ns sector size.

Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Change-Id: Ie63b5b7b3d99b5c51c10ec09e084fd30d6cb33dd
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9733
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-10-28 11:33:14 +00:00
Changpeng Liu
9f7cb6aa2d examples/nvme/cmb_copy: exit the test tool if no controller found
fix issue #2200.

Change-Id: I9d0374e56f199d562c48581ec6630547175d8627
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9927
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-10-20 07:24:47 +00:00
Jim Harris
4963728855 nvme_fio: add log_flags
Allow user to add something like:

log_flags=nvme,nvme_vfio

to their fio config file to enable log flags.  On
DEBUG builds, setting at least one flag will also
set the print_level to DEBUG.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I7a409445d23c96845bc7afa9641966a964b7f44f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9871
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-10-14 08:17:28 +00:00
Jim Harris
1b61d481cc bdev_fio: add log_flags
Allow user to add something like:

log_flags=app_config,nvme,bdev

to their fio config file to enable log flags.  On
DEBUG builds, setting at least one log flag will
also set the print_level to DEBUG.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I383512e47ad6ea86b8b2c71a212bdc88fadf2f65
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9870
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-10-14 08:17:28 +00:00
Changpeng Liu
e4913b3c7f examples/nvme/identify: print Command Retry Delay Time if exist
Change-Id: I7a9ea6054b853bd1abe7cbf7c918a220b1234485
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9263
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-10-08 07:26:45 +00:00
John Levon
890b657e6b examples: detect fio backend fd redirection
It's useful to be able to run fio daemonized, and very recent versions of fio
redirect stdout/stderr to /dev/null in order to avoid the problem that
previously caused the plugin to abort. Detect if this redirection has happened
and allow fio to run in this case.

Signed-off-by: John Levon <john.levon@nutanix.com>
Change-Id: I8ec13a4c26a0acc67b15fa5e8502dc28d341e441
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9691
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Andreas Economides <andreas.economides@nutanix.com>
2021-10-07 09:22:57 +00:00
Karol Latecki
65113c191b examples/bdev: update fio bdev plugin readme file
Update readme file with more detailed examples on how to run
fio workloads versus local and remote storage.

Includes some minor markdown styling fixes as well, to
keep the file style consistent.

Change-Id: I93e8079abbda689a2d3414d1dca959e34c7e5b94
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8821
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-09-24 07:39:41 +00:00
Alexey Marchuk
fad59a283f perf: Align output of transport statistics
It'll make output more readable and align all
transports output

Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Change-Id: I9ac420ffd947ed433ebefec50e7c287ddf2eb344
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9162
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@gmail.com>
2021-09-24 07:30:06 +00:00
Alexey Marchuk
ea86c035bb nvme/tcp: NVME TCP poll group statistics
Enable dump of transport stats in functional test.
Update unit tests to support the new statistics

Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Change-Id: I815aeea7d07bd33a915f19537d60611ba7101361
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8885
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-09-24 07:30:06 +00:00
root
69f89ecf4b examples/idxd: Add the perf tool for idxd
The purpose of this patch is to add an independent
idxd perf tool. Because users may not always
use SPDK application framework, io channel related
API.

With this tool, users can know to integrate spdk's
idxd api in their own software stack.

In this patch, batched submission is not supported yet.

Change-Id: I6e3e62cdb7c43da591ab0ae90fc17771222f7285
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9398
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-09-22 06:54:48 +00:00
Krishna Kanth Reddy
13905aa3e7 examples: Zone Descriptor Extension support
Signed-off-by: Krishna Kanth Reddy <krish.reddy@samsung.com>
Change-Id: I22038e65cc3c2d65c4857569185f14bce2ab1aaf
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9495
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-09-21 07:45:43 +00:00
Maciej Wawryk
cb85bf0b49 markdownlint: enable rule MD003
MD003 - Header style
Fixed all errors

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Ic00d38a3fa0af16581598fd49dc2ee7e1632c9d8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9287
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-09-08 21:53:48 +00:00
Maciej Wawryk
111d42765d markdownlint: enable rule MD046
MD046 - Code block style
Fixed all errors

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I0a5f711a54e1859a6c8d0f26dcabf210496fb819
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9273
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-09-08 21:53:48 +00:00
Maciej Wawryk
63ee471b64 markdownlint: enable rule MD040
MD040 - Fenced code blocks should have a language specified
Fixed all errors

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Iddd307068c1047ca9a0bb12c1b0d9c88f496765f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9272
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-09-08 21:53:48 +00:00
paul luse
5bdad0c1af examples/accel_perf: fix small bugs with copy + crc
Wrong opcode was used when deciding whether a dst buffer is needed
or not. Also on freeing we were freeing src instead of the vector
parts of the task.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I33eb9408ee20eca6c44911b4567d0367528347de
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9189
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
2021-09-01 07:34:02 +00:00
paul luse
df42f3582b lib/accel: eliminate need for caller to schedule completions
Previously the accel_perf tool would look at whether it had HW
or SW commands to know whether to execute the callback right away
or schdule to avoid blowing the stack (SW calls are sync).

Moved this to the SW module (part of the accel engine) so the
caller doesn't have to worry about.  Allowed for a few simplifcations
in the tool as well.

Also, instead of using send_msg to call the completion we add it to
a list in the sw module that a new poller uses to perform the
completions as this is more efficient the sending a message to the
same thread.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Ifc6c5b8635f51e3fa1a825c8573378b3752d7d91
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9171
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-09-01 07:34:02 +00:00
wawryk
1e1fd9ac21 markdownlint: enable rule MD025
MD025 - Multiple top level headers in the same document
Fixed all errors
Update check_format.sh to fit new header style in jsonrpc.md

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Ib5f832c549880771c99c15b89affe1e82acd3fa4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9045
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-08-26 19:27:22 +00:00
Ziye Yang
c260537919 accel_perf: check accel engine usage in accel_perf_start function.
In accel_done function, we use g_using_sw_engine to
determine to directly call _accel_done or use thread message
passing way.

However when we get the capability and determine the
value of g_using_sw_engine in mulitple cases. We do not
protect the value of g_num_workers. Then if we use CPU mode
to do test, g_using_sw_engine will not set to be false
in racing condition. Since the value of g_num_workers can be
> 1 when we do the check, i.e., it is a TOC2TOU issue.

The solution is that we check the g_using_sw_engine in
accel_perf_start function.

Fixes issue #2084

Change-Id: I55c18e0443120adb698d5bd27d4522df09f6dcab
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9151
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2021-08-16 08:48:43 +00:00
Jim Harris
a595959da0 test/nvme/abort: ensure admin queue is sized appropriately
This abort test app will send a lot of abort commands on
the admin queue.  The default admin queue size is
relatively small (32) so increase it if necessary to
account for the expected number of outstanding abort
commands as well as any extra admin commands that may
be sent during test execution such as Keep Alive.

Fixes issue #2048.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I5f64b7fc72a028299b860f09e30d430a64c95d2a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8812
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2021-08-11 06:51:13 +00:00
Felipe Franciosi
3afba66ca5 perf: Fix help message on --io-pattern usage
The --io-pattern option can be specified by the -w short opt. However,
the help message suggested that -o should be used. This fixes it.

Signed-off-by: Felipe Franciosi <felipe@nutanix.com>
Change-Id: I023d571fc3b165653257688fdbd7b1eeafd420e4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9116
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-08-10 08:18:47 +00:00
wawryk
e5b5eabe82 markdownlint: enable rule MD002
MD002 - First header should be a top level header
Fixed all MD002 errors

Signed-off-by: wawryk <maciejx.wawryk@intel.com>
Change-Id: Id0cb693c18f3e26556949ba2220db7b25e9edbe1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8940
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-08-05 07:42:35 +00:00
Alexey Marchuk
ec1b78dbd7 socket: Remove deprecated enable_zerocopy_send
This parameter is still part of API spdk_sock_impl_opts
structure but it is not used. Keep it to support ABI
compatibility since it is located in the middle of the
structure and removing it may break socket opts initialization
or parsing.

Change-Id: Ib641ad7d965d68bc9ebb65dba531408d88cf6fa1
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8914
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-07-23 10:30:25 +00:00
paul luse
a738acd505 examples/accel_perf: use local task element for CRC destination
For all CRC related functions.  Does not need to be DMA'able memory
as DSA returns the CRC to it's completion record and the lib
copies it to this address.  Done for consistency as this element
was added as part of adding the copy+CRC API.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Iefedcbc0a1c4e211eeb8aaf5c52f2881e9173bad
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8230
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
2021-07-15 21:04:08 +00:00
paul luse
221eb3f4dd examples/accel_perf: add support for copy_crc commands
For COPY+CRC operations, we allocate a destination buffer for each
task that's big enough to hold the copied input data for all stages
of the chain. For example, if the operation is 4 stages of 4K input
buffers, the destination will be 16K in size.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Id32a98c1dbada2881a8423c050e892d5c37bf6cc
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8204
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-07-15 21:04:08 +00:00
Shuhei Matsumoto
f6b19841da example/nvme_identify: Fix the bug that assumed ANA descriptor is 8-bytes aligned
This is the same fix as done for lib/nvme.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I9ca5e1ea9dbdba45b51b311e9a36f278b1b62770
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8437
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-07-14 09:15:59 +00:00
Matt Dumm
5288c4df83 nvme: separate admin timeout vs. io timeout
Signed-off-by: Matt Dumm <matt.dumm@hpe.com>
Change-Id: Ia1f105fdf154aae034ccfca2f0dc3b4c43c9fc84
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8072
Reviewed-by: Michael Haeuptle <michaelhaeuptle@gmail.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
2021-07-14 09:15:20 +00:00
Ziye Yang
9a0bd78b43 interrupt_tgt: Fix the interrupt mode usage issue.
When the application is not set with the interrupt mode,
we should fail the rpc.

Fixes issue #2023

Change-Id: I4157f220948b7c96f474682c41ac9d78977b1c40
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8676
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: <dongx.yi@intel.com>
2021-07-13 08:59:41 +00:00
yidong0635
673afb3074 identify: Fix truncation warning of GCC11.
Fixes issue: #2031.

Signed-off-by: yidong0635 <dongx.yi@intel.com>
Change-Id: I67055ad5a42f16f3f927b15cfa8fd4ead2ca8813
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8697
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-07-13 08:59:00 +00:00
Ben Walker
e1d06d9954 net: Remove library
Now that we've deprecated the RPCs for a release, we can remove the whole
library.

Change-Id: I0f1a357fcfb3404efac39aa021928841c2f22ff1
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4305
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-07-13 08:57:58 +00:00
Shuhei Matsumoto
efbd101b8b nvme: Rename cmic.multi_host by cmic.multi_ctrlr of spdk_nvme_ctrlr_data
Bit 1 in the CMIC of the Identify Controller Data Structure specifies
if the NVM subsystem may have multiple controllers or not.

However, multi_host indicated a particular use case such that the NVM
subsystem is used by multiple hosts.

multi_ctrlr will be more appropriate.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I0246096a5cc44721aeff3ff6f96473a2abe11964
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8719
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-07-13 08:57:33 +00:00
Alexey Marchuk
01bbf5d183 perf: Correct usage
Some parameters were accidentally added under
`#ifndef DEBUG`

Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Change-Id: Idd2fd67cf7b3bcc298a290bc45b9572dcf4c0076
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8466
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2021-07-05 14:45:11 +00:00
Alexey Marchuk
885331fe16 perf: Ignore trtypes that were not requested by user
If perf is connecting to a subsystem with listeners
of different transport types (e.g. TCP and RDMA) and
the user request a specific trtype via CLI (e.g. TCP),
discovery process will call probe_cb for every transport
type. As result, probe_cb in perf will return `true`
and undesired controllers will be created and used in
IO path.

This patch adds a check for trtype and trstring to ignore
controllers that are not of a requested type.

Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Change-Id: Id87afd03c7b38edfbbfecb5ad2239fe3e9ac9f83
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8465
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-07-05 14:45:11 +00:00
wanghailiangx
585543acd8 example/blob: fix a historical notes hello_blob.conf
Now, only hello_blob.json in examples/blob/hello_world.

Change-Id: Ib2606935b2466197d958fe6ced0db201c8c3e53a
Signed-off-by: wanghailiangx <hailiangx.e.wang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8590
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Chengqiang Meng <chengqiangx.meng@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
2021-07-05 14:36:53 +00:00
Ziye Yang
358b84b40f accel_perf: Fix the complier complaining.
Faced the following warnings during the compilation:

accel_perf.c:222:17: warning: ‘argval’ may be used uninitialized in this function [-Wmaybe-uninitialized]
  222 |   g_crc32c_seed = argval;

And this patch can be used to fix this issue.

Change-Id: I84ee542d629ff5d9e6a2873c0035bea81b376150
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8538
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Chengqiang Meng <chengqiangx.meng@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: wanghailiang <hailiangx.e.wang@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2021-07-01 15:34:18 +00:00
Shuhei Matsumoto
00833e651c example/nvme_identify: Use spdk_nvme_detach_async()
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ie026026fd64d36974e4a877aa9755bafdbe4fb29
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8473
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-06-30 22:54:19 +00:00
Shuhei Matsumoto
9a99ff90f9 example/nvme_hotplug: Use spdk_nvme_detach_async()
We can add one or more ctrlrs to the existing detach context even
after polling started as long as spdk_nvme_detach_poll_async()
returns -EBUSY.

By relying on this update, add a global variable g_detach_ctx and
use it to aggregate multiple detachments.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I28dcfa41daf1fe2bbe76fac80e1bc2abc24751f7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8471
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-06-30 22:54:19 +00:00
Shuhei Matsumoto
4fe4040a14 nvme: Add spdk_nvme_detach_poll() to simplify a common use case
Add a new function spdk_nvme_detach_poll() to simplify a common
use case to continue polling until all detachments complete.
Then use the function for the common use case throughout.

Besides, usage by simple_copy application was not correct, and
fix it in this patch.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ic14711cd8478bf221c0fe375301e77b395b37f26
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8509
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-06-30 22:54:19 +00:00
Jim Harris
e1bf63afc9 accel_perf: add -a option for allocate depth
For benchmarking purposes, we may want to use a
relatively low queue depth but spread the operations
across a wider range of memory.  A new -a option is
added where the user can specify an "allocate depth"
to do exactly that.  In this case, more tasks (and
their associated buffers) can be allocated than we
have actual queue depth.  Then when we pick a new
task for the next operation, it will use a different
memory range and avoid always using the same buffers
over and over again.

If not specified, we just allocate the same number
of tasks as the queue depth, which is the current
behavior.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I78042d905fd49d130c4a318e2c19eb11b84ff726
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8451
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2021-06-28 16:18:54 +00:00
Jim Harris
c82d5789e6 accel_perf: check that numeric arguments are non-negative
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ie512181e7e0537e8319abe0a5b53492c959964d2
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8491
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
2021-06-28 16:18:54 +00:00
Jim Harris
451462f692 accel_perf: cycle tasks through task_pool
When a task completes, always put it back in the task
pool (using TAILQ_INSERT_TAIL) and then use _get_task
to get a task for the next submission.  Currently this
will just allocate the task that was just put into
the TAILQ.  But an upcoming patch will allocate
more tasks than we have queue depth, and this patch
will ensure all of those tasks get used evenly.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ia6247a76ba60337fe6da3189e6c305f4dfb0a041
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8450
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
2021-06-28 16:18:54 +00:00
Jim Harris
4cd7ca9bc0 accel_perf: assign task->worker when building tasks_pool
Previously we would only assign the worker in _get_task().
But future patches will use _get_task() for each I/O,
not just during startup.

Also increment the current_queue_depth during startup
only, rather than every time _get_task() is called.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I2713591d36b8353eaab8119ee230261f40857019
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8449
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Community-CI: Mellanox Build Bot
2021-06-28 16:18:54 +00:00
Niklas Cassel
3edf255572 bdev/fio_plugin: spdk_fio_get_zoned_model() is too permissive
The .get_zoned_model() callback is supposed to reject unsupported
file types. Right now, we do not reject unsupported file types.

For our specific ioengine, this isn't strictly needed, since our
ioengine unconditionally sets f->filetype to FIO_TYPE_BLOCK, and if
it fails to find a SPDK bdev that matches the --filename, it will
return an error that it couldn't find the bdev matching filename.

However, all .get_zoned_model() callbacks in the fio in-tree ioengines
have a check that a given file has a file type that is supported by
the ioengine itself. This is needed since they do not set f->filetype
themselves, but instead let fio generic code initialize f->filetype.

Since we reuse --filename to mean something in the SPDK namespace, we
are force to initialize filetype manually. So that is the only reason
why we know that the file type will be FIO_TYPE_BLOCK. Anyway, let's
try to keep our code as similar to the in-tree ioengines as possible.

The SPDK nvme ioengine already has this check, so adding it in the
SPDK bdev ioengine makes our ioengines more consistent as well.

Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Change-Id: Ib5e19c738dea0f8d41641b63d0fabe055a930827
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8329
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-06-15 08:34:36 +00:00
Niklas Cassel
1a7a599ba0 bdev/fio_plugin: initialize filetype to FIO_TYPE_BLOCK
fio will do certain actions depending on the backing file
type of the target that it runs against.
(E.g. if the zbd code in fio detects that the backing file
is FIO_TYPE_FILE, it will emulate zones inside the regular
file.)

Both SPDK ioengines reuse the filename option to not point
to an actual path exposed by the OS, but to instead point to
a device in the SPDK namespace.

Because of this, the file type detection in fio will fail,
and will always initialize filetype to FIO_TYPE_FILE.
Therefore, the SPDK ioengines will need to initialize
f->filetype themselves.

The SPDK nvme ioengine already initializes f->filetype to
FIO_TYPE_BLOCK unconditionally. Do the same in the SPDK
bdev ioengine.
(Just like in the SPDK nvme ioengine, we also need to call
fio_file_set_size_known(), so that fio generic code does
not try to initialize f->real_file_size.)

Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Change-Id: I37df185524ed262cb875105f989685b740b430a3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8328
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-06-15 08:34:36 +00:00
Niklas Cassel
5172337d95 nvme/fio_plugin: spdk_fio_get_zoned_model() is too permissive
The .get_zoned_model() callback is supposed to reject unsupported
file types. Right now, we only reject FIO_TYPE_PIPE.

However, in attach_cb() our ioengine unconditionally initializes
f->filetype to FIO_TYPE_BLOCK. This means that the only file type
that our ioengine supports is FIO_TYPE_BLOCK.

Therefore, fix spdk_get_zoned_model() to reject everything that
we do not support.

Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Change-Id: I115bb8ff9c8ea9f647baebc0196900681a36c410
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8327
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-06-15 08:34:36 +00:00
Niklas Cassel
bd53d09c31 nvme/fio_plugin: remove ZBD_IGNORE usage
The define ZBD_IGNORE has been removed from upstream fio.

fio will now return an error if --zonemode=zbd is used with a
--filename that points to something that is unsupported by either
the zbd code in fio or by the ioengine itself.

ioengines are now supposed to return -EINVAL for unsupported files
(instead of returning 0 with ZBD_IGNORE for unsupported files).

This change does not need any special ifdef FIO_IOOPS_VERSION check.

Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Change-Id: I7bec0b1f5dc8f166ebf683f6f3937b2ef295a21e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8326
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Community-CI: Mellanox Build Bot
2021-06-15 08:34:36 +00:00
GangCao
9cedc7202c Util: check the error case of NULL pointer
Change-Id: I0be36b584c99de822920b935c34708576617ffc3
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8176
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-06-08 08:26:12 +00:00
paul luse
475fadf365 examples/accel_perf: don't allocate channel just to get engine caps
Getting/putting the channel races with getting real IO channels
causing issues with an upcoming design. Avoid it entirely by using
the first worker channel to get the capabilties.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I750a8d59e294f4cda9f71f3327afc03e15342768
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8170
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-06-04 15:48:22 +00:00
paul luse
3e2e2a5765 examples/accel_perf: don't schedule final callback when using HW
This results in a significant performance improvement when using
HW offload engines.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I54d3e39f186c9f878fcf8bed8e9242e29d1e1bf1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8162
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
2021-06-04 15:48:22 +00:00
Ziye Yang
b85127cc6f util/crc32c: Add spdk_crc32c_iov_update.
Purpose: To support caculating crc32c for iovs.

Change-Id: I7ea6d8b71ea2cf6e8b81903439787870923b9bef
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8093
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2021-06-02 11:37:17 +00:00
Ben Walker
dde419084b event: Move json config loading to init
Loading subsystems and restoring state from a JSON config file is useful
outside of the SPDK application framework, so move it to lib/init.

Change-Id: I7dd3ceace2e7b1b28eef83c91ce6a4eedc85740e
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6645
Reviewed-by: Tom Nabarro <tom.nabarro@outlook.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2021-05-26 09:13:34 +00:00
Ben Walker
dc9374dd2f init: Declare a formal public API
The functions to initialize the SPDK subsystems or tear them down
was previously an internal-only API. Make it public for use by
applications that aren't leverage SPDK's application framework.

Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: I2ebfd020e6fa4c1947fa1c1a2ac509ce9b0242f8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6643
Reviewed-by: Tom Nabarro <tom.nabarro@outlook.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2021-05-26 09:13:34 +00:00
Changpeng Liu
51b6cbb9a7 examples/nvme/identify: pass nsid to Error Recovery feature
The P5800X drive will report the following error when getting Error Recovery
feature, according to the specification we should pass valid nsid.

nvme_qpair.c: 238:nvme_admin_qpair_print_command: *NOTICE*: GET FEATURES ERROR_RECOVERY cid:23 cdw10:00000005 PRP1 0x0 PRP2 0x0
nvme_qpair.c: 452:spdk_nvme_print_completion: *NOTICE*: INVALID NAMESPACE OR FORMAT (00/0b) qid:0 cid:23 cdw0:0 sqhd:000f p:1 m:0 dnr:1
get_feature(0x05) failed

Fix issue #1955

Change-Id: I30ce60cddcddbf78e57a8c172a2056a82ac72802
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8009
Reviewed-by: Chengqiang Meng <chengqiangx.meng@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2021-05-25 07:12:20 +00:00
Changpeng Liu
7ee25c2e4c examples/nvme/identify: rename get_features to get_ctrlr_features
This will help us to add get namespace features later.

Change-Id: I7e809cd9e3bcb3874ce141183b9c4ee82543616c
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8008
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: <dongx.yi@intel.com>
2021-05-25 07:12:20 +00:00
Ben Walker
af935f768e event: Shift subsystem initialization code to a separate library
This is useful for applications even if they elect not to use the SPDK
event framework.

This doesn't shift everything in one go - just the subsystem
initialization logic. Configuration file loading also needs to move
in a separate patch later.

Change-Id: Id419df1045442d416650ed90e5ee78adfdd623d7
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6641
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-05-24 10:12:50 +00:00
Jim Harris
40a9b64352 nvme/perf: add -F option for zipf distribution
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I818c5671574f556176a6cc5a0e372735a9e8a51a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7792
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
2021-05-19 07:50:28 +00:00
Jim Harris
6ee7dd5375 nvme/perf: move rand_r seed to ns_entry_
This eliminates the thread-local seed variable.  But
we're also adding zipf distributions in an upcoming
patch, and we'll want to store that context in the
ns_entry rather than making it thread local.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Icc4a8b7bdbc9cd35525f2d35c9ada8e3ec0ba76c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7791
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
2021-05-19 07:50:28 +00:00
Jim Harris
e6b7e58574 examples: add zipf example app
This app shows how to use the SPDK zipf functionality,
and uses spdk_histogram to show the cumulative
distributions.

Usage:

build/examples/zipf 1.2 100000000 10000

This will generate 10000 random numbers from a range
[0, 100000000) using zipf theta parameter = 1.2.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I36486399b54f57e771cbc5530ae4905afca60b64
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7780
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2021-05-19 07:50:28 +00:00
Niklas Cassel
99550a756d bdev/fio_plugin: fix typo in README.md
The definition is that:
max active zones >= max open zones.

The REAME.md incorrectly writes max active twice.
Fix this typo in the README.md.

Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Change-Id: I4b7d24d7bd1efcbe1608d3ee04d8900f83857f5e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7912
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-05-18 12:42:38 +00:00
Niklas Cassel
6e57273783 nvme/fio_plugin: implement support for fio .get_max_open_zones callback
Implement support for the recently added fio .get_max_open_zones callback.

If our ioengine does not implement this callback, fio will always result
in an error when using --zonemode=zbd, on platforms which does not have a
fio oslib implementation for this callback, e.g. FreeBSD.

On Linux, fio will by default try to parse sysfs, which will of course not
work on SPDK.

Implement this callback so that our ioengine will be able to provide fio
with the proper max open zones limit.

This will ensure that fio will be able to fetch the proper max open zones
limit, regardless of OS.

While our SPDK nvme ioengine did overwrite the max_open_zones option if it
was set to zero, this is a bit of a hack. The new fio callback is the
proper way to inform fio about the max open zones limit, so that fio itself
can have access to the actual device limit.
(Just overwriting the requested max_open_zones option will not allow fio
to know if the requested max_open_zones option exceeds the device limit.)
Remove the SPDK specific hack and update our README.md accordingly.

Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Change-Id: I532a0fa065b9e215ee6229b9100135e5403f198e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7898
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-05-18 12:42:38 +00:00
Niklas Cassel
ddf27c3d15 nvme/fio_plugin: move get_fio_qpair() out of ifdef and reuse it
Move get_fio_qpair() out of ifdef and reuse it in spdk_fio_queue(),
since the code is next to identical.
The only difference is that get_fio_qpair() returns NULL if qpair->ns
is not set.

Since we know that get_fio_qpair() returns NULL if qpair>ns is not set,
we can also remove the equivalent check in spdk_fio_queue().

Also use ns pointer instead of qpair->ns pointer in two other places
in spdk_fio_queue().

Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Change-Id: Ie647a814ff0b7a6f2bb81cbc9d7e95537fbe5256
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7897
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-05-18 12:42:38 +00:00
Niklas Cassel
afc432ba96 bdev/fio_plugin: implement support for fio .get_max_open_zones callback
Implement support for the recently added fio .get_max_open_zones callback.

If our ioengine does not implement this callback, fio will always result
in an error when using --zonemode=zbd, on platforms which does not have a
fio oslib implementation for this callback, e.g. FreeBSD.

On Linux, fio will by default try to parse sysfs, which will of course not
work on SPDK.

Implement this callback so that our ioengine will be able to provide fio
with the proper max open zones limit.

This will ensure that fio will be able to fetch the proper max open zones
limit, regardless of OS.

Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Change-Id: Ia9c281290e11e4204d270ba4090edb73212ce20f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7896
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-05-18 12:42:38 +00:00
Jaylyn Ren
5f179d8bd1 perf: Fix an error about perf cores usage
There is an error when collecting the cpu usage data in perf.c.

The cpu busy data exceeds 100% when running perf tool with mutli-cores.

Signed-off-by: Jaylyn Ren <jaylyn.ren@arm.com>
Change-Id: I9a00f86a51fb52e115c37199aee6f849b235d868
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7431
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2021-05-13 10:27:10 +00:00
Niklas Cassel
acca0749df nvme/fio_plugin: change zone append print to SPDK_DEBUGLOG
Change zone append print to SPDK_DEBUGLOG() instead of an unconditional
print. Since we already print a warning when zone append isn't supported,
there is no need to unconditionally print something for the success case.
This also matches the behavior of the SPDK fio bdev plugin.

Also improve the zone append help text to match the SPDK fio bdev plugin.

Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Change-Id: Ic7604ea4bdf17e0cddfe2638f3db7944ee652474
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7804
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-05-12 09:18:10 +00:00
Niklas Cassel
367784a641 bdev/fio_plugin: update README.md and add example zbd config
Update README.md with information specific to zoned block devices.
Additionally, add an example config that can be used to test the
--zonemode=zbd fio option using the virtual zoned block device
SPDK module on top of a Malloc bdev.

Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Change-Id: I68bbc07ab70ad1ce8e38abc91dd8c15c49f90656
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7758
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-05-12 09:18:10 +00:00
Niklas Cassel
e415c253a3 bdev/fio_plugin: add support for --zone_append
Add support for using zone append commands instead of write command,
for bdevs that support it.

In the SPDK NVMe plugin, the target struct (called spdk_fio_qpair) is
allocated in .setup(), but the qpair is first created in .open_file.

In the SPDK bdev plugin, the target struct (called spdk_fio_target)
is allocated in spdk_fio_bdev_open(). This function also creates the
I/O channel. (spdk_fio_bdev_open() is called by .init().)

Since certain options has to be saved in the struct spdk_fio_target,
which is allocated quite late, we cannot reuse spdk_fio_handle_options(),
which has to be called before threads are created.

Therefore, we unfortunately need another option parsing function in the
SPDK bdev plugin.

If we fail to handle any of the per target options, the target is never
added to the list of targets, and is freed immediately.

Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Change-Id: Iacd1924c8f0d3d324f2a6f1b0a54f86459f39d31
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7727
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-05-12 09:18:10 +00:00
Changpeng Liu
67eb36c7fa nvme: remove aligned attribute
When testing vfio-user target with QEMU, there is following error log:

ctrlr.c:2143:31: runtime error: member access within misaligned address 0x7fe526a73d98 for type 'struct spdk_nvme_ctrlr_data', which requires 16 byte alignment

For vfio-user transport, the memory buffer is allocated by clients, so
we can't assume the memory is always aligned, just remove the
aligned attribute.

Change-Id: Ie80530415013ebd7bcb8cdabca97d7d0e34857f9
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7797
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-05-12 08:48:47 +00:00
Krishna Kanth Reddy
78a794b9a3 examples/identify: Identification of Simple Copy Command support
Signed-off-by: Krishna Kanth Reddy <krish.reddy@samsung.com>
Change-Id: I251f3b1a6c7e36b946ddc8bd8a07b0f588e23c9e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7693
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-05-11 12:04:54 +00:00
Niklas Cassel
e3997dc23b bdev/fio_plugin: add support for --initial_zone_reset
--initial_zone_reset has to be done before fio creates its threads.
Since --zonemode=zbd only supports --create_serialize=1 (default),
this is possible.

Since this requires an I/O channel, and is called before threads
are created, follow the same pattern as spdk_fio_report_zones()
and simply initialize the thread early if this option is used.

Just like spdk_fio_report_zones(), thread cleanup has to be
called explicitly in case of error. (Since fio will only call
.cleanup() on threads that it has called .init() on.)

Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Change-Id: I7d4dedce88309e4c6e5800ed3d56cd5ccb297551
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7726
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-05-07 11:41:46 +00:00
Niklas Cassel
36728cd80d bdev/fio_plugin: implement callbacks needed for fio --zonemode=zbd
Implement callbacks needed for fio --zonemode=zbd, so that we can use fio
to exercise the bdev-zone layer in SPDK.

spdk_fio_report_zones() will be called before SPDK normally initializes
the threads.

spdk_fio_report_zones() simply initializes its spdk thread when called.
Threads not running against a zoned block device are not affected.

Each spdk thread is still destroyed after the fio run has completed,
as normal. If spdk_fio_report_zones() encounters an error, we have
to call spdk_fio_cleanup() so that the thread is added to the global
list of spdk threads that are cleaned up after main loop exit.

Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Change-Id: Ia094eae8c9e44d8cfc66a5d9b7100f09f9db456a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7725
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-05-07 11:41:46 +00:00
Niklas Cassel
af72965519 bdev/fio_plugin: fix scan-build error
spdk_fio_init_thread() assigns td->io_ops_data.
spdk_fio_init_thread() can return an error, but currently lacks error
handling code.

Add error handling code, and add an assertion that fio_thread
(td->io_ops_data) is assigned when spdk_fio_init_thread() didn't return
an error.

Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Change-Id: I8de7d59db0373599c90aa57fffa476c6707b6104
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7732
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-05-07 11:41:46 +00:00
Swapnil Ingle
6cebe9d06b nvmf: Support physical block size if exposed by bdev
(Note: this patch was previously applied as b32cfc46 and then reverted
as 63642bef.)

Today the in-guest nvme device shows physical_block_size=512 even though
the backend iSCSI bdev supports physical_block_size=4K

iSCSI targets exposes physical block size using
logical_block_per_physical_block_exponent in READ_CAPACITY_16

NPWG is one of the way to let Linux nvme driver set
physical_block_size of the nvme block device.

This patch adds spdk_bdev.phys_blocklen which is updated if the iSCSI
backend exposes physical_block_size.
Later phys_blocklen is used in nvmf to set NPWG and NAWUPF to report
back during NS identity.
Linux driver uses min(nawupf, npwg) to set physical_block_size.

Similarly in scsi_bdev fill lbppbe in READ_CAP16 response
based on spdk_bdev.phys_blocklen.

Fixes #1884

Signed-off-by: Swapnil Ingle <swapnil.ingle@nutanix.com>
Change-Id: I0b6c81f1937e346d448f49c927eda8c79d2d75c0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7739
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-05-06 07:14:40 +00:00
Ben Walker
63642befd3 Revert "nvmf: Support physical block size if exposed by bdev"
This reverts commit b32cfc467b.

This commit fails the ABI checks and only got through because the checks
were disabled until 21.04 hit.

Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: Id26b8f8ba551193d99b1ccbd31b35378b4095a20
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7731
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-05-04 00:48:34 +00:00
Swapnil Ingle
b32cfc467b nvmf: Support physical block size if exposed by bdev
Today the in-guest nvme device shows physical_block_size=512 even though
the backend iSCSI bdev supports physical_block_size=4K

iSCSI targets exposes physical block size using
logical_block_per_physical_block_exponent in READ_CAPACITY_16

NPWG is one of the way to let Linux nvme driver set
physical_block_size of the nvme block device.

This patch adds spdk_bdev.phys_blocklen which is updated if the iSCSI
backend exposes physical_block_size.
Later phys_blocklen is used in nvmf to set NPWG and NAWUPF to report
back during NS identity.
Linux driver uses min(nawupf, npwg) to set physical_block_size.

Similarly in scsi_bdev fill lbppbe in READ_CAP16 response
based on spdk_bdev.phys_blocklen.

Fixes #1884

Signed-off-by: Swapnil Ingle <swapnil.ingle@nutanix.com>
Change-Id: I0b6c81f1937e346d448f49c927eda8c79d2d75cf
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7310
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-05-03 18:59:34 +00:00
Alexey Marchuk
8e85b675fc sock: Add new params to configure zcopy for server, client sockets
When zcero copy send is enabled and used by initiator,
it could significantly increase latency in some payloads.
To enable more fine graing configuration of zero copy
send feature, add new parameters enable_zerocopy_send_server
and enable_zerocopy_send_client to spdk_sock_impl_opts to
enable/disable zcopy for specific type of sockets.
Exisiting enable_zerocopy_send parameter affects all types
of sockets.

Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Change-Id: I111c75608f8826980a56e210c076ab8ff16ddbdc
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7457
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-04-27 08:13:32 +00:00
Niklas Cassel
d69349af6b nvme/fio_plugin: fix zone_append option with multiple files per thread
Each fio thread can have multiple files that it writes to.
Which is why the per thread spdk_fio_setup() fio callback does
for_each_file() {...}.

One of these files can be e.g. a zoned namespace with append support,
another file could be a zoned namespace on another controller without
append support, and a third file could be a conventional namespace
(which never supports the zone append command).

Right now, we will return a fatal error if a thread has e.g. a zoned
namespace (with append support) together with a conventional namespace.

Instead of returning a fatal error, enable zone append only on the
namespaces that support zone append, and allow namespaces that do
not support zone append to continue as usual (using regular writes).

Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Change-Id: Ic6456d408cbe91563acd337a4b70c6e871fe34c6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7611
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-04-26 16:16:42 +00:00
Niklas Cassel
09dd961b35 nvme/fio_plugin: fix report_zones
Commit f69367c788 ("fio_nvme: defer qpair allocation to file_open
callback") moved the qpair allocation from spdk_fio_setup() to
spdk_fio_open(). This broke spdk_fio_report_zones(), which needs a
qpair in order to get the initial state of the zones.

setup_files() in FIO calls td->io_ops->setup() (spdk_fio_setup()),
followed by zbd_init_files(), which calls zbd_init_zone_info(),
which calls zbd_create_zone_info(), which calls parse_zone_info(),
which calls zbd_report_zones(), which calls td->io_ops->report_zones()
(spdk_fio_report_zones()).

i.e. spdk_fio_report_zones() will always be called directly after
spdk_fio_setup(). .report_zones() is even called before the per
thread ioengine .init() callback.

Therefore, spdk_fio_report_zones() is called before the ioengine
.open_file() callback.

This is done in order to ensure that all threads will share the same
zbd_info struct, which contains the per zone locks.

Since SPDK nvme ioengine no longer initializes the qpairs in .setup(),
create a temporary qpair in .report_zones().

Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Change-Id: Ic376ac7844e40fceff092900ae7e4714bccf38e6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7590
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-04-26 16:16:42 +00:00
Niklas Cassel
e9e97cb620 nvme/fio_plugin: fix initial_zone_reset
Commit f69367c788 ("fio_nvme: defer qpair allocation to file_open
callback") moved the qpair allocation from spdk_fio_setup() to
spdk_fio_open(). This broke --initial_zone_reset, which needs a qpair
in order to perform the initial zone reset.

While at it, move the initial zone reset from spdk_fio_setup() to
attach_cb(), as this is where all the other fio options are verified.
By placing it in attach_cb(), after the duplicated file check, we
avoid the need to loop through the whole fio_thread->fio_qpair list.

Since SPDK nvme ioengine no longer initializes the qpairs in .setup(),
create a temporary qpair, if the --initial_zone_reset option was used.

Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Change-Id: I7950304c58aef3ec783f7cd99cfb1e7d7817a197
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7589
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2021-04-26 16:16:42 +00:00
Tomasz Zawadzki
a8d700419d examples/interrupt_tgt: mark reactor_set_interrupt_mode RPC private
The RPC is for testing purposes, as such does not need to provide
documentation in SPDK proper.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ia153ad326e4c2ac43b511f9c2fba4aa95f1e3295
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7607
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-04-26 08:39:59 +00:00
Liu Xiaodong
74db63dcda interrupt_tgt: add RPC to set interrupt mode
Plugin RPC method 'reactor_set_interrupt_mode' is added
to set specific reactor run in interrupt mode or back
to poll mode.

Based on this RPC, scenarios later can be implemented
to verify interrupt mode functionality.

Change-Id: I9785e6942299bab7c949780faf18216e293f0b1d
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7347
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-04-26 06:58:36 +00:00
Jim Harris
6c75f0a266 env_dpdk: save hotplug timeout timestamps separate from rte_devargs
DPDK recently clarified some semantics on the rte_devargs 'data'
and 'args' fields.  This actually breaks our use of the 'data'
field to store the 2 second timeout timestamp for delaying
attach to newly inserted devices.  Investigating this further,
it does not seem our use of the 'data' field was valid - it just
happened to work until now.

We could use the 'args' field now.  But knowing whether to use
'args' or 'data' would then be dependent on the DPDK version.
We cannot use RTE_VERSION_NUM to decide, because this is a
compile time decision, and it is possible in shared library
use cases that we could actually link and execute against a
different version of DPDK than we built against.

So instead we will create our own env_devargs structure that
will store these allowed_at timestamps. Currently it's just
a linked list (which is exactly how DPDK does it) - we could
make it more optimal with a hash table down the road, but this
code only executes when we are doing PCI enumeration so it is
not performance critical.

Fixes #1904.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I3ee5d65ba90635b5a96b97dd0f4ab72a093fe8f7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7506
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: <dongx.yi@intel.com>
2021-04-23 08:16:24 +00:00
Monica Kenguva
204e6278d4 nvme/fio_plugin: fix nvme fio always reporting success
status code and type is inspected and reported
Fix issue #1893

Signed-off-by: Monica Kenguva <monica.kenguva@intel.com>
Change-Id: I6f181d8c9464182b23c658f4c268b900398fd751
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7567
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-04-23 07:58:25 +00:00
Krishna Kanth Reddy
e04da24cf6 examples/pmr_persistence: Add PMR example application
Added tests to verify the PMR support.

Signed-off-by: Krishna Kanth Reddy <krish.reddy@samsung.com>
Change-Id: I63f0a18647bfb8b16b8a5fbedb902c2f28b1d36d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6562
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-04-22 20:10:21 +00:00
Krishna Kanth Reddy
c821049d2f examples/identify: Get PMR size
Signed-off-by: Krishna Kanth Reddy <krish.reddy@samsung.com>
Change-Id: Id0dc422eefc6aa143e63d4630e6e74757df50e6e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6561
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-04-22 20:10:21 +00:00
Jim Harris
f69367c788 fio_nvme: defer qpair allocation to file_open callback
All jobs are created at boot, meaning the setup callback
is invoked for all jobs before any are executed.

But it may be useful to put 'stonewall' parameters in
the job file to execute a bunch of workloads in succession,
starting one workload when the previous one completes.
But since qpairs are created currently during setup, the
total number of workloads that can be expressed is limited
since qpairs for all workloads are allocated up front.

So instead defer allocation of the io qpairs until the
file_open callback.  These don't get called until the
job associated with the 'file' (in this case, the
nvme namespace) is ready to execute.

Note that we cannot free the qpairs in the file_close
callback, since fio may 'close' the file before all
I/O have been completed.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I3c60cf27c3660a3c94042c0de719f5bebdb9b417
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7481
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: <dongx.yi@intel.com>
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-04-22 01:06:02 +00:00
Alexey Marchuk
754de777a6 perf: Use max_completions when process poll group
Perf tool allows to specify g_max_completions
parameter but it is not used.

Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Change-Id: Ib46aaa949ddda1b0ba25c28849a402986a7a3d8e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7373
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-04-21 23:57:57 +00:00
Alexey Marchuk
a3080090b0 perf: Add parameter to set IOVA mode
SPDK app framework supports a CLI parameter to
specify IOVA mode, add support for this to perf

Change-Id: Id72755d02c63c443ae3bced1823fc547d9e4b050
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7372
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-04-21 23:57:57 +00:00
Alexey Marchuk
a7fefe05a5 perf: Use spdk_env_opts struct to parse args
When perf tool parses CLI arguments, it saves
each argument to a global value. Later these
values are checked for non-zero and copied to
spdk_env_opts structure. It can be simplified
by saving parsed arguments directly to env_opts
structure. It is applicable to env options only,
options specific for perf are still copied to
global variables.

Change-Id: I298392348df959d8165ea72333764c6505373fc2
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7371
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: <dongx.yi@intel.com>
2021-04-21 23:57:57 +00:00
Changpeng Liu
b46ce57e30 example/nvme: use API to return the dlfeat.read_value
Some earlier NVMe drives don't implement Deallocate
Logical Block Feature(dlfeat) read value field, but
it can return zeroes in deallocated ranges, that's
the quirk NVME_QUIRK_READ_ZERO_AFTER_DEALLOCATE, so
here we use the API to cover earlier NVMe drives.

Change-Id: I6867618c60dd5488a5d4820d663da7f074af229c
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7525
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-04-21 23:19:59 +00:00