Commit Graph

6106 Commits

Author SHA1 Message Date
Ben Walker
e7602c158f nvme: Hold namespaces in an RB_TREE
Since this is now sparsely populated, a tree is a better choice.

Change-Id: Ie66d913fa1d298de56a7d22ef55f0adf7f8803b8
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10031
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-01-14 08:35:10 +00:00
Ben Walker
b4dace738e nvme: Do not allocate inactive namespace objects
Some subsystems report a very large maximum value for the number of
namespaces, but in essentially every case the subsystem is sparsely
populated with active namespaces. To save memory, don't allocate
objects for the inactive ones.

Change-Id: I4cbeb5a7a898d3c685f4a3a9ec4c2ce45efffb92
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9898
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2022-01-14 08:35:10 +00:00
Changpeng Liu
6bc3a3e630 nvme/compliance: also delete CQ at last in a test case
For the test case, we only delete the SQ for a queue pair,
this will make SPDK NVMe driver not to send delete CQ when
shutting down the controller, so here also delete CQ at
last.

Change-Id: Ibdfb727710ca51524858158edac415e93bb1c1e3
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10528
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-01-13 08:48:10 +00:00
Yifan Bian
daacb4264a test/llvm_nvme_fuzz: add NAMESPACE ATTACHMENT and NAMESPACE MANAGEMENT
command test cases

Change-Id: I48a9a37d54269b2d7038a96dace9aefba334ede8
Signed-off-by: Yifan Bian <yifan.bian@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11011
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-01-12 08:21:14 +00:00
Konrad Sztyber
d9f1d4e1dd test/nvmf: run bdevperf in the foreground
There's no point in running bdevperf in the background only to send a
framework_wait_init.  Moreover, since bdevperf is executed with a 1s
timeout, there's a risk that it'll finish before that RPC is sent
causing a false positive failure.

Fixes #2046

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Icb5a8535d39954b7dfd3b4386cd008214d180bf3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11014
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
2022-01-12 08:20:53 +00:00
Ben Walker
1cfae16563 accel: Use vectored crc32 operations instead of chaining
Chaining may be faster, but this is really an implementation detail of
the idxd driver. Push the decision on how to implement a vectored crc
down into the individual drivers and eliminate it from the generic
framework.

Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: Iedbdc5a6dbd3f7d1674d0a83f6827588f4b6b2fb
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10291
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2022-01-12 08:20:39 +00:00
Jim Harris
932ee64b8f bdev/nvme: add bdev_nvme_stop_discovery RPC
This RPC will stop the specified discovery service,
including detaching from any controllers that were
attached as part of that discovery service.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I9222876457fc45e1acde680a7bd1925917c22308
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10832
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
2022-01-12 08:20:23 +00:00
Jim Harris
61a640b2a6 test: add nvmf/host/discovery.sh
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I7c279594f5298f9a0d7476a639866e75aea4043a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10811
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
2022-01-12 08:20:23 +00:00
Konrad Sztyber
b542979620 test/nvmf: zero-copy abort test
Extended the zcopy test with a step checking aborts.  It's similar to
`test/nvmf/target/abort.sh`, with the difference being that here the
aborted requests are executed using zero-copy.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ia25936b7097a94d278c467ce07c009091ea3a55a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10799
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-01-12 08:20:11 +00:00
Konrad Sztyber
da2fb51627 test/nvmf: zero-copy test
The test has two phases: first it runs a bdevperf with rw=verify to
check data consistency and then runs bdevperf in the background while
sending RPC requests causing the subsystem to be constanty
paused/resumed.

In-capsule data is set to 0 to make sure all IO requests are using
zero-copy.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I1c1d7eb04714c8506307cb95b6cbc5988c8946a3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10797
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
2022-01-12 08:20:11 +00:00
Konrad Sztyber
6631c2a8aa nvmf/tcp: initialize zcopy phase in nvmf_tcp_req_get
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ia74148fb36733deaf7b2f833ac0247859311a805
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10794
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-01-12 08:20:11 +00:00
Konrad Sztyber
a50a70ecdf nvmf: abort outstanding zcopy reqs in qpair disconnect
Zero-copy requests are kept on the outstanding queue for the whole
duration of the request - from the initial zcopy_start submission to the
completion of zcopy_end.  This means, that there's a period in which a
request doesn't wait for a completion from the bdev layer, but is still
on the oustanding queue (after zcopy_start callback, before zcopy_end
submit).  If a qpair gets disconnected while a request is in this state,
we need to manually force its completion, as otherwise it might hang
indefinitely (e.g. waiting for host data).

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I53731b8e363b725efa564ca3c7d89b46f5fb2a24
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10793
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
2022-01-12 08:20:11 +00:00
Konrad Sztyber
974a32b72e nvmf: resume queued zcopy requests
The zero-copy requests can also be queued when a subsystem is paused, so
we need to properly resume and submit them by using zcopy_start.

Since only requests that haven't received the zero-copy buffer (i.e.
before zcopy_start was called) can be queued, we don't need to bother
with checking zcopy_phase.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ie629688f6961eb2ae05741df496720b91be4d80d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10792
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-01-12 08:20:11 +00:00
Shuhei Matsumoto
521a9bb22c bdev/nvme: Fix race between failover and add secondary trid
We sort secondary trids to avoid using disconnected trids for failover.
However the sort had a bug.

This bug was found by running test/nvmf/host/multipath.sh in a loop.

Verify the fix by adding unit test.

Fixes #2300

Signed-off-by: Shuhei Matsumoto <shuheimatsumoto@gmail.com>
Change-Id: I22b0ede4d2ef98b786c3e0d1f5337a2d568ba56d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10921
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2022-01-10 22:18:46 +00:00
Changpeng Liu
e57252f98a test/app: add missed spdk_env_fini() or spdk_app_fini() when exiting
Change-Id: I9d511713277ea85d34e0a0e59b570fcbfa014fb1
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10911
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2022-01-10 18:36:14 +00:00
Jim Harris
b68f2eeb0b bdev_nvme: add bdev_nvme_start_discovery RPC
This patch adds the framework for a discovery
service in the bdev/nvme module.

Users can specify an IP/port of a discovery service.
The bdev/nvme module will connect to a discovery
controller, get the discovery log page, and then
register for AERs.  It will connect to each
subsystem specified in the initial log page.
AER completions will trigger fetching the log
page again, at which point new subsystems will
be connected to, or removed subsystems will be
detached.

This patch does the following:
* Adds the new start_discovery RPC
* Connects to the discovery controller
* Gets the discovery log page
* Registers for AERs
* Detach from discovery controllers at shutdown

Subsequent patches in this series will:
* Connect to subsystems listed in discovery log page
* Detach from subsystems that were listed in earlier
  discovery log pages but subsequently removed
* Add a stop_discovery RPC

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I54bfa896a48c5619676f156b5ea9f2d1f886c72f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10694
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
2022-01-10 15:23:39 +00:00
Jim Harris
9035986e35 test: add iptables entry to forward packets across veth bridge
It seems like some distros or kernels do not automatically
forward packets across the bridge we set up in nvmf_veth_init.
So add an iptables entry to explicitly add a forwarding rule.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I50bc203afcfee888c27c0e1d77609e554dedc61e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10810
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2022-01-10 15:23:39 +00:00
Konrad Sztyber
7a374fbc0b nvmf: make zcopy_end void
Since spdk_bdev_zcopy_end() cannot really fail (it only fails if we pass
a bad bdev_io), we can simplify the nvmf zcopy_end functions by making
them void and always expect asynchronous completion.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I6e88ac28aba13acadea88489ac0dd20d1f52f999
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10790
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Anil Veerabhadrappa <anil.veerabhadrappa@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-01-06 18:53:42 +00:00
Konrad Sztyber
92d7df1f47 nvmf: use spdk_nvmf_request_exec to submit zcopy_start
Since this path now supports sending zero-copy, use it for zcopy_start.
Additionally, it makes it possible make zcopy_start void, as it reports all errors
asynchronously via request_complete(), and remove some of the duplicated
error checks.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I41f43ce1651432d9a7d74e3680d4a3f780128a1d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10789
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-01-06 18:53:42 +00:00
Konrad Sztyber
686b9984b3 nvmf: return async/complete status in bdev zcopy operations
Additionally, the NVMe completion status is now updated and the IOs are
queued if the bdev layer doesn't have enough IO descriptors.  It makes
the zcopy operations behave similarly to the other IO operations.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I455ae781e32aa6e60d144d2c91f109bd8be46664
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10787
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-01-06 18:53:42 +00:00
Konrad Sztyber
0e09df57dd nvmf: rename zcopy operations to zcopy_(start|end)
It makes their names consistent with the bdev API.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I314051f0980b46959d6560aa25885f13b4c28f2a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10786
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Anil Veerabhadrappa <anil.veerabhadrappa@broadcom.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-01-06 18:53:42 +00:00
Konrad Sztyber
f65099d378 nvmf: remove zcopy check in spdk_nvmf_request_exec
It will make it possible to submit zero-copy requests through
spdk_nvmf_request_exec().

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ibc14fe77cd477b11ed55d1350a7486caaad81add
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10783
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-01-06 18:53:42 +00:00
Konrad Sztyber
7d23ac8657 nvmf: remove zcopy phase checks from IO functions
The code should never reach these functions for requests using
zero-copy.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: If9f30e05a43b340a982604d5b985242d63ce252b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10782
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-01-06 18:53:42 +00:00
Konrad Sztyber
aa1d039836 nvmf: zero-copy enable flag in transport opts
It makes it possible for the user to specify whether a transport should
try to use zero-copy to execute requests when possible.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I40a92b0d7a6707f4c9292795f380846acb227200
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10780
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-01-06 18:53:42 +00:00
Krzysztof Karas
39b7e10fd4 hw_hotplug.sh: fix the incorrect path to log.txt file
This test fails due to incorrect path to log.txt file needed
to detect start of hotplug application.

The change that introduced the bug:
Change-Id: Ia14507a282796ad28f067c086b9112ae6463b654

This change corrects the file path.

Change-Id: I5d43c47044f46100fa4ca7b845973c427cae237c
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10976
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-01-06 18:52:31 +00:00
Yifan Bian
89f253dd20 test/llvm_nvme_fuzz: add DELETE I/O COMPLETION QUEUE and DELETE I/O SUBMISSION QUEUE command test cases
Change-Id: Idf20d03077ea24d347170ada955dda6d92efce49
Signed-off-by: Yifan Bian <yifan.bian@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10984
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-01-06 18:52:05 +00:00
Changpeng Liu
2a6c2c289c nvmf: support static CNTLID
SPDK NVMf subsystem supports dynamic controller model, for
transports other fabrics, users should use static controller
model.

Change-Id: I364ea61a71b04d51932fd9e0e16f401a383ff67c
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10149
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-01-06 01:20:32 +00:00
Michal Berger
e0482c503e autopackage: Move extra RPM tests to nightly
This is done to save some time in per-patch testing

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I777e30ed90a9cd02ec12c0fd0c19a5616c491f6a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8637
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2022-01-05 09:24:59 +00:00
Michal Berger
f3b3c5bd84 test/common: Don't exit from within the killprocess()
This breaks the flow of EXIT traps that use this function together
with errexit. Consider:

set -e
trap 'killprocess $pid_not_found; nvmftestfini' EXIT
...

if killprocess() suddenly exits the remaining parts of the trap won't
be executed.

Also, if the process is not found, allow killprocess() to return
success - if it's gone then the main goal of the function is already
achieved. If the process terminated abruptly errexit should pick that
up regardless.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I93c00c2a2da5457affb2ed2b5bf0ebae9c4bf291
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10853
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-01-05 09:24:22 +00:00
Michal Berger
89df50bc2d scripts/core-collector: Don't limit core dump collection
We should gather the core regardless of the RLIMIT_CORE setting.
Also, don't limit number of parallel executions of the collector
in case more than 2 processes crashed (unlikely to happen anyway,
but just in case make sure we cover it). Lastly, wait a bit longer
for the potential cores.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Ic7de8f8000676122282469b729f45b235ab6f9ca
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10852
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-01-05 09:24:22 +00:00
Michal Berger
5d49965bb9 test/scheduler: Ignore failures while removing dummy cgroups
Upon removal, there's a small chance that new threads would jump into
these cgroups AFTER we read the list of PIDs to migrate. In such a
case, there may be some processes left preventing us from finally
removing the cgroup. Since we can't fully control that without
continuously reading the list, simply ignore it.

Fixes issue #2298

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I0d50b975742a8b15edd25c6b4e51ab337b017eaa
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10872
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-01-05 09:23:57 +00:00
wanghailiangx
da7c73016e test/nvmf: distinguish between two nvmf_fio cases
When we find the corresponding nvmf_fio cases in the log,
we won't be confused.

Change-Id: If37372d287b0ac70142234a4e29208a38d027fbd
Signed-off-by: wanghailiangx <hailiangx.e.wang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10680
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-01-05 09:23:24 +00:00
Maciej Wawryk
8dafa5632a test/nvme: add timeout to bdevperf perform_tests
By default, this timeout waiting for response is set to 60 seconds,
we want to wait a bit longer than defined RUNTIME value.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I6f91da6c3996967271dda7c49320a2b8573bc49f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10169
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-01-05 09:22:50 +00:00
Yifan Bian
dd6a8a80df test/llvm_nvme_fuzz: add CREATE I/O SUBMISSION QUEUE command test case
Change-Id: I4353543acf8662b14f35d36b30b2eb1fad88a871
Signed-off-by: Yifan Bian <yifan.bian@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10970
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-01-05 08:49:15 +00:00
Yifan Bian
0ac16751a5 test/llvm_nvme_fuzz: add CREATE I/O COMPLETION QUEUE command test case
Change-Id: Id5c6e81c2691942380450f2ebabdf818494f76a2
Signed-off-by: Yifan Bian <yifan.bian@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10953
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
2022-01-05 02:40:31 +00:00
Changpeng Liu
c7f33deaae test/llvm_nvme_fuzz: don't use NVME_OPC_FABRIC for ADMIN fuzz tests
When doing ADMIN fuzz tests, the NVME_OPC_FABRIC is special for
fabric transports, so here we pick up a different one.

Change-Id: I00376c08eb9eabdb109656d631615eeb37c9d09c
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10847
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-01-05 02:39:04 +00:00
Mao Jiang
dc1386e686 nvme/overhead: add vfio_user transport support
Change-Id: Ib696c7787151e4898b63f57749bd134333e94a23
Signed-off-by: Mao Jiang <maox.jiang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10413
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2022-01-04 07:53:00 +00:00
junwenx
f9c496b8e2 test/vfio_user: Restart vm, check if memory region register/unregister failed
Signed-off-by: Jun Wen <junx.wen@intel.com>
Change-Id: Ib18e8c3971da53f91de5352eaa476b4acd0ce4db
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10377
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2021-12-31 10:10:24 +00:00
Mao Jiang
a0d255ac02 test/vfio_user: Build vfio-user test directory
Add test script for launching VMs with vfio-user transport,
this test script not active until vfio-user capable qemu available,
new flag SPDK_TEST_VFIOUSER_QEMU will decide case whether to run.

Change-Id: I9b885cec797fe3bb28860f1ec436c31a6fa2f131
Signed-off-by: Mao Jiang <maox.jiang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10096
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2021-12-31 10:10:24 +00:00
Bianyfan
5d5b293387 test/llvm_nvme_fuzz: add ABORT command test case
Change-Id: Ibfdb75771abc20ecf0c436314d26c1791cf0a518
Signed-off-by: Yifan Bian <yifan.bian@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10922
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-12-31 09:21:50 +00:00
Alexey Marchuk
3c4a68cafc nvme: Do not create IO qpair during ctrlr initialization
If nvme ctrlr is resetting or initializing, free_io_qids
bitmap is already freed or not created yet. In that case
an attempt to create IO qpair leads to segmentation fault.

Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Change-Id: I6a97bf81d5a568db20d23b3f88cf01e994ba42e3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10827
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuheimatsumoto@gmail.com>
2021-12-27 08:43:03 +00:00
Alexey Marchuk
eb09178a59 nvme/rdma: Correct qpair disconnect process
In current implementation RDMA qpair is destroyed right after
disconnect. That is not graceful qpair shutdown process since
there can be requests submitted to HW and we may receive
completions for already destroyed/freed qpair.

To avoid this, only disconnect qpair in ctrlr_disconnect_qpair
transport callback, all other resources will be released in
ctrlr_delete_io_qpair cb.

This patch is useful when nvme poll groups are used since in
that case we use shared CQ, if the disconnected qpair has WRs
submitted to HW then qpair's destruction will be deferred to
poll group.

When nvme poll groups are not used, this patch doesn't change
anything, in that case destruction flow is still ungraceful.
However since CQ is destroyed immediately after qpair,
we shouldn't receive any requests which point to released
resources. A correct solution for non-poll group case
requires async diconnect API which may lead to significant
rework.

There is a bug when Soft Roce is used - we may receive
a completion with "normal" status when qpair is already
disconnected and all nvme requests are aborted. Added
a workaround for it.

Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Change-Id: I0680d9ef9aaa8737d7a6d1454cd70a384bb8efac
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10327
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Shuhei Matsumoto <shuheimatsumoto@gmail.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-12-23 08:44:40 +00:00
Konrad Sztyber
b6fba11db7 test/nvmf: set max adminq size in abort test
Set the maximum admin queue size to 256 to make sure it can hold the
number of admin requests required by the abort application.

This patch fixes the following error and actually allows the test to
start:

ctrlr.c: 720:_nvmf_ctrlr_connect: *ERROR*: Invalid SQSIZE for admin queue 135 (min 1, max 127)

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I4042481ffb2792381ef2e4ad295be067a610bd7e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10771
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-12-23 08:41:41 +00:00
Maciej Wawryk
257d06f7bc test/openstack: restart whole devstack before tests
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I126039482b5c9e3caba6565b067c747203dea476
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10822
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
2021-12-23 08:36:21 +00:00
Changpeng Liu
0bc229b314 test/llvm_nvme_fuzz: add IDENTIFY command test case
Also name the fuzzer test case with `fuzz_admin` prefix
for ADMIN commands.

Change-Id: I6e5eeb71a5f795fee8afba034f3ad436220e3c20
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10815
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-12-23 08:35:54 +00:00
Changpeng Liu
7bb78fb95b test/llvm_nvme_fuzz: print '-D' usage menu
Fix #2286.

Change-Id: Iadf7b58618ee94bc95bea705b4549ed199599149
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10814
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-12-23 08:35:54 +00:00
Michal Berger
977a53573f test/scheduler: Use separete cgroup for the tests
Two cgroups are created prior running the scheduler tests:
  - /cpuset/spdk
  - /cpuset/all

/cpuset/spdk is the cgroup dedicated for the tests, i.e., the SPDK
processes executed along the way. The resources consist of the cpus
that are initially picked up by isolate_cores.sh.

/cpuset/all is the "dummy" cgroup that holds most of the remaining
processes that run on the target system - "most" since not every
process (especially kernel threads) can be migrated between cgroups.
This cgroup's resources include all the online cpus except those
selected for the /cpuset/spdk.

This should allow for lowering the noise on the target SPDK's cpus
and make sure that load on each cpu is generated exclusively by the
SPDK.

Fixes issue #1950

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Ic45149f55052ff03bead0b9bea086f95c87ea75d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10584
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-12-23 08:33:15 +00:00
Michal Berger
1fd9dccf78 test/scheduler: Add cgroups routines
These functions are meant to help in isolating processes to make sure
SPDK is exclusively run on the dedicated cpus.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Ie7575ce4ef68c48f6f63dbedace34f23fdb8441c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10583
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-12-23 08:33:15 +00:00
GangCao
10f32b9f19 lib/blob: do not assume realloc(NULL, 0) returns a not-NULL value
There is situation that num_extent_pages is zero and original pointer is
also NULL, the realloc() could return a Not NULL pointer.

Related UT has been added and updated.
1) In the default allocation (num_clusters == 0), the extent_pages is not allocated as expected.
2) In the thin provisioning allocation (num_clusters != 0), the extent_pages will be allocated if extent_table is used.

More related information as below:

The crux of the problem is that according to POSIX:

realloc: "If ptr is NULL, then the call is equivalent to malloc(size)"
malloc: "If size is 0, then malloc returns either NULL or a unique pointer value that can later be successfully passed to free"

blobstore was relying on realloc(NULL, 0) always return a unique pointer value, and not NULL.  This is not portable behavior.

Change-Id: Ibc28d9696f15a3c0e2aa6bb2371dc23576c28954
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10470
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-12-20 18:14:06 +00:00
Ben Walker
fca4262987 nvme: Remove nvme_ns_update
In the one place this was called, we can call nvme_ns_construct
instead. There's no harm in re-fetching the identify pages.

Change-Id: I91292ff9650bdc7edd5588a05837b671dcac1922
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10102
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-12-20 08:49:41 +00:00
Michal Berger
b9cac1d4e7 test/nvmf: Group all non-uring-dependent tests together
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I3ba8bfc650103ed19090d6be969d38ed01ef2e9d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10175
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2021-12-20 08:46:13 +00:00
Michal Berger
1fdfb7e332 test/dd: Simple tests utilizing uring RPCs
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: If9c5d0071d30b3a5ac40ea5085dbf564b98cb667
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9947
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-12-20 08:46:13 +00:00
Michal Berger
bf815486fc pkgdep/git: Patch for the ice driver targeting latest linux kernel
These changes are needed for the 5.15.x kernel that's currently being
shipped with fedora35.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I7e79bff77a93f81bd93376b84f4ef7f7230cc322
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10610
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-12-20 08:45:00 +00:00
Krzysztof Karas
0c15ca6a84 pkgdep/git: bump up the version of nvme-cli
Bump up the version of nvme-cli to 1.15.

Change-Id: I1ed024bfd70027f947962ec76dfaf94c1660957f
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9912
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-12-20 08:42:30 +00:00
Peng Lian
4c1757ffb9 nvmf: update discovery log when removing hostnqn
In NVMF Revision spec 1.1a, discovery log should be updated
when removing hostnqn of subsystem.

Update unit test to check the discovery log when removing
hostnqn and destroying subsystem.

Signed-off-by: Peng Lian <peng.lian@smartx.com>
Change-Id: I51c597a2493295a677a7aa68e4f13a887f7e1140
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10668
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-12-16 08:52:20 +00:00
Michal Berger
17a662f9fe test/json_config: Simplify events check
Consider multiple nvme devices loaded to subsystem prior running
the checks.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Ie4ce8be6ec22c26c1d46ded42ded468b92d6cc57
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7868
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
2021-12-16 08:49:53 +00:00
Anil Veerabhadrappa
68f0c6160a ut/fc : fix fc_ls_ut compilation failure
This regression was introduced when 'accept' was removed from
spdk_nvmf_transport_ops structure.

Signed-off-by: Anil Veerabhadrappa <anil.veerabhadrappa@broadcom.com>
Change-Id: I5d880791db258a97a1861dbd841e97a7c068ce12
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10676
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-12-16 08:43:39 +00:00
Jim Harris
050565e5ab test/nvmf: fuzz nvmf target using LLVM's libFuzzer
LLVM provides libFuzzer which does coverage-guided
fuzzing of a library or application under test.  For
SPDK, we can use this as a new and better way to
generate random commands to the SPDK nvmf target.

By default, libFuzzer provides the main() and your
source file just provides the function called by
LLVM for each iteration of random data.  But this
doesn't really work for SPDK since we need to start
the app framework and the nvmf target.  So we
specify -fsanitizer=fuzzer-no-link, explicitly
specify the location of the fuzzer_no_main library
and then call LLVMFuzzerRunDriver to start the
fuzzing process once we are ready.

Since this is all coverage-guided, we invoke the
fuzzer inside the nvmf target application.  So this
patch creates a new target application called
'llvm_nvme_fuzz'. One core is needed to run the
nvmf target, then we spawn a pthread to run the
fuzzer against it.

Currently there are two fuzzers defined.  Fuzzer 0
does random testing of admin commands.  Fuzzer 1
is focused solely on GET_LOG_PAGE and fuzzes a
smaller subset of the bytes in the spdk_nvme_cmd.

Additional fuzzers can be added in the future for
other commands, testing I/O queues, data payloads,
etc.

You do need to specify CC and CXX when running
configure, as well as specify the location of the
special clang_rt.fuzz_no_main library. The path of
that library is dependent on your clang version and
architecture. If using clang-12 on x86_64 platform,
it will look like:

CC=clang-12 CXX=clang++-12 ./configure --with-fuzzer= \
  /usr/lib/llvm-12/lib/clang/12.0.0/lib/linux/libclang_rt.fuzzer_no_main-x86_64.a

Then just do the following to demonstrate the fuzzer
tool.

make
test/nvmf/target/llvm_nvme_fuzz.sh --time=60 --fuzzer=0

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iee0997501893ac284a3947a1db7a155c5ceb7849
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10038
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-12-15 04:32:05 +00:00
Changpeng Liu
723adbaf32 UT/vfio-user: fix clang-12 compilation error
Add missed STUBs.

Change-Id: I20989bf4ea66720d62f8ecc9668bb8f74e459666
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10638
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2021-12-15 04:32:05 +00:00
Karol Latecki
654aa4b975 test/blobfs: update rocksdb version
Use one of the recent branches for rocksdb. This allows
to build with less -Wno-error options. Adjust C flags
for more recent GCC versions.

Change-Id: Icf176d909a92195da2ae744a53e0cb53530b4573
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10628
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-12-14 13:23:30 +00:00
Jacek Kalwas
43022da379 nvmf: remove accept poller from generic layer
Not every transport requires accept poller - transport specific
layer can have its own policy and way of handling new connection.
APIs to notify generic layer are already in place
 - spdk_nvmf_poll_group_add
 - spdk_nvmf_tgt_new_qpair

Having accept poller removed should simplify interrupt mode impl
in transport specific layer.

Fixes issue #1876

Change-Id: Ia6cac0c2da67a298e88956734c50fb6e6b7521f1
Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7268
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-12-14 13:18:33 +00:00
Krzysztof Karas
bace05499a logs: create a log level flag in abort and hotplug applications
Bloated log files appear during some tests
(nvmf_phy_autotest, during nvmf_abort test fot example),
what makes parsing them inconvenient. This change aims
to disable logs that cause this issue by adding a new flag
for SPDK target and disabling notice level logging completely
(using --silence-noticelog proved insufficient).

Fixes: #2149

Change-Id: Ibbad92d87d90fe73c23d6027e0ff8ec49b0393c2
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10311
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-12-14 09:12:01 +00:00
Michal Berger
ce2a2aed0b pkgdep/git: Encode whitespace in the ice driver's URL
Latest curl versions started to treat such URLs as invalid. See:
https://github.com/curl/curl/pull/7073

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Ic5ab34a566f89f411ec40cbcb8de57a8d2f3ea88
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10607
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-12-14 09:09:38 +00:00
Mao Jiang
6fbaa11534 nvme/deallocated_value: add vfio-user transport support
Change-Id: I1a87d9245ba8a4f4e01d510cae4a318fa3323ca2
Signed-off-by: Mao Jiang <maox.jiang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10257
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-12-14 09:09:18 +00:00
Jim Harris
59f3cdacb1 nvmf: don't always update discovery log when adding hosts
If a subsystem has no listeners, then there is no need
to update the discovery log when adding a host, or setting
a subsystem to allow all hosts.

This eliminates some unnecessary discovery log update
notifications, especially when setting 'allow any hosts'
on a subsystem immediately after it is created (and before
it has any listeners).

Update unit test to check the adding a host to a
subsystem without listeners does not rev the genctr.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I63dab5df564269e574bb925890088f52063aa378
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10546
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-12-10 17:32:18 +00:00
Jim Harris
3867f83dea test/nvmf: add local var for hostnqn string
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ia967512bfcc5d7b1df15b6f6b5c132f21d601dce
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10563
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-12-10 17:32:18 +00:00
Jim Harris
9ac2cf7ff0 nvmf: don't update discovery log on subsystem create/delete
The discovery log isn't updated when a subsystem is created
or deleted, it's only updated when a listener for a
subsystem is added or removed.

So remove the nvmf_update_discovery_log() in the subsystem
create and delete paths. They just generate extra AER
completions that potentially cause the host to do unneeded
work.

Note that if a subsystem is deleted with active listeners,
the subsystem delete path will remove each of the listeners
before deleting the subsystem itself.  So the discovery log
will still get updated when those listeners are removed.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Id01bbfa3b24d3e1279a614a2fd60be41387a03b1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10545
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-12-10 17:32:18 +00:00
Karol Latecki
17cd6c3488 test/nvmf: increase malloc size in filesystem test
btrfs.tools v5.15.1 in Fedora 35 complains about target
device being too small to create a filesystem.
Let's increase the bdev size to this test.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: Iedf59328b91516693c88af8c5d47dde849394337
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10614
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-12-10 09:18:26 +00:00
paul luse
fbb24d0ebe lib/accel: remove batching from the framework and plug-in modules
Batching will be made available for DSA specifically through the new
idxd_perf tool.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Ic51d9ad3692074805b1ffa705cea8be35737c778
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9846
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-12-08 16:35:40 +00:00
Shuhei Matsumoto
215518069a bdev/nvme: nvme_ctrlr_create() gets prchk_flags from nvme_async_probe_ctx
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Id3deca8e0aba23299347a6aee6f0f44ee683556e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10555
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-12-08 08:31:24 +00:00
Shuhei Matsumoto
696ad465d7 bdev/nvme: Remove the failover_in_progress flag from struct nvme_ctrlr
The failover_in_progress flag is used to decide the return value of
bdev_nvme_failover().

bdev_nvme_delete() calls bdev_nvme_failover() with remove=true to remove
nvme_ctrlr->active_path_id. However bdev_nvme_failover() returns zero
if nvme_ctrlr->failover_in_progress is true. bdev_nvme_failover() may
return zero even if it does not remove nvme_ctrlr->active_path_id.

The following will be better.

bdev_nvme_failover() returns -EBUSY if nvme_ctrlr->resetting is true,
and the caller repeats calling bdev_nvme_failover() until the target trid
becomes alternative path or bdev_nvme_failover() returns zero.

To do that, the failover_in_progress flag is not necessary any more.

Removing the failover_in_progress will also simplify the following
patches to unify ctrlr reset and failover.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I57ab944beb1d06ea4def144c81c69705860de35f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10441
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-12-08 08:31:24 +00:00
Shuhei Matsumoto
7cc66c0ab1 bdev/nvme: Check if ns can be shared when configuring multipath
We had not checked the bit 0 of the Namespace Multipath I/O and
Namespace Sharing Capabilities (NMIC) field in the Identify Namespace
data structure.

If the bit 0 of the NMIC is zero, it is likely that namespaces are not
identical.

We should check if the value of the NMIC first, and do it in this patch.

Additionally, it is not usual if the bit 0 of the CMIC and the bit 0 of
the NMIC do not match. So in unit tests rename the parameter multi_ctrlr
by multipath for ut_attach_ctrlr() and use it for the value of the NMIC.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I6aa7cbcc99be2507dbf18930f7b585a9ea7d0f90
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10380
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-12-08 08:31:24 +00:00
Shuhei Matsumoto
8afa746b4d bdev/nvme: Use new APIs in a reset ctrlr sequence
Replace the spdk_nvme_ctrlr_reset_async() and spdk_nvme_reset_poll_async()
calls by the spdk_nvme_ctrlr_disconnect(), spdk_nvme_ctrlr_reconnect_async(),
and spdk_nvme_ctrlr_reconnect_poll_async() calls in a reset ctrlr
sequence.

spdk_nvme_ctrlr_disconnect() can fail if ctrlr is already resetting or
removed. But both cases are not possible. reset is controlled and the callback
to the hot remove is called when the ctrlr is hot removed. So we assume
spdk_nvme_ctrlr_disconnect() always succeed.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I1299e198597b2a2110f80b9a868e2dae015682ee
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10092
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-12-08 08:31:24 +00:00
Michal Berger
dbc1c48aeb test/nvme/zns: Fix jq filter
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: If3a6e9b963393c0a4501fa7f10cabaf12357c0b1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10562
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2021-12-08 08:06:36 +00:00
Mao Jiang
1f085e8e23 example/nvme/hello_world: add support for vfio-user transport
Change-Id: I280fc2cd84f577a71dda00dbc0f72f01fead6fba
Signed-off-by: Mao Jiang <maox.jiang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10182
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-12-07 08:22:13 +00:00
Changpeng Liu
91bd8744a9 nvme/compliance: add a test case for GET LOG PAGE with LPO
Also consolidate Log Page Offset test cases together into a
new test function.

Change-Id: I0f9d25f1f5c3aa2238caf5766a9084bd561aefd1
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10216
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2021-12-06 23:17:07 +00:00
Changpeng Liu
f582de8bb3 nvme/compliance: add a case to verify PC bit of CREATE IO SQ
Change-Id: I1a2fa8e818835889999fd615e8c192ffe2ef735d
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10154
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2021-12-06 23:17:07 +00:00
Changpeng Liu
5f061fe939 nvme/compliance: rename existing test cases
With more test cases added, we may not know exactly how many
tests cases are covered by the compliance test tool, so here
we use `Command Type` + `Command` + `Test Case` as the function
name of each test case.

Change-Id: I9d1f90c2463d15657133043fdf70d8124ef3fd87
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10152
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2021-12-06 23:17:07 +00:00
Changpeng Liu
b769dcd4fd nvme/compliance: add a case to test CREATE IO SQ with out of range CQID
Change-Id: Ie1c80d33d7fcc704321948a4b1f713f6256dc6e5
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10151
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-12-06 23:17:07 +00:00
Changpeng Liu
632c8d5613 nvme: make get INTEL log pages can be executed asynchronously
Also we don't treat exceptions when getting INTEL log pages
as a fatal error, the initialization will still contine.

Change-Id: Ic2fd2be510fde2679c1546482934d0a180266936
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10341
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-12-06 23:17:07 +00:00
Michal Berger
d2fe6d7d96 test/nvme/zns: Add simple fio test for zoned nvme devices
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I2ecad040bcaa573ace5ea2adfb1c0735d8388ce2
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9182
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-12-06 08:34:46 +00:00
Michal Berger
0231fdc770 autotest: Skip use of any zoned nvme devices
Our tests, especially those which use nvme block devices for various
use-cases, won't be able to perform successful IO on such devices.
The idea is to skip them for now and introduce basic, dedicated,
tests for zoned nvmes in oncoming patches.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I67baad5c85c662921e3327f2101180283c89e96c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9181
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-12-06 08:34:46 +00:00
Michal Berger
ea71df4f48 scripts/vagrant: Drop OCSSD awareness from functional tests
This also translates into switching fully to upstream QEMU for the
vagrant setup.

This is done in order to move away from OCSSD and SPDK's qemu fork
and align with what upstream QEMU supports. Main changes touch the
way how nvme namespaces are configured. With >= 5.2.0 it's possible
now to configure multiple namespace under single nvme device. Each
namespace requires a separate disk image to work with. This:

-b foo.img,nvme,1...
-b foo.img
-b foo.img,,..

Will still configure nvme controller with a single namespace attached
to foo.img.

This:

-b foo.img,,foo-ns1.img:foo-ns2.img

Will configure nvme controller with three namespaces.

Configuring nvme controller with no namespaces is possible via:

-b none ...

Note that this still allows to define other options specific to nvme
controller, like CMB and PMR. E.g:

-b none,nvme,,true

This will create nvme controller with no namespaces but with CMB
enabled.

It's possible now to also request for given controller to be zoned.
Currently if requsted, all namespaces under the target controller
will be zoned with no limit set as to max open|active zones.

All nvme devices have block size fixed to 4KB to imititate behavior
of the SPDK's qemu fork.

Compatibility with spdk-5.0.0 fork is preserved in context of setting
up namespaces so this:

-b foo.img,nvme,2

is valid as long as the emulator is set to that of spdk-5.0.0's.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Ib5d53cb5c330c1f84b57e0bf877ea0e2d0312ddd
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8421
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-12-06 08:34:46 +00:00
Evgeniy Kochetov
1fd2af0150 nvmf/ctrlr_bdev: Set DNR bit in status for failed NVMe passthru
When NVMe passthru command (IO or admin) fails on submission (e.g. it
is not supported), set DNR bit in completion status field. There is no
sense in retrying the command in this case.

Signed-off-by: Evgeniy Kochetov <evgeniik@nvidia.com>
Change-Id: I55960c128bd9fc31f6defef0b9832259a71684b1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8578
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-12-03 08:13:52 +00:00
Evgeniy Kochetov
d03b31c61f nvmf/ctrlr_bdev: Fix status code for failed admin passthru command
If NVMe admin passthru command is not supported by underlying bdev,
set status code in NVMe completion to INVALID_OPCODE.

Signed-off-by: Evgeniy Kochetov <evgeniik@nvidia.com>
Change-Id: I29c4e1f8263b76b27c199cfd2d9b2474432ec70b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10517
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-12-03 08:13:52 +00:00
Evgeniy Kochetov
a9593c7981 bdev: Fail nvme passthru command if not supported by bdev
The originally detected problem is that SPDK NVMf target fails command
with invalid opcode with status code INTERNAL_DEVICE_ERROR instead of
INVALID_OPCODE. All unknown commands on IO queue are passed to
underlying block device layer as NVME_IO type. It is not checked if
this type of commands is supported and, when command fails,
INTERNAL_DEVICE_ERROR is set as status code. If command fails on
submission, status code is set to INVALID_OPCODE which is more
relevant.

This patch adds check if command type is supported to
bdev_nvme_*_passthru functions. If not supported, it is failed with
ENOTSUP.

Signed-off-by: Evgeniy Kochetov <evgeniik@nvidia.com>
Change-Id: I4d7f7639da17dd3b1dc3eee7eb1b4a4f876117a2
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8567
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2021-12-03 08:13:52 +00:00
Josh Soref
c9c7c281f8 spelling: test
Part of #2256

* achieve
* additionally
* against
* aliases
* already
* another
* arguments
* between
* capabilities
* comparison
* compatibility
* configuration
* continuing
* controlq
* cpumask
* default
* depends
* dereferenced
* discussed
* dissect
* driver
* environment
* everything
* excluded
* existing
* expectation
* failed
* fails
* following
* functions
* hugepages
* identifiers
* implicitly
* in_capsule
* increment
* initialization
* initiator
* integrity
* iteration
* latencies
* libraries
* management
* namespace
* negotiated
* negotiation
* nonexistent
* number
* occur
* occurred
* occurring
* offsetting
* operations
* outstanding
* overwhelmed
* parameter
* parameters
* partition
* preempts
* provisioned
* responded
* segment
* skipped
* struct
* subsystem
* success
* successfully
* sufficiently
* this
* threshold
* transfer
* transferred
* unchanged
* unexpected
* unregistered
* useless
* utility
* value
* variable
* workload

Change-Id: I21ca7dab4ef575b5767e50aaeabc34314ab13396
Signed-off-by: Josh Soref <jsoref@gmail.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10409
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-12-03 08:13:22 +00:00
Changpeng Liu
4518e4c34e examples/arbitration: fix wrong usage of arbitration for vfio-user transport
This fixes commit 74dcf4aa "example/nvme/arbitration: add vfio-user transport support".
For vfio-user transport, we should use static DPDK memory model for
sharing memory between client and target.  Also enable log option here.

Change-Id: Iea1b28cbf234f5fc935c54899023bdbf1733a671
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10510
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-12-03 00:46:52 +00:00
Michal Berger
2a1e635f49 test/nvme: Check active namespaces instead of the number of reported namespaces
Upstream QEMU initializes the Number Of Namespaces (NN) on each of
the emulated nvme subsystems to NVME_MAX_NAMESPACES (256) - this is
the case even for a device which doesn't have any active namespaces.

In case nvme_fio_test() finds such a device (no active namespaces)
it still attempts to run fio against it due to a false positive which
comes from looking up the Number Of Namespaces.

Adjust the check to look for strings related to active namespaces
and pick devices which report any.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I5b222f1152f2a799b0d600ca19ad13eda009878a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10503
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-12-02 08:49:38 +00:00
Michal Berger
a3ac80c105 test/config_converter: Drop config_converter tests
It's been some time since SPDK switched from ini to json,
there's no much benefit from keeping these tests around.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I0cf7fcd6587abe872553211e41bee12e14bf234f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10065
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-12-02 08:09:45 +00:00
Michal Berger
5673325a6f autobuild: Move DPDK patches into the repo
Fetching these patches from external forks of DPDK repo is a bit
cumbersome and hard to maintain so instead have them in one place
for easy apply.

All of the patches were fetched "as-is" from the used forks, with
a commit section trimmed to provide just the relevant info.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I6bf538ea15ef87fe04d5a77944de36c797cdf284
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10360
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-12-02 07:59:42 +00:00
Jim Harris
7e68d0baca nvme: configure AER for discovery controllers
Move the CONFIGURE_AER state before SET_KEEP_ALIVE to
make sure that we run the CONFIGURE_AER state for
discovery controllers.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ia4e24f6507c43e3fece06b9161ff8e0b8fa0e97d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10332
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-12-02 04:02:29 +00:00
Shuhei Matsumoto
f9fba507fe bdev/nvme: Redirect the reset ctrlr operation into nvme_ctrlr->thread
In the following patches, we want to retry reconnect if reconnect failed
in a reset ctrlr sequence but we want to delay the retry. While
we wait the delayed retry, we want to quiesce ctrlr completely.

As part of quiesce ctrlr operations, we want to pause adminq poller but
we need to do it on the nvme_ctrlr->thread.

If a reset ctrlr sequence runs on the nvme_ctrlr->thread, we can avoid
redirecting the pending destruct request at completion too.

So we redirect the reset ctrlr sequence into the nvme_ctrlr->thread.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I538b962e2a7b5cf00ebbac2a1e888482ddeeee61
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10075
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-12-01 09:20:09 +00:00
Mao Jiang
74dcf4aa58 example/nvme/arbitration: add vfio-user transport support
Change-Id: I4aff5ec209de46e4e5e279cd84ce4968df568a60
Signed-off-by: Mao Jiang <maox.jiang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10464
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-12-01 09:19:28 +00:00
Krzysztof Karas
ac1abb5be8 hotplug: add check for uio_pci_generic driver
Add a check for this driver and enable iova=pa mode when it is
detected.
Add an option for the hotplug application to force iova mode.

This is to avoid:

EAL:   Expecting 'PA' IOVA mode but current mode is 'VA', not initializing
EAL: Requested device 0000:86:00.0 cannot be used

while using hw_hotplug test.

Change-Id: I7ff819c04b1e567b5ef88fc8f551ecec901806c8
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9774
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-12-01 09:16:29 +00:00
Krzysztof Karas
1e6449d80a test/nvme/hw_hotplug.sh: respect return code of subprocess
Currently we only check return code of the pipe process and
not its subprocesses. This leads to unexpected passes during testing.

Change-Id: Ia14507a282796ad28f067c086b9112ae6463b654
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9711
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-12-01 09:16:29 +00:00
Tom Nabarro
921c92e2a5 test/config: verify extra key in config doesnt break parsing
Add autotest case to verify that additional top-level key in JSON config
file doesn't break parsing when loading and initializing subsystems from
said file. This is a regression test to help applications use the same
config file to communicate private and SPDK data.

Signed-off-by: Tom Nabarro <tom.nabarro@intel.com>
Change-Id: I3ef6fe20d21398d30fa8011e20de1524e60ee841
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10204
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-11-30 09:07:02 +00:00
Jim Harris
1c083e6200 nvme: set keep alive for discovery controllers
Discovery services using the SPDK nvme driver may
use long-lasting connections that detect AER completions
to determine when there are changes in the discovery
log. This means that we still need to send keep alives
on discovery controller admin queues. So move the
SET_KEEP_ALIVE_TIMEOUT state immediately after
IDENTIFY, and run the SET_KEEP_ALIVE_TIMEOUT state
even for discovery controllers.

Note, we need the IDENTIFY's KAS value to properly
set the keep alive timeout, so we have to keep the
IDENTIFY state before SET_KEEP_ALIVE_TIMEOUT.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I5c6403c28fb72d42629c5f9009a89c4bfd44d162
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10329
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-11-24 08:34:58 +00:00
Michal Berger
a57af488ff autotest: Add routines for cleaning up lingering SPDK processes
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I713b2908fb124c76380b66659a1f646548fe1b70
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10205
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-11-24 08:34:47 +00:00
Shuhei Matsumoto
4d9b0e3c13 test/nvmf_abort: Reduce run time of nvmf_abort from 3s to 1s to reduce noisy logs
Running the nvmf_abort test outputs so much error logs and it is very
noisy. Running nvmf_abort for 1 sec will be enough.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I2700828c228e1cc03e73675c5f3f683dbf3a57d6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10299
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-11-23 08:50:49 +00:00
junwenx
e8b093aa39 test/common: Change the version of VFIO_QEMU_BRANCH from vfio-user-v0.9 to vfio-user-v0.93
Signed-off-by: junwenx <junx.wen@intel.com>
Change-Id: I8f9126effc5e4b1f6cce76dc9256910b2252feec
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10265
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
2021-11-23 08:46:48 +00:00
Shuhei Matsumoto
50b10bc20e bdev/nvme: bdev_nvme_reset_io() redirect to the orig_thread at completion
In the following patches, bdev_nvme_reset() will execute the reset ctrlr
operation on the nvme_ctrlr->thread until completion as bdev_nvme_admin_passthru()
does. Hence change the callback bdev_nvme_reset_io_continue() to
redirect to the orig_thread by using bio. Furthermore, use bio->cpl.cdw0
to store the completion status of the reset processing. bdev_nvme_reset()
does not use bio->cpl.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I361cc44494190ba83ad6e360788d78851416c46c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10074
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-11-23 08:46:36 +00:00
Shuhei Matsumoto
b4447abf70 bdev/nvme: Retry failed admin passthru up to retry_count times
This patch supports admin passthrough retry when we get any error
with DNR=0 but ABORTED_BY_REQUEST up to retry_count times.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I1bf29570791fdbe8651fa70c4c8685bb740fb86b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9944
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-11-23 08:46:36 +00:00
Shuhei Matsumoto
a9a86a14c1 bdev/nvme: Retry admin passthru immediately if it got ctrlr path error
This patch supports admin passthrough retry when we get ctrlr path
error at completion.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ice0045b84054ec66a9db9ef23e21786d2c082b1d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9943
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-11-23 08:46:36 +00:00
Shuhei Matsumoto
35a2f4e22e bdev/nvme: Retry admin passthru a second later if any ctrlr may become available
When resetting ctrlr, adminq is disconnected first. If adminq is disconnected,
admin passthrough request is rejected with -ENXIO.

But resetting ctrlr may succeed. If resetting ctrlr succeeds, adminq is
connected again, and admin passthrough request will be
submitted successfully.

On the other hand, if ctrlr is failed, admin passthrough request is
rejected with -ENXIO. But when resetting ctrlr, ctrlr is set to unfailed.

Hence bdev_nvme_admin_passthru() skips any ctrlr which is resetting
or failed, and calls bdev_nvme_admin_passthru_complete() with -ENXIO
if no available ctrlr is found.

bdev_nvme_admin_passthru_complete() queues admin passthrough request
and retry it one second later if ctrlr is resetting.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ic748dc4faf29ebf717ae5c29dcf7c55fe2ea9243
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9942
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-11-23 08:46:36 +00:00
Mao Jiang
1fa4d590d8 test/vfio_user: add vfio-user fuzzing test
Change-Id: Ia1431a9aad103ebc80e9071b984ee152c7a27a53
Signed-off-by: Mao Jiang <maox.jiang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9891
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2021-11-22 08:36:27 +00:00
Changpeng Liu
0af4a7cd84 nvme: abort outstanding requests case by case
For DSM command, the NVMe drive may take a long time to finish it,
if we set a small timeout value for DSM command, the bdev/nvme module
will try to reset the IO queue pair when timeout happens,
in `spdk_nvme_ctrlr_free_io_qpair`, we will abort the outstanding
IO requests first, then in the `nvme_pcie_ctrlr_delete_io_qpair`,
we will poll the CQ for any requests that have been completed by
the NVMe controller, if there are NVMe completions in the CQ,
we will finish them again, thus double completions happened.

Here we rename `nvme_qpair_abort_reqs` to `nvme_qpair_abort_all_queued_reqs`,
so the common layer will just abort queued request, and let each
transport to abort outstanding requests case by case.

Fix #2233.

Change-Id: Icae6214239160c615418cb514fc51cfe77b59211
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10233
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-11-22 08:35:35 +00:00
Michal Berger
0887b1ea6a pkgdep/dnf: Don't install btrfs-progs on centos8
This packages is not available anymore. In addition, centos8, by
default, is not shipped anymore with a kernel that supports btrfs.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I185218c65f06b0e2fceab169139dcdc7b63657bf
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10228
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-11-22 08:34:00 +00:00
Alexey Marchuk
64fa301f67 rdma: Update for memory map
Add a parameter which determines the owner of the
map - target or initiator. It allows to set different
access flags when creating Memory Regions

Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Change-Id: I0016847fe116e193d0954db1c8e65066b4ff82bf
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10283
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-11-19 08:29:59 +00:00
Richael Zhuang
454561bf89 test/scheduler: add test of cppc_cpufreq
current test only include intel acpi-cpufreq and pstate. Add test
of cppc_cpufreq which works on most arm platforms.

Change-Id: I5b34c15be7992e5a521d6acf7b5c30a0c4baaf9a
Signed-off-by: Richael Zhuang <richael.zhuang@arm.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9404
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-11-19 08:29:15 +00:00
Alexey Marchuk
933d56426a test/dma: Update the test to run IO for some time
This test application can now be run on several cores,
queue depth, io size, mode and time can be specified.

Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Change-Id: I6aa6b9c0319fc22ed68aafa076ab6ee81e301474
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9809
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-11-18 08:21:43 +00:00
Maciej Wawryk
3cbd2dcc6c test/vhost: fix getting device traddr
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Ic9f19145d8e1f0caa7880dfc70cba9796273bb4a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10261
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-11-18 08:18:04 +00:00
Michal Berger
c1aef9bb1b pkgdep/pkg: Install llvm-devel
It provides LLVMgold plugin libraries needed by the ld.gold for the
LTO build.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I7686244278d19c6137661ef93eb70737dc8b6006
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10258
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-11-18 08:17:41 +00:00
Michal Berger
6c5d98817a autotest: Move revert_soft_roce() outside of autotest_cleanup()
This function is part of test/nvmf/common.sh which is not included
in the test/common/autotest_common.sh. This makes autotest_cleanup
unusable unless parent script sources test/nvmf/common.sh directly.

To make its scope more obvious, put the revert_soft_roce() inside
autotest.sh.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Id8a8cf448d73abb6fe924b452f984f6e5381266f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10206
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-11-17 16:58:39 +00:00
Michal Berger
ea81ed5dbb autotest: Drop unit from the sleep
This makes it compatible with the FreeBSD version.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I99707f07f8a74e5ca90074bfc69bdaa524ed2ad4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10207
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-11-17 16:58:39 +00:00
Michal Berger
5c75e3604f pgkdep: Add llvm package
Its tools are needed for the LTO builds under clang.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I221069b2e74d71cfa811b226fb6a4c20598bef12
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10240
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2021-11-17 10:58:43 +00:00
Jim Harris
d810a7458d idxd: change NOTICELOGs to DEBUGLOGs
The NOTICELOGs really clutter the output during
application start - it's better to make these DEBUGLOGs
instead.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I3ae37d5d057d7b972017befbc0834de414b9710b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10191
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-11-17 10:58:17 +00:00
Shuhei Matsumoto
7b8e7212a6 bdev/nvme: Abort the queued I/O for retry
The NVMe bdev module queues retried I/Os itself now.
bdev_nvme_abort() needs to check and abort the target I/O if it
is queued for retry.

This change will cover admin passthrough requests too because they
will be queued on the same thread as their callers and the public
API spdk_bdev_reset() requires to be submitted on the same thread
as the target I/O or admin passthrough requests.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: If37e8188bd3875805cef436437439220698124b9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9913
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-11-17 10:58:12 +00:00
Shuhei Matsumoto
72e4a4d46a bdev/nvme: Each nvme_bdev_channel caches its current io_path
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I3ec3a588ff741cf04383e89f5a701e33bf1987a6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9894
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-11-17 10:58:12 +00:00
Shuhei Matsumoto
ae7019417e iscsi: Merge immediate data into the following R2T data
The recent changes merged multiple Data-OUT PDUs within the same
sequence into a single subtask up to 64KB.

However, they were not enough.

For a large write operation, the hardware iSCSI HBA host sent an immediate
data whose size was not block size multiples and then more solicit
data through R2T exchanges.

One example for a 64KB write operation was as follows:
  host sent SCSI Write with 5792 bytes and F = 1
  target replied a R2T
  host sent Data-OUT with 15880 bytes
  host sent Data-OUT with 11536 bytes
  host sent Data-OUT with 2848 bytes
  host sent Data-OUT with 11536 bytes
  host sent Data-OUT with 5744 bytes
  host sent Data-OUT with 12200 bytes and F = 1

The hardware iSCSI HBA host can decide the size of the unsolicited data
but the SPDK iSCSI target can require the host to send the solicited data
whose size is block size multiples.

Hence we merge immediate data to the following R2T data if the immediate
data is not more than 64KB and more R2T data come.

Add another test case to check if the fix works for the above example.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I4906b4e1a8b61e08862f4ccc27a6caf165126530
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9708
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-11-16 09:08:27 +00:00
Alexey Marchuk
f72cab94dd lib/vhost: Fix compilation with dpdk 21.11
Structure vhost_device_ops was renamed to
rte_vhost_device_ops

Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Change-Id: Ie9601099d47465536500aa37fc113aeae03a8254
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10223
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-11-16 09:06:54 +00:00
Michal Berger
0814f8662a test/dd: Add simply copy test between malloc bdevs
Validates basic functionality prior running more complex setup from
bdev_to_bdev.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I4e10cba6e0284f82e180c4a05288035c61e3fe9f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9940
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-11-15 12:00:30 +00:00
Michal Berger
6155dadbf7 test/dd: Return from gen_conf() if no configuration was provided
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I9bbd8ad2d7e369959c27433905749adab2958d07
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9939
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-11-15 12:00:30 +00:00
Michal Berger
a6bbb3820c test/dd: Lower file|bdev sizes in bdev_to_bdev tests
This is done to speed up the tests - the bigger the data to write the
more time the spdk_dd needs to finish after io is completed. This
should not affect the functional nature of the test itself.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Id086b153cffbd8045c8dbceaac64a98e358fe1ea
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9938
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-11-15 12:00:30 +00:00
Michal Berger
1b5d4b5265 test/nvmf: Limit number of tests executed under uring job
The nvmf-tcp-uring-vg job is currently fully packed with tests what
causes random timeouts across the CI. Since most of these tests are
run as part of the nvmf-tcp-vg job and are not uring dependent we
can simply skip them to save some time (~ 5 minutes).

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I5bc85afee094697b74efbef4777cb4eb1be04583
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9937
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2021-11-15 12:00:30 +00:00
Ben Walker
84688fdb1c nvme: Rename max_active_ns_idx to active_ns_count
This was sometimes used as the maximum array index and sometimes as the
maximum count. Make it consistent everywhere and give it a better name.

Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: I518efd99a7d36584624490b0b3497bb6e81ce9ac
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10101
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-11-15 11:59:59 +00:00
Kai Li
8f633fa1c3 bdev/nvme: display all ctrlrs for this bdev when dump bdev nvme controller
After multipath feature is supported, one bdev will have more than one
nvme ctrlr. Fore ease of view, display each ctrlr's trid info.

Moreover, rename nvme_bdev_ctrlr_get as nvme_bdev_ctrlr_get_by_name here
to keep consistent with nvme_ctrlr_get_by_name.

Signed-off-by: Kai Li <lik271@chinatelecom.cn>
Change-Id: I417506699bbea6ed13dac0fee942749757d2ae47
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10129
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-11-11 23:24:26 +00:00
Niklas Cassel
b7ad5b0b90 bdev/zone: add support for get zone id
In the bdev-zone API, there are a few functions that takes a zone_id:
spdk_bdev_get_zone_info(), spdk_bdev_zone_management(), and the
spdk_bdev_zone_append() functions.

The way a zoned application is usually written is that it starts off
by getting the zone report for all zones (zone_id will be sent in as 0),
and then the application will keep the whole zone report in memory.

Therefore, an application usually have access to the zone_id/zslba for
all zones. However, there are cases, e.g. when getting an error on write,
where the completion callback will only have the lba of the write that
failed.

Add a helper function that can be used to get the zone_id/slba for a
given lba. Having this helper in bdev-zone will avoid SPDK applications
needing to provide their own implementation for this.

Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Change-Id: I978335f87f7d49bc33aed81afcaa6d9f0af8a1e4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10180
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2021-11-11 23:23:35 +00:00
Shuhei Matsumoto
eb739d0364 iscsi: Fix the case that incoming data is split between data segment and data digest
When data segment size is 64KB and data digest is enabled, if
data segment and data digest are split into different two packets,
- pdu->mobj[0] became full first when reading data semgment,
- pdu->mobj[1] was allocated but unused and data digest was read.

In this case, two SCSI write tasks were submitted by mistake and
the second SCSI write task had no data.

Fix the bug in this patch.

When iscsi_pdu_payload_read() is called and pdu->mobj[0] is full,
allocate pdu->mobj[1] only if any of data segment remains to read.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I9a0c36c05f90092c3c2122a7eb91e10976830b40
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9965
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2021-11-11 23:22:57 +00:00
Changpeng Liu
ab8b25793b nvme/compliance: add a case to check NPWG and NWOS in IDENTIFY NS
Change-Id: Ie5ba6f86d7c99e20f56a11fc1623ce74bf5c6bc9
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10110
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-11-10 23:11:13 +00:00
Michal Berger
ad672ce249 test/interrupt: Fix syntax
return was not used in proper context. Let the errexit do the job to
signal a failure.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I731545c913858286009168041030b4b362cebc32
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9987
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-11-10 20:30:23 +00:00
Changpeng Liu
d83b2ee4f4 nvme/compliance: add a test case for get log page
Change-Id: I31252b016ad64ba259807dee468adc69dd545138
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10094
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-11-10 20:23:37 +00:00
Changpeng Liu
69b6b6cc47 nvme/compliance: add a test case for CREATE IO CQ
Change-Id: I13d3b382e82a8fcdb2f783973df1358555ac88c0
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10081
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-11-10 20:23:37 +00:00
Ben Walker
6ccc8c1a39 test/nvme: Reset tool now only registers active namespaces
Only active namespaces can be used for I/O.

Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: Idf63e9f5e03244c1671288038fed54831463c3eb
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10098
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-11-10 19:36:27 +00:00
Ben Walker
2dbdb9945c test/nvme: Only test non-contiguous namespaces for NVMe 1.2 or higher
This wasn't supported before NVMe 1.2

Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: Ibf19cd77e522eb11c2091a9f4956f5616876986b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10097
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-11-10 19:36:27 +00:00
Ben Walker
52e432dff2 test/nvme: Fix buffer zeroing math
This meant to zero the entire active namespace list.

Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: I2da2293b53acd57d3480cf93b052eb1520de35d4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10028
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-11-10 19:36:27 +00:00
Changpeng Liu
800e54530e nvme/compliance: add an IDENTIFY NS test case
Change-Id: Ibe62c57202549e13eeefbc2b911d2dbf79b622d7
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10079
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
2021-11-09 05:04:57 +00:00
Changpeng Liu
fded6f0d6d nvme/compliance: add a case to create all IO queue pairs
Change-Id: I99768b4d9b1ef3c81431ea069af93e0a38b05746
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10013
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-11-05 00:53:27 +00:00
Jim Harris
823388489b nvme/compliance: add test case for PROPERTY_GET
This is a reproduction vehicle for issue #2218.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ib52d959436550461cf787b918b8870218536a4c1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10009
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2021-11-05 00:53:27 +00:00
Jim Harris
ec2ad00c92 test/unit/raid: fix set-but-not-used error
verify_io() keeps track of a buf pointer, but the
buf pointer never actually gets used.  So remove
this buf pointer.

Found by clang-13.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I79dfeac7f004b56f7d4404f41b2ff18b96968a20
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10056
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-11-03 18:30:55 +00:00
Shuhei Matsumoto
84ac18e545 bdev/nvme: Update ANA state if I/O failed by ANA error
If I/O got ANA error, ANA state may be out of date. So in this case
read ANA log page and update ANA states. Mark nvme_ns to be updating
to avoid using while updating ANA state.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ia43d38b3a589c84d6d0479dedcced033e76fb194
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9458
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-10-27 11:53:31 +00:00
Shuhei Matsumoto
f3fec96c20 bdev/nvme: Protect ANA log page from concurrent reads by using an new flag
If an I/O failed by ANA error, the corresponding ANA state might be
out of date. In the following patches, for this case, read the latest
ANA log page and update the ANA state. Such reading ANA log page may be
done on multiple threads concurrently including AER ANA change.
Hence protect ANA log page by adding an new flag ana_log_page_updating
to struct nvme_ctrlr and using it.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I8bb84091d50a5fdc0d9893b585be972dfd31c0f1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9526
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-10-27 11:53:31 +00:00
Shuhei Matsumoto
43adb646b8 bdev/nvme: Retry failed I/O up to retry_count times
Add bdev_retry_count to spdk_bdev_nvme_opts and retry_count to
nvme_bdev_io, respectively.

Set type of both to int because we want use -1 for infinite retry.

Set the default value of bdev_retry_count to zero for the backward
compatibility.

bdev_retry_count is configurable by the RPC bdev_nvme_set_options.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I9bc746fcea54aa8722c76f79c70c2ae2b375aa53
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9864
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-10-27 11:53:31 +00:00
Alexey Marchuk
3d8904c66b nvmf: Add discovery filtering rules
SPDK nvmf target reports all listeners on all subsystems
in discovery pages, kernel target reports only subsystems
listening on a port where discovery command is received.

NVMEoF specification allows to specify any addresses/
transport types. Ch 5: The set of Discovery Log entries should
include all applicable addresses on the same fabric as the
Discovery Service and may include addresses on other fabrics.

To align SPDK and kernel targets behaviour, add filtering
rules to allow flexible configuration of what should be
listed in discovery log page entries.

Fixes #2082

Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Change-Id: Ie981edebb29206793d3310940034dcbb22c52441
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9185
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
2021-10-25 22:57:48 +00:00
Jim Harris
e40bd53175 nvme/pcie: only set qpair state from qpair's thread
The qpair's state member is only 3 bits of a uint8_t,
and the in_completion_context bit is another bit in that
same uint8_t.

We know that the qpair's state is only ever updated by
one thread, but it is possible that the state could
be modified by one thread, while another thread
is modifying in_completion_context.

in_completion_context is only modified by the thread
that is polling the qpair (or the qpair's poll group).
But with async mode, another thread that has a qpair
on the same PCIe controller could poll its adminq and
reap the SQ completion for the qpair that's owned by
the other thread.

So do *not* set the generic qpair state to CONNECTED
from the SQ completion callback.  Instead just set
the pcie_state to READY, and let the thread that owns
the qpair detect the qpair is READY and set the state
to CONNECTED itself.

Fixes issue #2157.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I9efc0c954504f1841e1c3890ae78211ad0d1990e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9975
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
2021-10-25 19:53:14 +00:00
Liu Xiaodong
b1906912e9 thread: reap unreg pollers in intr
When intr mode is enabled, it will be common that
poller is unregistered during interrupt processing.
Since poller unregister is a delayed operation,
mark it in spdk_thread object, and reap unregistered
pollers out of poller execution.

Fixes #2143

Change-Id: Ieb61fc7685f85af5c15e833dd1dd56f8c97a3b12
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5770
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-10-21 21:25:53 +00:00
Jim Harris
84ab68c1a3 test/scheduler: add a for_each_reactor shutdown test
The scheduler test app is ideal for testing that the
reactors properly flush any existing for_each_reactor
operations during the shutdown process.  This is
because it has no SPDK subsystems, so the reactors
stop immediately after the application is signaled to
terminate.

So start a for_each_reactor operation when the
application starts, and then keep starting another
one once the previous one completes.

This serves as a regression test for issue #2206.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I69825b50ad6f3c059da11241f58a8319199e88d5

Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9929
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-10-21 14:47:15 +00:00
Shuhei Matsumoto
5466d5d4dc test/nvmf: Extend ANA state change timeout to 20 seconds
If a connection is lost, Linux NVMe host tries reconnecting after
10 seconds delay by default. To cover this case, extend the timeout
to 20 seconds. Usually Linux NVMe host recognizes the new ANA state
within 2 seconds.

This patch is for the github issue #2081

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I61fba2febcea81951c8b29f940d93863bc31b332
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9393
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-10-21 09:17:46 +00:00
GangCao
9072c4ad0d accel: create SW Engine Channel if HW Engine not supports
Currently either HW Engine Channel or SW Engine Channel will be used.

In the case that HW Engine Channel is used while does not support related
operations like IOAT for CRC, it will shift back to the SW Engine's handle.

So that this is an issue that it still refers to the HW Engine Channel
while needs SW Eninge Channel to handle.

This patch introduces the SW Eninge Channel and always initializes there
in case that HW Engine does not support some operations.

Related UT also added to simulate the case the IOAT does not support CRC
and then SW Eninge needs to properly handle it.

Change-Id: I4ecdcd09ab669a616b37c567b45b1e6499800ec9
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9874
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
2021-10-20 23:04:38 +00:00
Alexey Marchuk
2696886c75 dma: Update translation result to hold iovec pointer
In some cases a single virtually contriguos memory
buffer can be translated to several chunks of memory.
To make such translation possible, update structure
spdk_memory_domain_translation_result to use a pointer
to iovec.
Add a single iov structure or cases where translation
is always 1:1, it will make easier translation callback
implementation. For RDMA transport translation of address
is always 1:1, so treat iovcnt other than 1 as an
error.

Change-Id: I65605575d43a490490eba72c1eb19f3a09d55ec6
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Signed-off-by: Max Gurtovoy <mgurtovoy@nvidia.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9779
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-10-20 22:55:52 +00:00