Commit Graph

1856 Commits

Author SHA1 Message Date
Konrad Sztyber
eb94193bd1 nvmf/tcp: tracepoints for zero-copy request states
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I173ccb4febf88f56da6e28e59072619755bfd130
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10801
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-06-30 13:32:30 -04:00
Konrad Sztyber
b5270ddbb6 nvmf/tcp: make request state values explicit
It makes it easier to read the logs, as the state values are printed as
integers.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I70a9e8860401c18e9305a5fc5771df0bc564d337
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10800
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-06-30 13:32:30 -04:00
Konrad Sztyber
ffd751fc0b nvmf/tcp: zero-copy support
This patch adds support for using zero-copy operations to execute IO
requests in the TCP transport.  Of course, they're only used if the
underlying bdev supports them.  Additionally, only requests with no
in-capsule-data can be executed using this mechanism.

Added several new states to accommodate for the difference in a way
zero-copy is handled.  Also, these flows very depending on the type of a
request (read or write).  It stems from zero-copy semantics: to perform
a write we need to wait for zcopy_end completion, while for reads
zcopy_end can only be submitted once we send all of the requested data
to the host.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ie02b494c24bc1acc98557cb4b02e867abf9064e4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10796
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
2022-06-30 13:32:30 -04:00
Konrad Sztyber
36581269aa nvmf/tcp: initialize zcopy phase in nvmf_tcp_req_get
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ia74148fb36733deaf7b2f833ac0247859311a805
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10794
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-06-30 13:32:30 -04:00
Konrad Sztyber
677af66f69 nvmf: abort outstanding zcopy reqs in qpair disconnect
Zero-copy requests are kept on the outstanding queue for the whole
duration of the request - from the initial zcopy_start submission to the
completion of zcopy_end.  This means, that there's a period in which a
request doesn't wait for a completion from the bdev layer, but is still
on the oustanding queue (after zcopy_start callback, before zcopy_end
submit).  If a qpair gets disconnected while a request is in this state,
we need to manually force its completion, as otherwise it might hang
indefinitely (e.g. waiting for host data).

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I53731b8e363b725efa564ca3c7d89b46f5fb2a24
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10793
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
2022-06-30 13:32:30 -04:00
Konrad Sztyber
4bc93006f0 nvmf: resume queued zcopy requests
The zero-copy requests can also be queued when a subsystem is paused, so
we need to properly resume and submit them by using zcopy_start.

Since only requests that haven't received the zero-copy buffer (i.e.
before zcopy_start was called) can be queued, we don't need to bother
with checking zcopy_phase.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ie629688f6961eb2ae05741df496720b91be4d80d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10792
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-06-30 13:32:30 -04:00
Konrad Sztyber
b6c8b8b373 nvmf: check for zcopy_start failure in request_complete
It ensures that we decrement io_outstanding counter for requests for
which zcopy_start failed.  Also, removed a note stating that such
requests are reverted to regular IO path, as this is not the case.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I8eaee88abfd94b73614b367fef9bb938c9962617
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10791
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-06-30 13:32:30 -04:00
Konrad Sztyber
7554f8174e nvmf: make zcopy_end void
Since spdk_bdev_zcopy_end() cannot really fail (it only fails if we pass
a bad bdev_io), we can simplify the nvmf zcopy_end functions by making
them void and always expect asynchronous completion.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I6e88ac28aba13acadea88489ac0dd20d1f52f999
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10790
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Anil Veerabhadrappa <anil.veerabhadrappa@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-06-30 13:32:30 -04:00
Konrad Sztyber
2e3989f04c nvmf: use spdk_nvmf_request_exec to submit zcopy_start
Since this path now supports sending zero-copy, use it for zcopy_start.
Additionally, it makes it possible make zcopy_start void, as it reports all errors
asynchronously via request_complete(), and remove some of the duplicated
error checks.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I41f43ce1651432d9a7d74e3680d4a3f780128a1d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10789
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-06-30 13:32:30 -04:00
Konrad Sztyber
041fe828f8 nvmf: complete resubmitted requests
If a request gets resubmitted and is completed immediately (i.e. the
processing function returns SPDK_NVMF_REQUEST_EXEC_STATUS_COMPLETE), the
upper layer needs to be notified via spdk_nvmf_request_complete().

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I236d6e68d1d7d83afdaa30d8bb07e2b133f43155
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10788
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-06-30 13:32:30 -04:00
Konrad Sztyber
ddc162fccd nvmf: return async/complete status in bdev zcopy operations
Additionally, the NVMe completion status is now updated and the IOs are
queued if the bdev layer doesn't have enough IO descriptors.  It makes
the zcopy operations behave similarly to the other IO operations.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I455ae781e32aa6e60d144d2c91f109bd8be46664
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10787
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-06-30 13:32:30 -04:00
Konrad Sztyber
76da2187af nvmf: rename zcopy operations to zcopy_(start|end)
It makes their names consistent with the bdev API.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I314051f0980b46959d6560aa25885f13b4c28f2a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10786
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Anil Veerabhadrappa <anil.veerabhadrappa@broadcom.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-06-30 13:32:30 -04:00
Konrad Sztyber
24d93aa9e6 nvmf: execute zcopy requests through process_io_cmd
The zcopy_start requests are now executed through
nvmf_ctrlr_process_io_cmd.  It makes the zero-copy share checks with the
regular IO path.

Note, that zcopy_end doesn't utilize this path and is directly submitted
to the bdev layer, as it doesn't need to perform these checks (they were
already verified in the accompanying zcopy_start).

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ic9ffecb15e7c351a9d60e731cc711d6500b845db
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10785
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-06-30 13:32:30 -04:00
Konrad Sztyber
fe1801711a nvmf: don't check req for zcopy when adding to outstanding queue
This will allow the zero-copy requests to share more code with the
regular IO path.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Idb57447694903a56d984f95aa2f1a3bddc3e4e82
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10784
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-06-30 13:32:30 -04:00
Konrad Sztyber
559edd6f5b nvmf: remove zcopy check in spdk_nvmf_request_exec
It will make it possible to submit zero-copy requests through
spdk_nvmf_request_exec().

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ibc14fe77cd477b11ed55d1350a7486caaad81add
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10783
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-06-30 13:32:30 -04:00
Konrad Sztyber
334df143d9 nvmf: remove zcopy phase checks from IO functions
The code should never reach these functions for requests using
zero-copy.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: If9f30e05a43b340a982604d5b985242d63ce252b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10782
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-06-30 13:32:30 -04:00
Konrad Sztyber
11ea8da9aa nvmf: rename spdk_nvmf_using_zcopy to spdk_nvmf_request_using_zcopy
It's more descriptive that way, as it's clear the function works on a
single request. Also, passing a request instead of zcopy_phase makes it
more convient to use.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: If4d7b087511e128f3590ac7b3b5adcb8ace12003
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10781
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-06-30 13:32:30 -04:00
Konrad Sztyber
63658a02f8 nvmf: zero-copy enable flag in transport opts
It makes it possible for the user to specify whether a transport should
try to use zero-copy to execute requests when possible.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I40a92b0d7a6707f4c9292795f380846acb227200
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10780
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-06-30 13:32:30 -04:00
Changpeng Liu
0589050d3e nvmf/vfio-user: use static CNTLID
Change-Id: I5c52f296bfea21968604d31f8e8ee3490317b440
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10150
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-06-30 13:32:30 -04:00
Changpeng Liu
9af65927c5 nvmf: support static CNTLID
SPDK NVMf subsystem supports dynamic controller model, for
transports other fabrics, users should use static controller
model.

Change-Id: I364ea61a71b04d51932fd9e0e16f401a383ff67c
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10149
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-06-30 13:32:30 -04:00
Changpeng Liu
02d1c8e62a nvmf/tcp: do cleanup in nvmf_tcp_poll_group_add()
we don't remove the socket fd from socket group when
nvmf_tcp_poll_group_add() return error, and when
closing the socket there is an assertion.

This was found via llvm_nvme_fuzz via TCP transport.

Change-Id: Ib4ab6fc3fc5e2bc6a9545f6ce854bae8f1157fd5
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10849
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-06-30 13:32:29 -04:00
Shuhei Matsumoto
b6e01459f3 nvmf: Disconnect qpair whose ctrlr is not set yet when removing listener
We have very frequent failures when we run test/nvmf/host/multipath.sh
in the test pool.

Call stack showed nvmf_stop_listen_disconnect_qpairs() accessed
qpair->ctrlr even if qpair->ctrlr was NULL.

nvmf_stop_listen_disconnect_qpairs() did not check if qpair->ctrlr is
not NULL before accessing qpair->ctrlr->subsys.

When a qpair is added to a poll group, qpair->ctrlr is cleared to NULL.

The test code test/nvmf/host/multipath.sh executes multiple
reconnects for path error.

So a conflict might occur between adding a qpair to a poll group and
disconnecting a qpair in a poll group.

In this case, it may be acceptable even if we disconnect a qpair whose
qpair->ctrlr is NULL. It will be better than SIGSEGV.

Fixes one of the issues in #2300

Signed-off-by: Shuhei Matsumoto <shuheimatsumoto@gmail.com>
Change-Id: I308fcb886dd410d01e3361c1850dec9a8eacbccf
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10860
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-06-30 13:32:29 -04:00
Alexey Marchuk
95130152bc nvmf/rdma: Move definitions of rxe vendor IDs to common rdma.h
These definitions will be used in the next patch to check if
device is rxe

Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Change-Id: Icc073344103991ff24fc3bb88a1ceb9867de6f6a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10727
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-06-30 13:32:29 -04:00
Alexey Marchuk
371b0c0b09 nvmf/rdma: Check device max_srq_wr when creating SRQ
Some devices may support SRQ depth lower than defaulut
value 4096

Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Change-Id: I58da0ac268a6d4c4a7e3b500ae37b8fad4810e17
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10654
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-06-30 13:32:29 -04:00
Konrad Sztyber
bce0f3bd45 nvmf/tcp: update completion cid when aborting requests
Otherwise, this field is left unassigned and the host receives some
garbage cid.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: If1e1fe8c7543bcedfbb897200696e05b71c57e0c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10770
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-06-30 13:32:29 -04:00
Konrad Sztyber
dc47cc928a nvmf/tcp: process NEED_BUFFER requests after aborting
When aborting a request in a NEED_BUFFER state, we set it's completion
status and remove it from the pending_buf_queue.  Since it's no longer
on that queue and there's no completion it's waiting for, we need to
manually kick.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I1272d441aec3b3090cd8c143a2112a8a6866fcf0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10769
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-06-30 13:32:29 -04:00
Konrad Sztyber
86ef159f89 nvmf/tcp: fix trace definition for TRACE_TCP_QP_ABORT_REQ
The tracepoint passes qpair pointer as an argument, while not specifying
it in its definitions, which makes the following assertion to fail:

trace.c:83: _spdk_trace_record: Assertion `0 && "Unexpected number of tracepoint arguments"' failed.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I315e9bf0465db7033ac0f1169536c459ac4e9250
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10761
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2022-06-30 13:32:29 -04:00
Jim Harris
2feb4c6350 nvmf: fix disconnect logic for remove_listener RPC
Currently if we remove a listener from a subsystem, we
disconnect *all* qpairs that have the same transport ID
as the listener being removed.

Fix that, since we should only disconnect qpairs from
controllers associated with the subsystem that had the
listener removed.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I6cf7422d14f23bf02ba6c4b034b172870694b3e6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10690
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2022-06-30 13:32:29 -04:00
Jim Harris
768dee6daf nvmf: add new 'id' to spdk_nvmf_subsystem_listener
This id can be used as the 'portid' for discovery
log entries.  Previously we were putting the entry
index in the portid field which was incorrect.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I9f373585fe671ba7e69eb8e07f603f8e8ac1e270
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10589
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-06-30 13:32:29 -04:00
Peng Lian
26855b047f nvmf: update discovery log when removing hostnqn
In NVMF Revision spec 1.1a, discovery log should be updated
when removing hostnqn of subsystem.

Update unit test to check the discovery log when removing
hostnqn and destroying subsystem.

Signed-off-by: Peng Lian <peng.lian@smartx.com>
Change-Id: I51c597a2493295a677a7aa68e4f13a887f7e1140
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10668
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2022-06-30 13:32:29 -04:00
Changpeng Liu
aaaf290ca7 nvmf/vfio-user: fix the compilation issue on kernel 4.9.x
Macro PCI_CFG_SPACE_EXP_SIZE isn't defined in kernel 4.9.x, so
we use a fixed value here instead.

Fix issue #2282.

Change-Id: Ife1444e919e4c1dc9328437002c15befe2f393e7
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10691
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-06-30 13:32:29 -04:00
Anil Veerabhadrappa
64849dc6e0 nvmf/fc: remove unused variable
Signed-off-by: Anil Veerabhadrappa <anil.veerabhadrappa@broadcom.com>
Change-Id: I671afd0d18a74df46acb3c617f5a380c66efef29
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10675
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2022-06-30 13:32:29 -04:00
Jacek Kalwas
9d20f8b0b5 nvmf: allow to override aer limit by specific transport
it is possible that some specific transport doesn't support
NVMF_MAX_ASYNC_EVENTS (although it is a recommended value by spec)

with that change it is possible to reduce aerl on transport specific
layer so it can be advertised correctly during identify controller

Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: Ife6465b5324fb39f9b343c6f42b860e9dd1164b2
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10422
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2022-06-30 13:32:29 -04:00
Jacek Kalwas
763d058efd nvmf: remove accept poller from generic layer
Not every transport requires accept poller - transport specific
layer can have its own policy and way of handling new connection.
APIs to notify generic layer are already in place
 - spdk_nvmf_poll_group_add
 - spdk_nvmf_tgt_new_qpair

Having accept poller removed should simplify interrupt mode impl
in transport specific layer.

Fixes issue #1876

Change-Id: Ia6cac0c2da67a298e88956734c50fb6e6b7521f1
Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7268
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-06-30 13:32:29 -04:00
Rui Chang
2d53d32872 nvmf/vfio-user: Fix doorbell polling not working on some ARM platform
On aarch64 platforms, doorbells update from guest VM may not be seen
on SPDK target side. This is because there is memory type mismatch
situation here. That is on guest VM side, the doorbells are treated as
device memory while on SPDK target side, it is treated as normal
memory. And this situation cause problem on ARM platform.
Refer to "https://developer.arm.com/documentation/102376/0100/
Memory-aliasing-and-mismatched-memory-types". Only using spdk_mb()
cannot fix this. Use "dc civac" to invalidate cache may solve this.

Profiling data did not show big performance degradataion.

Signed-off-by: Rui Chang <rui.chang@arm.com>
Change-Id: I9a18718f8c4307b3007b18c32ab02e6796548958
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10222
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2022-06-30 13:32:29 -04:00
Jim Harris
01cdc4afa1 nvmf: don't always update discovery log when adding hosts
If a subsystem has no listeners, then there is no need
to update the discovery log when adding a host, or setting
a subsystem to allow all hosts.

This eliminates some unnecessary discovery log update
notifications, especially when setting 'allow any hosts'
on a subsystem immediately after it is created (and before
it has any listeners).

Update unit test to check the adding a host to a
subsystem without listeners does not rev the genctr.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I63dab5df564269e574bb925890088f52063aa378
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10546
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2022-06-30 13:32:29 -04:00
Jim Harris
42bb8ae842 nvmf: don't update discovery log on subsystem create/delete
The discovery log isn't updated when a subsystem is created
or deleted, it's only updated when a listener for a
subsystem is added or removed.

So remove the nvmf_update_discovery_log() in the subsystem
create and delete paths. They just generate extra AER
completions that potentially cause the host to do unneeded
work.

Note that if a subsystem is deleted with active listeners,
the subsystem delete path will remove each of the listeners
before deleting the subsystem itself.  So the discovery log
will still get updated when those listeners are removed.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Id01bbfa3b24d3e1279a614a2fd60be41387a03b1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10545
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2022-06-30 13:32:29 -04:00
Changpeng Liu
8955399aef nvme/compliance: add a case to test CREATE IO SQ with out of range CQID
Change-Id: Ie1c80d33d7fcc704321948a4b1f713f6256dc6e5
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10151
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2022-06-30 13:32:28 -04:00
Changpeng Liu
ac7b1d3760 nvmf/vfio-user: define SPDK_PCI_VID_NUTANIX macro and use it in vfio-user
Change-Id: I09820f484be3e962bcc4e80964d152e64957b331
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10550
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-06-30 13:32:28 -04:00
Changpeng Liu
beefbd3152 nvmf/ctrlr: let transport can set VID SSVID and IEEE values
The nvmf library will use INTEL VID/SSVID/IEEE values by default,
each transport can overwrite them if needed.

Change-Id: I9dad521c4d080b6f0cc1aaeb4b5d5f6863c6846d
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10095
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-06-30 13:32:28 -04:00
Evgeniy Kochetov
a9c616d4b9 nvmf/ctrlr_bdev: Set DNR bit in status for failed NVMe passthru
When NVMe passthru command (IO or admin) fails on submission (e.g. it
is not supported), set DNR bit in completion status field. There is no
sense in retrying the command in this case.

Signed-off-by: Evgeniy Kochetov <evgeniik@nvidia.com>
Change-Id: I55960c128bd9fc31f6defef0b9832259a71684b1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8578
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2022-06-30 13:32:28 -04:00
Evgeniy Kochetov
1ac23b0f0b nvmf/ctrlr_bdev: Fix status code for failed admin passthru command
If NVMe admin passthru command is not supported by underlying bdev,
set status code in NVMe completion to INVALID_OPCODE.

Signed-off-by: Evgeniy Kochetov <evgeniik@nvidia.com>
Change-Id: I29c4e1f8263b76b27c199cfd2d9b2474432ec70b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10517
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2022-06-30 13:32:28 -04:00
Josh Soref
950f234a1b spelling: lib
Part of #2256

* accessible
* activation
* additional
* allocate
* association
* attempt
* barrier
* broadcast
* buffer
* calculate
* cases
* channel
* children
* command
* completion
* connect
* copied
* currently
* descriptor
* destroy
* detachment
* doesn't
* enqueueing
* exceeds
* execution
* extended
* fallback
* finalize
* first
* handling
* hugepages
* ignored
* implementation
* in_capsule
* initialization
* initialized
* initializing
* initiator
* negotiated
* notification
* occurred
* original
* outstanding
* partially
* partition
* processing
* receive
* received
* receiving
* redirected
* regions
* request
* requested
* response
* retrieved
* running
* satisfied
* should
* snapshot
* status
* succeeds
* successfully
* supplied
* those
* transferred
* translate
* triggering
* unregister
* unsupported
* urlsafe
* virtqueue
* volumes
* workaround
* zeroed

Change-Id: I569218754bd9d332ba517d4a61ad23d29eedfd0c
Signed-off-by: Josh Soref <jsoref@gmail.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10405
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-06-30 13:32:28 -04:00
Jim Harris
6790eb0c17 nvmf: update discovery log when listener is removed
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iaef6e7b1c133a49390acd985a77b5f4349f78653
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10358
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2022-06-30 13:32:28 -04:00
Jacek Kalwas
16646306f4 nvmf: fix negative path for add transport to tgt
in case of failure groups shall be destroyed

Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: I8933f4128a7a3361bbb55d6a9c08a540521e5bda
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10435
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-06-30 13:32:28 -04:00
Changpeng Liu
3d4b5dd204 nvmf/vfio-user: only set S_IRUSR and S_IWUSR for mmapped BAR
Change-Id: Ic02089cd9ec49341b86d92377da734dc013d5cc7
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10335
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-06-30 13:32:28 -04:00
Changpeng Liu
dabe8419e8 nvmf/vfio-user: define NVME PCI configuration macro
Change-Id: I8a43abe597fe7139ec578322e565c5aa52c2f312
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10365
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-06-30 13:32:28 -04:00
Changpeng Liu
fba7cf6d66 nvmf/vfio-user: rename NVMF_VFIO_USER_DOORBELLS_OFFSET macro
Doorbell offset starts from 0x1000 is defined by the NVMe
specification, so rename it to remove `VFIO_USER` prefix.

Change-Id: Ie34b12b3d2618f9b0ad0cf7ccbb103ad2c900f47
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10364
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-06-30 13:32:28 -04:00
Changpeng Liu
e3eac2cb98 nvmf/vfio-user: use macro for BAR size
Calculate supported maximum number of queue pairs based on
BAR0 size, this value isn't allowed to change at runtime, also
define BAR4/5 based on number of MSIX vectors.

Since the maximum number of queues is a large value(512), so we
still define a default value when starting, users still can
overwrite this value with a number no greater than 512.

Change-Id: I1b4b6bdf2ff9d129c8bdd493ffdf0a51f8772d51
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10334
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-06-30 13:32:28 -04:00
Changpeng Liu
e172ba7b81 nvmf/vfio-user: use stack variable for sparse mmap
libvfio-user will save a copy inside the library.

Change-Id: If7bb052b03fb92e46abe50fa945b812d149ef01d
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10363
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-06-30 13:32:28 -04:00