In the following patches, nvme_rdma_poll_group_set_cq() will
touch not only CQ but also SRQ and receive WR objects.
All these resources are of a poller.
Hence for clarification, rename nvme_rdma_poll_group_set_cq()
by nvme_rdma_qpair_set_poller().
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: Ic59ba5a45833e39b1b2647c000c8b953f1031d6b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14910
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
In the following patches, poll group will have rsps objects and to share
the code between poll group and qpair, option for creation will be used.
As a preparation, merge nvme_rdma_alloc_rsps() and
nvme_rdma_register_rsps() into nvme_rdma_create_rsps(). For consistency,
merge nvme_rdma_alloc_reqs() and nvme_rdma_register_reqs() into
nvme_rdma_create_reqs().
Update unit tests accordingly.
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Signed-off-by: Denis Nagorny <denisn@nvidia.com>
Signed-off-by: Evgeniy Kochetov <evgeniik@nvidia.com>
Change-Id: I92ec9e642043da601b38b890089eaa96c3ad870a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14170
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
When SRQ is supported, recv objects will be allocated by poll group
and qpair will associated and use them. In this case, we do not want
qpair to allocate and free recv objects. When connection is established,
it will be decided if SRQ is used or not. Hence, defer recv objects
allocation until connection is established.
Send objects are not affected directly by SRQ, but
nvme_rdma_register_reqs() no longer does any registration and deferring
send objects allocation makes the code more consistent. Hence, defer
send objects allocation until connection is established too.
Even after this patch, we rely on nvme_rdma_ctrlr_delete_io_qpair()
to free resources completely.
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Signed-off-by: Denis Nagorny <denisn@nvidia.com>
Signed-off-by: Evgeniy Kochetov <evgeniik@nvidia.com>
Change-Id: Ic151fad01009d92a7fc809a730e6e9dff1a365f3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14169
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
nvme_tcp_read_pdu in a loop
nvme_tcp_read_pdu itself has a loop in it that runs until no more data
is available, so the extra loop does nothing.
Change-Id: I1471018e396c43187d1f06bd18ce8a6846a71c94
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15139
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Also add unit tests that explicitly test this
condition. They fail without the nvme driver changes
in this patch.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iaa369be341eb4eba394f248990e56dce001d3940
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15579
Reviewed-by: Mariusz Barczak <mariusz.barczak@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
per Intel policy to include file commit date using git cmd
below. The policy does not apply to non-Intel (C) notices.
git log --follow -C90% --format=%ad --date default <file> | tail -1
and then pull just the 4 digit year from the result.
Intel copyrights were not added to files where Intel either had
no contribution ot the contribution lacked substance (ie license
header updates, formatting changes, etc). Contribution date used
"--follow -C95%" to get the most accurate date.
Note that several files in this patch didn't end the license/(c)
block with a blank comment line so these were added as the vast
majority of files do have this last blank line. Simply there for
consistency.
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Id5b7ce4f658fe87132f14139ead58d6e285c04d4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15192
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
NVMe-RDMA target has a helper function get_rdma_qpair_from_wc() and
uses it to identify a qpair from a WC.
NVMe-RDMA initiator has a similar function
nvme_rdma_poll_group_get_qpair_by_id().
NVMe-RDMA initiator will support SRQ in the following patches, and
it will want to identify a qpair from a WC.
get_rdma_qpair_from_wc() of NVMe-RDMA target uses wc->qp_num internally
anyway.
However, the upcoming custom transport for RDMA will have to use other
variables of WC.
Hence, it will be convenient to pass WC instead of qp_num if we consider
future enhancements.
Based on these thoughts, for NVMe-RDMA initiator rename
nvme_rdma_poll_group_get_qpair_by_id() by get_rdma_qpair_from_wc().
remove unnecessary declaration, and pass WC instead of qp_num.
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Signed-off-by: Denis Nagorny <denisn@nvidia.com>
Signed-off-by: Evgeniy Kochetov <evgeniik@nvidia.com>
Change-Id: I01ead4730207e2c6ac53b83f151bd5f977a11465
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14279
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
With RDMA, the admin poller can experience a remote disconnect when
processing completions. The admin qpair will be disconnected to handle
this. The disconnect code path will manually complete queued aborts.
However, the completion callback for the abort will attempt to resubmit
other queued aborts from the queue, which will result in a very large
stack and can eventually cause a segfault.
The fix is to not resubmit queued aborts if the admin qpair is in any
kind of failed state.
Change-Id: I4a6f959232c8a1bd30c87ca50459014e556cbaa0
Signed-off-by: Vasuki Manikarnike <vasuki.manikarnike@hpe.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15114
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Michael Haeuptle <michaelhaeuptle@gmail.com>
psh len is not the same with header len.
Add an assert in nvme_tcp.c to prevent this happen again.
Signed-off-by: MengjinWu <mengjin.wu@intel.com>
Change-Id: Ibc250752bedf3da8994f79c51fb01577a222d364
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14521
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This "if" is of no use here.
The state machine has the "NVME_TCP_PDU_RECV_STATE_AWAIT_PDU_CH"
state means the pdu does not receive enough length of header.
Signed-off-by: MengjinWu <mengjin.wu@intel.com>
Change-Id: Id50943f77b570fd337e2bb4e3b45281018d159e4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14504
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
These functions used to allocate resources
using calloc/spdk_zmalloc depending on the
g_nvme_hooks pointer. Later these functions
were refactored to always use spdk_zmalloc,
so they became simple wrappers of spdk_zmalloc
and spdk_free. There is no sense to use them,
call spdk memory API directly.
Signed-off-by: Aleksey Marchuk <alexeymar@nvidia.com>
Change-Id: I3b514b20e2128beb5d2397881d3de00111a8a3bc
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14429
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Since now cmds and rsps buffers are allocated
from huge pages, there are already registered
MR for this memory. In that way we can avoid
registering 2 additional MRs per qpair, just
perform memory translation to get lkey.
Signed-off-by: Aleksey Marchuk <alexeymar@nvidia.com>
Change-Id: I2cb39a15e5d224698c293ac18af00a909840eaa8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14428
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
state change function do not need to use swtich to do some work.
Do memset in state machine.
Signed-off-by: MengjinWu <mengjin.wu@intel.com>
Change-Id: Ie66454d8f31860f403171f20858a6b4a24e3c76f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14502
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
For non-fabric controllers, the corresponding I/O qpairs are simply
re-enabled at controller reset.
This had a issue when I/O qpairs span multiple threads and poll group
is used.
spdk_nvme_ctrlr_reconnect_poll_async() calls
nvme_transport_ctrlr_connect_qpair() with qpair->async being false.
Then nvme_transport_ctrlr_connect_qpair() calls
spdk_nvme_poll_group_process_completions() until the qpair is connected.
spdk_nvme_poll_group_process_completions() may poll other qpairs.
This may cause I/O to complete on a wrong thread.
For PCIe controller, spdk_nvme_poll_group_process_completions() calls
spdk_nvme_qpair_process_completions() simply for each qpair.
Hence change nvme_transport_ctrlr_connect_qpair() to call
spdk_nvme_qpair_process_completions() if the controller is non-fabrics.
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: Ieb270c2fb154124021ef6d25577b817d05e5ca9e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14295
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Michael Haeuptle <michaelhaeuptle@gmail.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
There's no reason for this parameter to be non-const and it makes this
functions pain to use when you want to hardcode a specific sock
implementation.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ifed4426a02ab54cbd51c8a2051b1eac010f86db9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14303
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
If the being disconnected qpair is the last of a poller of a poll group,
CQ is destroyed and the poller is released before the qpair is actually
disconnected.
This patch destroy CQ and release the poller after the qpair is actually
disconnected.
One exception is when spdk_nvme_ctrlr_free_io_qpair() is called to a
connected qpair. In this case, the qpair is removed from a poll group
before the qpair is actually disconnected. In this case, destroy CQ and
release the poller when the qpair is removed from the poll group.
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: Idf266bbb6dbb40f04ae6313db724fabf80865763
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14253
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Avoid putting a new req on the outstanding_reqs
TAILQ until we know it can be initialized
successfully. This avoids adding to the TAILQ
only to remove it just after.
This allow simplifies the outstanding_reqs TAILQ
handling, since reqs are now only inserted and
removed in one place each.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I5ccc41c14abd541ffcf2a602246e0671386840c7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13991
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
ioccsz is specific for fabrics. spdk_nvme_ctrlr_is_fabrics() returns
true for custom fabrics transport. Hence we can use
spdk_nvme_ctrlr_is_fabrics() safely in nvme_ctrlr_update_nvmf_ioccsz().
Before this change, in the unit tests, ctrlr->trid.trtype was set to
zero at initialization. After this change, for most cases,
spdk_nvme_ctrlr_is_fabrics() should return false for most cases.
SPDK_NVME_TRANSPORT_PCIE did not work. Hence, initialize
ctrlr->trid.trtype by SPDK_NVME_TRANSPORT_CUSTOM_FABRICS instead.
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I4bedcab4a9f2876c1c9463ff10ad0966754f1713
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13948
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Get a PD for the device from the PD pool managed by the RDMA provider
when creating a QP, and put the PD when destroying the PD.
By this change, PD is managed completely by the RDMA provider or the hooks.
nvme_rdma_ctrlr::pd was added long time ago but is not referenced
anywhere. Remove nvme_rdma_ctrlr::pd for cleanup and clarification.
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: If8dc8ad011eed70149012128bd1b33f1a8b7b90b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13770
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
This is another preparation to create and use ibv_context and pd.
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Signed-off-by: Denis Nagorny <denisn@nvidia.com>
Signed-off-by: Evgeniy Kochetov <evgeniik@nvidia.com>
Change-Id: Id594fa1ccb2daf535b1aaaef0a397bda2ec98578
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13710
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
The function `nvme_ctrlr_init_ana_log_page` is exactly
same with `nvme_ctrlr_update_ana_log_page`, so remove it.
Change-Id: I1ad51635f47cf95cfa6de217e3b9144885c3b74e
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13652
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Original implementation creates pollers and CQs for all discovered
devices at poll group creation. Device (ibv_context) that has no
references, i.e. has no QPs, may be removed from the system and
ibv_context may be closed by rdma_cm. In this case we will have a CQ
that refers to closed ibv_context and it may crash in ibv_poll_cq.
With this patch pollers are created on demand when we create the first
QP for a device. When there are no more QPs on the poller, we destroy
the poller. This also helps to avoid polling CQs that don't have any
QPs attached.
Signed-off-by: Evgeniy Kochetov <evgeniik@nvidia.com>
Change-Id: I46dd2c8b9b2902168dba24e139c904f51bd1b101
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13692
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
That is possible to get/set registers from any thread,
during regs processing we are polling admin qpair to
get a completion. At the same time, another thread
can also poll admin qpair and that can lead to
undefined behavior.
This patch fixes an issue when bdev_nvme is configured
with io_timeout. If remote target becomes unresponsive
(e.g. due to link down), IO timeout occurs and bdev_nvme
tries to get csts registers in timeout_cb. At the same
time another thread can process adminq, so we may have
2 simultaneous adminq polls. If admin qpair is disconnecting
at that time (RDMA transport) we may destroy resources
twice from different threads.
We don't see a problem with set_regs function but it
won't be redundant to lock mutex in set_regs as well.
Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: I7ec3984d25d0249061005533d13b22315b44ddf2
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13687
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
In the multi-process case, a process may call `spdk_nvme_ctrlr_free_io_qpair` on
a foreign I/O qpair (i.e. one that this process did not create) when that qpairs
process exits unexpectedly.
The variable `qpair->poll_group` isn't multi-process safe, we can't use it
in `spdk_nvme_ctrlr_free_io_qpair` and related transport poll group APIs.
Change-Id: Ic13a6a2c7d760477be5be5a56a45caa2b5518717
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13573
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
This follows similar logic in the pcie and tcp
completion paths, including omitting error
messages when aborting aers by adding a print_on_error
parameter to the completion function.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Id558d0af2cdd705dfb60abb842bd567a0949ccce
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13525
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
By default, the SPDK nvmf target reports vid==INTEL,
which results in the SPDK nvme driver trying to enable
Intel vendor-specific log page. Fix this by trying to
enable those log pages only for PCIE transport
controllers.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I78ebf365d4fa6295d1f610697266c3ead765988d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13524
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
We can only get to this code path if the controller
has vid==INTEL, so make that more clear by changing
the check to an assert.
Remove unit test that calls
nvme_ctrlr_construct_intel_support_log_page_list()
for a controller that is not VID==INTEL - this is
no longer valid.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I3b58451bc95992bf641e7452f0ac4c2bac9fe31c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13523
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
This follows similar logic in the pcie completion
path, including omitting error messages when aborting
aers by adding a print_on_error parameter to the
completion function.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I96df72280bb8fcbee3847fdc27f38e14a1bf3251
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13522
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
nvme_qpair_abort_all_queued_reqs() aborts error injections, queued
requests, aborting queued requests, and outstanding requests. (Aborting
outstanding requests depends on transports.) However, it did not abort
queued aborts.
Include nvme_ctrlr_abort_queued_aborts() into
nvme_qpair_abort_all_queued_reqs() to do really the name of the
function indicates.
nvme_ctrlr_abort_queued_aborts() has been called in a few cases, but
we do not care duplication.
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I19102cc6603a72ce5c398a7947cb4d606b692991
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12849
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Vasuki Manikarnike <vasuki.manikarnike@hpe.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
In SPDK, declarations have the return type on the same line. Definitions
have the return type on a separate line. Astyle has an option for
enforcing this. Unfortunately, it seems to have two bugs:
1) It doesn't work correctly at all on C++ files.
2) It often fails on functions that return enums, or long type names
Deal with 1) by adjusting the check_format.sh script to only tell astyle
to fix return type line breaks for C files and not C++. Deal with 2) by
adding a few typedefs to work around the problem.
Change-Id: Idf28281466cab8411ce252d5f02ab384166790c6
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13437
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Many open source projects have moved to using SPDX identifiers
to specify license information, reducing the amount of
boilerplate code in every source file. This patch replaces
the bulk of SPDK .c, .cpp and Makefiles with the BSD-3-Clause
identifier.
Almost all of these files share the exact same license text,
and this patch only modifies the files that contain the
most common license text. There can be slight variations
because the third clause contains company names - most say
"Intel Corporation", but there are instances for Nvidia,
Samsung, Eideticom and even "the copyright holder".
Used a bash script to automate replacement of the license text
with SPDX identifier which is checked into scripts/spdx.sh.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iaa88ab5e92ea471691dc298cfe41ebfb5d169780
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12904
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: <qun.wan@intel.com>
This SGL type was missed in the original commit
that added the pretty printing.
Fixes: 4d9ab1e9a1 ("nvme: pretty print dptr")
Reported-by: Ramanjaneya Burugula <burugula@gmail.com>
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ibc655db4e65009071f39f55f691c94a094cea0bc
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12705
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
As described in the previous patches, we need to delete all I/O
SQ/CQs before aborting trackers when disconnecting a controller.
The following patches reorder the operations. This patch changes
adminq disconnection to initiate a Controller Level Reset and
adminq completion processes it if ctrlr->is_disconnecting is true.
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I64f06bae2ce8a9127124029fd042db0028198e3c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12560
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Michael Haeuptle <michaelhaeuptle@gmail.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
In the following patches, nvme_ctrlr_process_init() will be used to
disable the controller when disconnecting the admin qpair for PCIe
transport. In this case, we will have to exit nvme_ctrlr_process_init()
after CSTS.RDY is 0. However, spdk_nvme_ctrlr_reset() and
spdk_nvme_ctrlr_reconnect_poll_async() have to continue
nvme_ctrlr_process_init() until the controller becomes ready.
To differentiate stop and continue clearly, add a new state
NVME_CTRLR_STATE_DISABLED to enum nvme_ctrlr_state.
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: Ic0a5fb7114d4eeb1cefec28bc404184768fb0a96
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12613
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michael Haeuptle <michaelhaeuptle@gmail.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
The code to handle the lingering qpair when deleting it was really
complicated.
The RDMA transport can connect or disconnect qpair asynchronously.
Then we can include the code to handle the lingering qpair into the
code to disconnect qpair now.
If the disconnected qpair is still busy, defer completion of the
disconnection until qpair becomes idle.
If poll group is not used, we can complete disconnection immediately
because cq is already destroyed.
The related data and unit test cases are not necessary anymore.
So delete them in this patch.
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: Ic8f81143fcad0714ac9b7db862313aa8094eeefb
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11778
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Add three states, INITIALIZING, EXITING, and EXITED to the rqpair
state.
Add async parameter to nvme_rdma_ctrlr_create_qpair() and set it
to opts->async_mode for I/O qpair and true for admin qpair.
Replace all nvme_rdma_process_event() calls by
nvme_rdma_process_event_start() calls.
nvme_rdma_ctrlr_connect_qpair() sets rqpair->state to INITIALIZING
when starting to process CM events.
nvme_rdma_ctrlr_connect_qpair_poll() calls
nvme_rdma_process_event_poll() with ctrlr->ctrlr_lock if qpair is
not admin qpair.
nvme_rdma_ctrlr_disconnect_qpair() returns if qpair->async is true
or qpair->poll_group is not NULL before polling CM events, or polls
CM events until completion otherwise. Add comments to clarify why
we do like this.
nvme_rdma_poll_group_process_completions() does not process submission
for any qpair which is still connecting.
Change-Id: Ie04c3408785124f2919eaaba7b2bd68f8da452c9
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11442
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
The process of matching qpair to poll group is split into
two distinct parts that occur on different threads.
See spdk_nvmf_tgt_new_qpair().
This results in a race condition for TCP between spdk_sock_map_lookup()
and spdk_sock_map_insert(), which are called in spdk_nvmf_get_optimal_poll_group()
and spdk_nvmf_poll_group_add() respectively.
Fixes#2113
This patch picks a hint from nvmf_tcp for next poll group,
which is then passed down to spdk_sock_map_lookup().
When matching placement_id exists, but does not have
a poll group assigned - the hint will be used.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I4abde2bc9c39225c9f5dd7c3654fa2639bb0a27f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10271
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
