This makes the function and file/line info actually useful (instead of
pointing to the helper function itself).
Change-Id: I22bac68827115880a49d456706a7eaecdc12e9b5
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Each transport should handle its own qpair cleanup internally.
Change-Id: I7dd737be820ea6bad686f4aad7d74044fad58a47
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Let the transport access the controller options during
ctrlr_construct().
Change-Id: I83590c111e75c843685dd9315f0f08416168356d
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
nvme_rdma_req_get() is an internal function, and its only caller already
checks for a valid rqpair, so the NULL check is unnecessary.
Also clean up the redundant STAILQ_EMPTY/STAILQ_FIRST logic and use
STAILQ_REMOVE_HEAD.
Change-Id: Ic3828e8b5e881879173cb59350e39c5fac90e6ef
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
nvme_rdma_pre_copy_mem() does not have any failure cases, so remove its
return value and remove the never-taken branch in its only caller,
nvme_rdma_qpair_submit_request().
Change-Id: I91011734ed0c20f8db691d62172fe1a3021dd3a1
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
nvme_rdma_req_put() is an internal nvme_rdma.c function, and all of the
callers already have the rqpair, so pass it directly. We also already
verify that all of the callers have a valid rqpair and req before
calling nvme_rdma_req_put(), so it doesn't need to check for NULL
pointers.
This also means that spdk_nvme_rdma_req doesn't need to hold a pointer
to its rqpair anymore.
Change-Id: I893a46a9074f0a843e379d10c123f9292eb3b1a4
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
The only place where outstanding_reqs was checked was in
nvme_rdma_req_put(), but the error case there could only happen if some
kind of internal programming error occurred (e.g. calling
nvme_rdma_req_put() on an invalid request).
Change-Id: I71e40ce562a8720dfaf70437ffd4c6493327c091
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
nvme_rdma_ibv_send_wr_init() was only called in one place, so just move
its contents into nvme_rdma_qpair_submit_request() since it allows
simplification of the code:
- req was always NULL, so remove the code that used req entirely.
- wr and sg_list are never NULL, so remove the checks for those.
Change-Id: I12a4f3502219d3681607686945e343f6808c0d2f
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
We currently don't handle discovery service referrals, so skip those, as
well as any other unknown subsystem type.
Change-Id: I64f889e9272fb57b5cf9bb5467b3abca3955baf5
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
QEMU's virtual NVMe controller device does not support the AER Set
Feature, so ignore its failure and continue.
Change-Id: I8b5c217a3112edabb6f76ec3e5f4ef774981a1d7
Signed-off-by: Cunyin Chang <cunyin.chang@intel.com>
Catch SIGBUS and handle it by remapping new memory into the
location where the BAR previously was.
Change-Id: Ie8d00a60a0bbe7f7ec57a5c39c0a63c5d9443206
Signed-off-by: Cunyin Chang <cunyin.chang@intel.com>
These functions will attach or detach from a PCI device. Attaching
typically means mapping the BAR.
Change-Id: Iaaf59010b8a0366d32ec80bb90c1c277ada7cfe7
Signed-off-by: Cunyin Chang <cunyin.chang@intel.com>
This patch make sure the connection in normal state before any further
operation on this connection.
Change-Id: I776740b5b33b1de6707990c09d9131c385adf556
Signed-off-by: Cunyin Chang <cunyin.chang@intel.com>
spdk_nvme_probe frees ctrlr when nvme_ctrlr_process_init is failed. But
ctrlr has already been freed while calling nvme_ctrlr_destruct. So
spdk_nvme_probe doen't need to free ctrlr.
The generic NVMe library controller initialization process already
handles enabling the controller; the RDMA transport should not need to
set EN itself.
For now, the discovery controller is cheating and not using the normal
initialization process, so move the EN = 1 hack to the discovery
controller bringup until it is overhauled to use the full
nvme_ctrlr_process_init() path.
The previous code where CC.EN was set to 1 before going through the
controller init process would cause an EN = 1 to EN = 0 transition,
which triggers a controller level reset.
This change stops us from causing a reset during the controller
startup sequence, which is defined by the NVMe over Fabrics spec as
terminating the host/controller association (breaking the connection).
Our NVMe over Fabrics target does not yet implement this correctly, but
we should still do the right thing in preparation for a full reset
implementation.
This patch also reverts the NVMe over Fabrics target reset
handling hack that was added as part of the NVMe over Fabrics host
commit to its previous state of just printing an error message.
Change-Id: I0aedd73dfd2dd1168e7b13b79575cc387737d4f0
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Most of the NOTICE level messages should have been TRACE.
Change-Id: Icbc4d398ab2580cf3a2349be11441b7a09603020
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Verify that qpair is not NULL before doing pointer math on it.
The NULL check after calling nvme_rdma_qpair(qpair) would not
trigger if qpair was NULL.
Fixes a crash if the Connect command failed, causing
nvme_rdma_ctrlr_create_qpair() to return NULL.
Change-Id: I158a5b1752892a7d5a72a9ac20c0c5b2cd781a81
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
This allows us to print better error messages when connecting to a
subsystem that exists but does not allow a specific host.
Additionally, we can now return the correct error code for a host that
is not allowed.
Change-Id: I16cd4ac2745cf50bb54601b464b0d23954f86fda
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Official installs of DPDK place headers in a 'dpdk'
subdirectory under include, so detect that.
Change-Id: If64421c84c91cae31688994484c22fce398dc622
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
The status.done flag polled by nvme_ctrlr_set_keep_alive_timeout()
was never initialized.
Change-Id: I323fae5f4ce12209a9699965ce07894bc3c6205a
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
The code in virtual.c and direct.c was identical - move it to session.c
to share it.
Change-Id: Ic6e4e9238e8ffacb212e76293c440109aa839f8c
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Move the current Virtual mode implementation to session.c and use it for
Direct as well.
Change-Id: I3f0ac93b4247b93d158b0dcb77e257b4b91be129
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Store the host identifier from the Connect command and report it via Get
Features.
Change-Id: I79bc27e05c5944549e7986aadb919c19748e7474
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Also return Invalid Field rather than Invalid Opcode to be more
accurate. The spec doesn't seem to define any more specific error code
for this case.
Change-Id: I992c6cca3020ff80b8495c71170222bc75316800
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
None of the log pages are actually implemented yet, but at the very
least, we don't want to leak random bits of uninitialized data.
Change-Id: Ic889260eb18d49122f2f250b645bdc5be3561dc5
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Use the NVMe over Fabrics spec definitions for TRTYPE rather than the
internal library transport type.
Change-Id: Idead559a8f8d95274fc580d10e82033822e6eda8
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
These need to be available for the lifetime of the probe_info structure,
so they can't be pointing at e.g. temporary buffers on the stack.
Change-Id: I5aaa898acf9314aab51600dd756f966965d37fd0
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
-Wformat-nonliteral needs to be disabled since clang triggers it on the
call to vsnprintf() now that it is nested two calls deep.
Change-Id: I228b9d099cfc2b65181941cbb4798b7f8eae3baa
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
This is a counterpart to spdk_strcpy_pad() which determines the length
of a string in a fixed-size buffer that may be right-padded with a
specific character.
Change-Id: I2dab8d218ee9d55f7c264daa3956c2752d9fc7f7
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
It always points to the same internal RDMA request complete function, so
just call that function directly.
Change-Id: Ic1fb6236bf43eaad62413df77d43be9ab855e5c7
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
We can't transfer more than the bounce buffer in a single command, so
report that rather than some bogus value.
Change-Id: I39b147916dcc2ee478470917298763a239a6a35a
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Record the user-provided asynchronous event configuration set via Set
Features, and return it in Get Features.
This value is not actually used, since AER is not implemented yet in the
virtual controller model, but it at least implements the mandatory
Set/Get Features.
This allows the hack in the NVMe host code that ignored the Set Features
failure to be reverted.
Change-Id: I2ac639eb8b069ef8e87230a21fa77225f32aedde
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Fill in the cached copy of CAP in the generic NVMe controller to match
the PCIe transport.
This is not really early enough, since CAP is used during the reset
process to determine the reset timeout, but that will have to be fixed
separately by rearranging some of the transport callbacks.
Change-Id: Ia8e20dbb8f21c2871afb9e00db56d0730e597331
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Make sure the entire NQN field is zero-padded, rather than using
strlen() on the input.
Change-Id: Icee68bd033feed057813beeb30cec102ed90840e
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
This fixes a compiler warning about unhandled enum cases in a switch.
Change-Id: Icecb56b47a05c13f390f03b877f8eae243b481a6
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
- add SPDK_NVME_OPC_KEEP_ALIVE to admin_opcode
- add SPDK_NVME_SC_INVALID_SGL_OFFSET, SPDK_NVME_SC_INVALID_SGL_OFFSET,
SPDK_NVME_SC_HOSTID_INCONSISTENT_FORMAT, SPDK_NVME_SC_KEEP_ALIVE_EXPIRED
and SPDK_NVME_SC_KEEP_ALIVE_INVALID to generic_status
Make it easier to use SPDK libraries by putting them all in a single
directory that can be added with -L rather than scattered around the
source tree.
Change-Id: I5c0f5dd6e7058b5f92fa9bc41548190ffc064761
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Track the maximum copy task size as modules are registered rather than
recalculating it every time spdk_copy_task_size() is called.
Change-Id: I141aca61e7075402dac41915080d1b43faee32ce
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Make the public API clearer - if the user wants to allocate a
spdk_copy_task directly, they need to allocate spdk_copy_task_size()
bytes.
Also change the return type to size_t for consistency.
Change-Id: I0f3757056757c510421d680c5b4532edd9bc2561
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
The SPDK_TRACELOG macro depends on a CONFIG setting (DEBUG), so it
should not be part of the public API.
Create a new include/spdk_internal directory for headers that should
only be used within SPDK, not exported for public use.
Change-Id: I39b90ce57da3270e735ba32210c4b3a3468c460b
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Remove usage of the conf structs so they can be moved out of the public
API header.
Change-Id: I1c7375ec7708b323f50af09aeb7b2b2c9c770df4
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Considering the process can be terminated in the cases like ctrl+c,
kill command or memory fault, the ref is tracked in the per process
structure spdk_nvme_controller_process and whenever there is other
process attaches or detaches the controller, a scan will be issued
to cleanup those unexpectedly exited processes.
Change-Id: Ib4f974f567a865748d42da4ead49edd383dfc752
Signed-off-by: GangCao <gang.cao@intel.com>
These APIs can be used to register/unregister regions
of pinned, huge page memory that are separate from
huge page memory allocated by the default DPDK
allocations. These APIs will be used by an upcoming
SPDK vhost-scsi target to enable SPDK to target
NVMe DMA operations directly to VM memory that has
been allocated by QEMU using pinned huge pages.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I649a4adeeb758b29bd29cd42c8872eed3d5d6ce9
Now that the env PCI framework already requires enumerating devices
based on an enum of specific device types, it is not useful to query the
class code of a PCI device handle.
It is currently unused and does not work in its current form on FreeBSD
(it reads a file from /sys). This lets us drop a big chunk of file
reading and parsing code.
Change-Id: I1d720398416ba3d6f91e077b807ec11a6de562cf
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
The details of the structure were removed earlier, but
now remove all references even to a pointer to the
structure. The user can refer to transports by their
string name.
Change-Id: I273356f46329ea5372dcd951eda6f14767477d69
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
This is a step toward abstracting away the definition
of the subsystem.
Change-Id: I88b2aa107b27152620f51a1ca2a153792b4c85e9
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Remove 4k allocation size in spdk_scsi_task_alloc_data(). From now on
all commands must obay allocation length.
Change-Id: Ica9384c62d431483ae1d0bd2e6fdee18b570861f
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
This refactor MODE SENSE 6 and 10 related functions to respect buffer
size parameter.
Change-Id: I03bad456bac0554a8bf7b56f69d1f9cf5b1991f6
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
This patch is preparation for fixing alloc_len overrun in SENSE 6/10 and
READCAP 6/10. To simplify code forbid usage of iov outside of
scsi/task.c.
This also drop SPDK_SCSI_TASK_ALLOC_BUFFER flag that obfuscate code. As
a replacement assume that if field alloc_len is non zero it mean that
iov.buffer is internally allocated. Functions
spdk_scsi_task_free_data(), spdk_scsi_task_set_data() and
spdk_scsi_task_alloc_data() manage this field.
Change-Id: Ife357a5bc36121f93a4c5d259b9a5a01559e7708
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Use the len field from the generic spdk_bdev_io instead of duplicating
it in blockdev_rbd_io.
Change-Id: I3ebfab8dd1303add83bc2206fc87319ba7d605b3
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
This function needs to check for SGEs that straddle a
2MB page boundary, and ensure it does not return
a length that will cross that boundary.
This cannot happen in practice currently with SPDK
since all buffers are allocated using rte_malloc(),
but an upcoming vhost-scsi target may produce
SGEs from a guest VM's physical memory that span
a 2MB boundary.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I8b83c7c39c4cf33815abb22ff2ebc90941b21e28
