Instead of searching the global list of connections to find a matching
cm_id, we can just store the pointer back to the spdk_nvmf_conn in the
rdma_cm_id context field.
Change-Id: I39ea16be6a633a1136d65743747b63b600f20e63
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
All of the variables are private to conn.c, so they don't need global
visibility.
Change-Id: I7c24cfc6249a9f8164b162b4b8de0e24c452e0df
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
It is always set to nvmf_process_async_completion and is only used
within the library.
Also rename nvmf_process_async_completion to spdk_nvmf_request_complete
to clarify its purpose.
Change-Id: Ie737fb60688329bfe329a8553c4a40ff2e5f8f1d
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Create spdk_nvmf_request_prep_data(), which handles SGL processing and
data transfer for all command types, and spdk_nvmf_request_exec(), which
executes a command after data transfer has completed.
Change-Id: I51c2196260dd0686db8acca4d8f7c93e17618c2f
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
The pending type can be determined based on the command opcode.
This also moves the "issue pending RDMA reads" case out of the I/O queue
handling into the generic continuation code; this should not make any
difference for the current case, since the Fabrics Connect command is
the only other continuation case currently, and there cannot be any
pending RDMA reads in that case.
Change-Id: Idddfa496b6e5b7e6da772aa3ab1b9d1a5344771f
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
nvmf_connect_continue() no longer needs the RDMA-specific tx_desc.
Change-Id: I95f6938063e9853aa7dcd419f488b91422ff9b60
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Let the calling function handle the tx_desc if nvmf_connect_continue()
fails.
Change-Id: I25a8cbc4c3be0608bcec8db2fb8c50e55fbe3e8c
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Everything necessary for processing an admin command is now stored in
nvmf_request.
Change-Id: I74e75a5b7bb3b406ad167c2b31cab1af7a1f270a
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Everything necessary for processing an I/O is now stored in
nvmf_request.
Change-Id: I3f390707ebe83ea66a116dcfda4d0388a6823629
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Keep a pointer to the local bounce buffer in the transport-agnostic
struct nvmf_request rather than groveling in tx_desc/rx_desc to get it.
Change-Id: Ic328d8e2b3a15759ccb149a89fb3562e928ca500
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Now that we have xfer to track data direction, the length field can be
populated correctly for all transfers (including in-capsule data).
Change-Id: I7b2228f3fac80aab983a4103ba095c7bc38e0b21
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
This field is used to decide whether data needs to be transferred back
to the host after a command is completed.
Previously, this was determined using the length field, and length was
cleared to 0 after a transfer was completed. However, length will be
used in future patches after a host to controller transfer completes, so
we need some other way to tell what kind of data transfer is required.
Change-Id: I6b27cf7816908394735fc95c15bd5eb40a7c0157
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
vtophys() was checking for out-of-range addresses incorrectly: vfn_2mb
is already shifted to account for 2 MB hugepages, but it was being
compared with a mask that did not account for the shift. This would
allow out-of-bounds access to the 128tb map array for certain invalid
addresses (it had no effect on addresses within the valid userspace
range).
Change-Id: Ida7455595e586494c9025f9ba65d050abb16b1b9
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Other subsystems can depend on this one and then use
SPDK_RPC_REGISTER to register RPC functions.
Change-Id: I557f774331ce7146d299d06b3f81426e2103a11f
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Since we used u64 to mask CPU cores, the available number of CPU
is 64, for default RTE_MAX_LCORE in DPDK, the value is 128, in some
cases(e.g.: when nr_io_queues > 4) we can get the wrong lcore ID.
Change-Id: Icc334b1bf5b068a310839118be341e61071cff65
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
A transfer using less than the total bounce buffer size is a normal
occurrence and not worthy of a tracelog. Also drop the pointless
conditional.
Change-Id: Ibcdcf693fea439d5034fa51b08b3fbd8fd7df8f2
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Avoid using the RDMA-specific tx_desc when the transport-agnostic
nvmf_request will suffice.
Change-Id: Id35bbdfb353cb72e0feb4f5af19e5bd5c86d3ff4
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Should set tx_desc=NULL here. If not,When
nvmf_post_rdma_recv(conn, rx_desc) fails,
we would make tx_desc deactive again, and
this is wrong.
Change-Id: Ieabc7e3864b7f124b003d052f66ab8799a1d632e
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
The variable cur could be null, so if we use do while
will cause a segment fault
Change-Id: I19ec26e88948a0c3fd957e03e717b68750f40c62
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
The conventional rule for returning errno is negative, hence there is no
need to modify caller's code to adjust this NVMe library.
Signed-off-by: Minfei Huang <mnghuan@gmail.com>
Signed-off-by: Minfei Huang <minfei.hmf@alibaba-inc.com>
If we just pass NULL to rdma_create_qp, it will do
the right thing.
Change-Id: I9621a5110ace6237a1e47c6e5defb4cac3afc4ae
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
The wrappers are much simpler to use than the low
level ib verbs calls.
Change-Id: I4b09a96a60020bc27df9396d40d955733f618837
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
They are only ever called in sequence and do related
operations.
Change-Id: I825abe08deba1dafb405757bb4f2d52062a801ca
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
This enables SPDK_NVMF_BUILD_ETC to be moved out of the library as well,
since only authfile was using it before
Change-Id: I10d1145881f9a0358d7effe2d2d9851899413e1b
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
SPDK_NVMF_BUILD_ETC will be cleaned up in another commit; it is
currently used both in the lib and in nvmf_tgt.
Change-Id: Ibc5f15cc4341f9d52b29c84defcd332bec4a4d09
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Most of the #include statements in nvmf.h aren't part of the public API.
Change-Id: I0d43dd542a28744a91a4fd0c4c806a991d1e194e
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
It is not part of the NVMf library's public API.
Change-Id: I665d5713343c9185cbdadaef4fedfdc83b8232d6
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
There is only a single global g_nvmf_tgt that can be passed to this
function, so remove the parameter and use the global directly.
Change-Id: Ia1a2a1e6cd3801101ddeb4de5526dd115fa7ef8f
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
The section is really defining a subsystem as defined
by the NVMf specification. There does not appear to be
any need for a group of subsystems.
This change only updates the configuration file. It does
not remove all references to a subsystem group from
the code.
Change-Id: I38e62735a5ac924dcafacb3c9a332a103d751d4a
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
The specification refers to this concept as a Host,
so use that term. This only changes the configuration
file usage. Initiator groups are still referenced in
the code and will be removed later.
Change-Id: I897f4dbdfb65d94da1e5a77434fc07a2c18bcdc2
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Eliminate the misconception of reactor, it actually not a subsystem.
Change-Id: I63ea46f0dfa34661f16526a71c47e8fba9813474
Signed-off-by: Cunyin Chang <cunyin.chang@intel.com>
The index should be 0 for fabricintf.
Moreover, when there is no fabricintf found, error should
be returned
Change-Id: I3aa04566a5a318b8c921dd37c8573ed075254266
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
There is no logical split between nvmf.c and framework.c, so combine
them and drop nvmf.c.
Change-Id: I91230c01ed7f171bfed04456b0bfcf0e7ddbc263
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
The mutex is initialized, but otherwise is unused.
Change-Id: Ia68adbd430fad391cc465c07dd6e937e90dd2c5c
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
The code that actually removed items from the list was removed in
addition to the free() call, which caused a hang on shutdown.
Change-Id: If0e843d0d0ebfa28638b12104da880e70b3e548a
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Previously, there was no way to determine what namespace ID was assigned
when a namespace was created via the NVMe library interface.
Also drop the incorrect comment about calling
spdk_nvme_ctrlr_process_admin_completions(), since
spdk_nvme_ctrlr_create_ns() checks the admin queue internally.
Change-Id: If90a6e9fc773aefa220ebbf6effc2d033c9f20cc
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
This lets spdk_nvme_qpair fit in 128 bytes exactly.
Change-Id: I7c42582f22ece72a7f1d651468e63d4fe05babd6
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Avoid potential double free cases.
This fixes a clang warning during scan-build.
Change-Id: I487d6fcd485d1f8ebb96b6f8cb54511628461f39
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
All trackers in outstanding_tr should have a non-NULL request. Add an
assert to verify this.
Fixes a clang warning during scan-build.
Change-Id: I0ac4d2bad17449f684808cbb98777627d890b65b
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Document the requirement that qpair and cmd can't be NULL.
This placates clang, which previously generated a warning
during scan-build.
Change-Id: Ic2d5e808faee0028c890ce1312444fb3dc95f223
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
These allow linear searches of the configuration file
sections.
Change-Id: I8d8b9594bc8a974c16d999689a6195434c1efac8
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
The user can determine whether submission queues will be placed in the
controller memory buffer by checking the controller options use_cmb_sqs
flag in the attach callback.
Change-Id: I8a925ef99a48665a0e2ffaa90d9ff2b79b90b2fa
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Add a special "all" trace flag name to set or clear all registered trace
flags.
Change-Id: Ib579df7c41ce4aca72174e04734df20f2752035c
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
The D3700/D3600 series support Controller Memory Buffer(CMB) feature,
CMB is available for holding submission queues, for those controllers
which can support submission queues in CMB, user can set the option
whether to enable it or not.
Change-Id: I8b0dc9e28dd6f5bb01bee99a532087212c04e492
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
When recursively calling spdk_json_write_val() on another object or
array, the child call will handle printing out the whole
subobject/array, so the parent call should skip over all of its values.
Also fix the return value for the array/object case - if we get to the
end of the array or object, we should return 0 for success.
Change-Id: I1da80c88ab8759620114c1ab141baaaaf9f0023a
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Resolve relative paths before using them to clean up command lines.
This should also help shorten the overall command line length that gets
embedded in the binary and used when locating the executable from a
coredump.
Change-Id: Ibff9849ede198bb04313496c8b7131485ffaf14f
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
This patch add support for Intel specific log pages :
marketing description page.
Change-Id: I87bccb2af286279598c9dd3c870094b384a0d2f7
Signed-off-by: Cunyin Chang <cunyin.chang@intel.com>
For existing \file markers, move them to the top of the header and tweak
the wording for consistency.
Change-Id: Icce748effe4dbe97d79a8c87d31caf0ee5797058
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Function to trim leading and trailing whitespace from a string.
Originally based on code imported from istgt.
Change-Id: I87abe584130bdf4930098fadb8e57291f18eda7f
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Parsing function for delimited strings with embedded quotes.
Originally based on code imported from istgt.
Change-Id: I448feb53ea232048ed8c68738e12bc3660eb4235
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Add function to convert string to lowercase in place.
Originally based on code imported from istgt.
Change-Id: Ica9fe2208e6ee09b22c9a652a33c5affe5be23cc
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
For those controllers which can support end-to-end data protection
feature, add the support in the driver layer.
Change-Id: Ifac3dd89dec9860773c850416a6116113a6ce22a
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
When using DPDK PCI support via VFIO, the PCI device is reset
immediately before calling the PCI driver's init function. In some
cases, the device seems to not be ready to handle MMIO accesses right
away. Until the cause of this issue is fully understood, add a 500 ms
sleep as a workaround.
Change-Id: Ic893080a6f34d57eee80df3e6aa68c220c08df3e
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
All children must be removed from a parent request before the parent is
freed.
Change-Id: I073ff0e9c5bcdd6181d90b918bfe4cce054f6c0b
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Rename nvme_remove_child_request() to nvme_request_remove_child() and
move it next to nvme_request_add_child() to make the symmetry clear.
Change-Id: I78747c44ab3db1a656b33555a45f634dc5a55b31
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
The NVMe specification recommends destroying all I/O submission and
completion queues before setting CC.SHN.
Change-Id: Iad71dd3fe03d897858034f3ca6ee02e0c55cc2b0
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
The NVMe specification recommends that orderly shutdown should just
write CC.SHN while the controller is still enabled rather than writing
CC.EN = 0 first.
This also allows removal of the now-unused nvme_ctrlr_disable() and
nvme_ctrlr_wait_for_ready() functions.
Change-Id: I4702ffda153f218ebb8ed92f0e36144b7ceded93
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
This can happen if the controller is still resetting as the SPDK NVMe
driver takes control.
Change-Id: I263ae8f2e7b271e0448450557452a115c90c4fb6
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
This patch is used to handle the memory leak issue when
a parent nvme_request is free. In our current code,
we did not free the nvme_request allocated by the children
in the exceptional case.
Change-Id: Iabd1f1c3594af60c38e74e3d96c14f78d1aa1aed
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
This patch is used to add a nvme_request remove child
helpler function
Change-Id: I1e5bb228d53333ca3601f4ae30fcd801ea39e532
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Drop the "data_transfered" variable and just update length, since length
was not used otherwise after this point in the loop.
Change-Id: Icd2991e4e85de7e8c951ba14c441434e871ea4ef
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
If only one descriptor is needed, there is a special case in the spec for
SGL1 using the Data Block descriptor type. Add a comment to make it
clear what is going on.
Also tweak the SGL1 setup to copy from the first SGL descriptor list
element instead of relying on the last value from the loop above, since
that could be easily broken by accident.
Change-Id: I49ef97fe5bf18d2bf1d86b4310a7d3abdfd03e57
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
tr->u.sgl is already a struct spdk_nvme_sgl_descriptor pointer.
Change-Id: Ie2c8c052fc28e6369d1d095b8d566acae47975d1
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
_nvme_qpair_build_hw_sgl_request() will only be called for payload_size
!= 0, so every SGL will have at least one segment. Drop the 'else' that
was handling nseg == 0, and add an assert to document the payload_size
requirement.
Change-Id: I48e2a862a7657ba85605c0d35c0b65dfac072167
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
