Commit Graph

186 Commits

Author SHA1 Message Date
Allen Zhu
52a0feb404 bdev/nvme: handle namespace resize event
while the size of namespace is changed,
the resize event will be notified.

Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Signed-off-by: Evgeniy Kochetov <evgeniik@mellanox.com>
Signed-off-by: Allen Zhu <allenz@mellanox.com>

Change-Id: I5d85f17df898dc21c0ae1eb9f529dcb624a457ac
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/849
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-02-21 09:38:42 +00:00
Alexey Marchuk
c3ba9127d0 nvme: Store NVMEoF ioccsz and icdoff in ctrlr structure
This allows to avoid calculation of ioccsz bytes on each request
and removes access to "cold" ctrlr structures in data path.
Add UT to check validness of calculation

Change-Id: I55ceff99eb924156155e69a20f587a4f92b83f0b
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/519
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-02-17 10:06:30 +00:00
Alexey Marchuk
63c50be82e test/nvme: correct nvme controller version
nvme_ctrlr_identify_active_ns checks ctrlr.vs bits but not ctrlr.cdata.ver

Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Change-Id: I2d2e4e81de7308dc275f48fda52d7d3ab0cdb74d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/505
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-02-10 10:25:38 +00:00
Alexey Marchuk
93988db982 test/nvme: add tests to cover spdk_nvme_wait_for_completion
Change-Id: I5dc600b6ae7993d93494092d5fb4a81f840a06a9
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/504
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-02-10 10:25:38 +00:00
Alexey Marchuk
48f38636ce tcp: Cleanup nvme_tcp_pdu structure
With the recent changes which added usage of writev_async to
both TCP target and initiator, nvme_tcp_pdu::writev_offset
becomes useless since it is not updated in data path. This
field is only used in UT. Remove this field from nvme_tcp_pdu
structure, now nvme_tcp_build_iovs builds iov which fully
describes the PDU. Update UT accordingly.

Field padding_valid_bytes is not used at all, delete it too

Change-Id: I2d6040ae64d6847cb455f59f65ec5677de8e5192
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/483374
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
2020-02-04 18:18:49 +00:00
Alexey Marchuk
8818ace2f4 nvme: Don't use stack variable to track request completion
A pointer to a stack variable is passed as an argument to
nvme_completion_poll_cb function, later this variable is used
to track completion in the spdk_nvme_wait_for_completion() function.
If normal scenario a request submitted to the admin queue will be completed
within the function which submitted the request.
spdk_nvme_wait_for_completion() calls nvme_transport_qpair_process_completions
which may return an error to the caller, the caller may exit from the
function which submitted the request and the pointer to the stack variable
will no longer be valid. Thereby the request may not be completed at that time
and completed later (e.g. when the controller/qpair are destroyed)
and that will lead to call to nvme_completion_poll_cb with the pointer
to invalid stack variable.
Fix - Dynamically allocate status structure to track the completion;
Add a new field to nvme_completion_poll_status structure to track status
objects that need to be freed in a completion callback

Fixes #1125

Change-Id: Ie0cd8316e1284d42a67439b056c48ab89f23e0d0
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/481530
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-01-27 22:48:18 +00:00
Alexey Marchuk
52f1e4b029 nvme: Update spdk_nvme_wait_for_completion_robust_lock error handling
Update error handling of spdk_nvme_wait_for_completion_robust_lock to
differentiate cases when request is completed (possibly with error)
or polling was aborted by transport/device error
The function returns 0 on success, -ECANCELED if transport/device
error occurred and -EIO if the request is completed with error

Change-Id: Ibf7f3c330317af0d8f27ba9cd10d8b773f6a796b
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/481529
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-01-17 08:59:46 +00:00
Alexey Marchuk
9ad2046ae1 nvme: Update spdk_nvme_wait_for_completion_timeout error handling
Update error handling of spdk_nvme_wait_for_completion_timeout to
differentiate cases when request is completed (possibly with error)
or polling was aborted by timeout or transport/device error
The function returns 0 on success, -ECANCELED if transport/device
error occurred or operation timed out and -EIO if the
request is completed with error

Change-Id: I314f40d1acaa6cfa9b88e5417b1ee2c9801bbbd6
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/481528
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-01-17 08:59:46 +00:00
Seth Howell
f6cf92a31f lib/nvme: make transport.c use fn tables.
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: Ida58785784b4ed50393e1d43a9cd902de74a2eaa
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478873
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-01-16 09:10:38 +00:00
Seth Howell
e4eef6975c lib/nvme: add function tables for all transports.
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I4e7af1c42a19346f4abcb17910a41f8104a2de1b
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478871
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-01-16 09:10:38 +00:00
Seth Howell
5cddf93b47 test/nvme: add common_stubs.h file.
A lot of the stubs between these transports will end up being common.

Change-Id: Ib9c8ff947b95f34633eb13953405d3153a7f4ac7
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/479602
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-01-16 09:10:38 +00:00
Seth Howell
771d759312 lib/nvme: add spdk_nvme_transport_available_by_name
This new api function will enable us to work with custom transports.

This is needed to enable properly parsing and comparing custom transport
IDs that may all resolve to the same enum value.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I26aa3cb8f76f8273f564799d9b2af8041ea0d219
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478752
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-01-16 09:10:38 +00:00
Seth Howell
a050dcf21d lib/nvmf: add a default value to trtype enum.
For custom transports, we should use a range outside the spec value
to identify them.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I82b29c349e143b8906f79ce2de818def116a3fe4
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478747
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-01-16 09:10:38 +00:00
Seth Howell
7ed0904b9b lib/nvme: update trid struct with trstring.
The trtype should be stored as both an enum and string. This is intended to
help pave the way for pluggable NVMe-oF transports.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I6af658d7a17c405e191ff401b80ab704c65497e7
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478744
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
2020-01-16 09:10:38 +00:00
Maciej Szwed
393afcb1d6 nvme: Unit test for spdk_nvme_ns_cmd_comparev_with_md
This is vectored version of test_nvme_ns_cmd_compare_with_md
test.

Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I8af3a50ae764908024dc2758bb53b119b86edb2c

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478947
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-01-15 12:25:26 +00:00
dongx.yi
1044fe25e6 test/unit: Add set_host_id_cmds to unittest.
This function was ignored and a little different from others.
Add this to increase code coverage rate.

Signed-off-by: dongx.yi <dongx.yi@intel.com>
Change-Id: I35b51feb8894aca54a3b21436bc21ec12e37adf7
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/475814
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-08 09:12:10 +00:00
Tomasz Kulasek
1c5d980300 lib/nvme: add hw support for fused operations
Change-Id: I2a184a2d60f18439947afa2d4855d51b92ed282e
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476137
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2020-01-07 12:31:32 +00:00
dongx.yi
94f2def65a test/unit: Add nvme_uevent_ut.c to unittest.
Add nvme_uevent_ut.c to unittest, and test parse uevents.

Signed-off-by: dongx.yi <dongx.yi@intel.com>
Change-Id: Id8bb8336790ae5e031b357dc49163d86dfa1cd57
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/475080
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-07 12:20:01 +00:00
Changpeng Liu
ff9516bdcc nvme: call the callback for the queued requests when there is submission failure
For the requests which don't have children requests, SPDK may queue them to
the queued_req list due to limited resources, in the completion path, we
may resubmit them to the controller.  When the controller was removed
the submission path will return -ENXIO and we will free the requests directly,
so the callback will not be trigerred for these requests.  Here we added a
flag to indicate the request is from queued_req list or not, so for the failure
submission, we can triger user's callback.

Fix issue #1097

Change-Id: I901ac81733c2319e540d24baf5b8faa1c649eb35
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/477754
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
2019-12-20 10:04:57 +00:00
Changpeng Liu
10cd9e86b9 UT/nvme: use STUB macro for the empty function APIs
Change-Id: Id037a0cdc1e054dad0454e0897b04629b78e7d50
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478018
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-12-20 10:04:57 +00:00
Ben Walker
bed4cdf6c7 nvme: Use sgls, if available, even for contiguous memory
The hardware sgl format can describe large contiguous
buffers using just a single element, so it's more
efficient that a prp list even for a single memory
segment. Always use the sgl format.

Change-Id: I9c62582829f0d64dcd1babdbc48930ddb4d9e626
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/475542
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-12-11 11:06:57 +00:00
Changpeng Liu
f4e3f59e0e nvme: fix potential memory leak when there is controller scan failure
The nvme_transport_ctrlr_scan() may return failure while there are
multiple controllers, so the probe context's init_ctrlrs list may
not null for this case, so when free the probe context, let's ensure
there is no controller in the init_ctrlrs list.  Also added a UT to
cover this case.

Fix issue #1095.

Change-Id: I4d9a10ad73cf00bbe159edd1f5b919797333feb6
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476969
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-12-09 13:58:01 +00:00
Seth Howell
61537a190e nvme: replace nvme_qpair_state_equals.
nvme_qpair_get_state fits more closely with the semantics in other
modules.

Change-Id: I6ea8e02abe27253d9b4d779a43ac1963be56356a
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476920
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
2019-12-09 13:55:41 +00:00
Seth Howell
3911922005 nvme: remove redundant transport_qp_is_failed checks
The qpair state transport_qpair_is_failed is actually equivalent to
NVME_QPAIR_IS_CONNECTED in the qpair state machine.

There are a couple of places where we check against
transport_qp_is_failed and then immediately check to see if we are in
the connected state. If we are failed, or we are not in the connected
state we return the same value to the calling function.

Since the checks for transport_qpair_is_failed are not necessary, they
can be removed. As a result, there is no need to keep track of it and it
can be removed from the qpair structure.

Change-Id: I4aef5d20eb267bfd6118e5d1d088df05574d9ffd
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/475802
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-12-09 13:55:41 +00:00
Changpeng Liu
0c9057f031 nvme: replace cdw11 with specific union strucutre
Change-Id: I1152b5d6d5f8e3c2f96dcca1353d85a410924fb4
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/475467
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-12-06 14:43:41 +00:00
dongx.yi
44ac51f00d test/nvme: Fix static memory accessing overflows.
This is a workaround for issue #1083.
Using constructing nvme_pcie_ctrlr to access spdk_nvme_ctrlr,
this avoid reporting :
Memory access at offset * overflows this variable.

Signed-off-by: dongx.yi <dongx.yi@intel.com>
Change-Id: Ided625f95ff33df277c28e0410e946f99b787550
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476420
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: JinYu <jin.yu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
2019-12-04 15:13:57 +00:00
Ben Walker
5c5b167a0d test/nvme: Convert some mocked functions to DEFINE_STUB
Change-Id: I77593507af570fa8999d43f26077a52a3fc29fe1
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/475778
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-11-28 12:38:21 +00:00
Ben Walker
15427fd1ac test/nvme: Simplify the nvme_pcie_hotplug_monitor test
Change-Id: I6eef2a81b03df7fc257b59fe22b3679830249c59
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/475777
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-11-28 12:38:21 +00:00
Ben Walker
8c81d7ed01 test/nvme: Remove dead code in nvme_pcie_ut.c
This was an old set of unit tests that was commented out. It's now
hopelessly out of date, so remove it. The rest of this series will
add in equivalent tests that work on the new code.

Change-Id: Ib74072c7a9156647ecff9ebd76997d46fec3c0cf
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/475776
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2019-11-28 12:38:21 +00:00
Tomasz Kulasek
9eb0ffa90c lib/nvme: change api for io message
API changes in this patch:

 1) nvme_io_msg_ctrlr_start                         => nvme_io_msg_ctrlr_register
 2) nvme_io_msg_ctrlr_stop with (shutdown == false) => nvme_io_msg_ctrlr_unregister
 3) nvme_io_msg_ctrlr_stop with (shutdown == true)  => nvme_io_msg_ctrlr_detach


Change-Id: I60153ebbfb0d0b22575128d106f9333c3887213d
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/474096
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-11-21 08:01:57 +00:00
dongx.yi
94ec4785ef test/unit: Add test_nvme_pcie_hotplug_monitor to nvme_pcie_ut.c.
Add this test to increase code coverage rate.

Signed-off-by: dongx.yi <dongx.yi@intel.com>
Change-Id: I46533dcd1a528ae7385cdaa143aea9c396e2942c
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/472915
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-11-20 10:03:00 +00:00
Seth Howell
f6646fd9fa nvme/tcp: detect cq errors.
We should alert the upper layer when the qpair becomes unusable due to
qpair errors.

Change-Id: Icdee3b55a14441a60111f3bd7a44dceef93bbb09
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/474095
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-11-15 18:27:27 +00:00
Jim Harris
e93449910a nvme: use -EFAULT for vtophys-related failures
Currently we have a mix of -1 and -EINVAL which
is confusing, especially since these types of failures
also result in the caller's callback routine getting
invoked.

While here, document this new -EFAULT return code for
all of the functions that could return it.

Fixes issue #797.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I8dfbba0ec0b83db0f2ec055b15830981af1965df

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/473054
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: yidong0635 <dongx.yi@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2019-11-06 15:19:48 +00:00
Seth Howell
13f30a254e nvme: don't disconnect qpairs from admin thread.
Disconnecting qpairs from the admin thread during a reset led to an
inevitable race with the data thread. QP related memory is freed during
the disconnect and cannot be touched from the other threads.

The only way to fix this is to force the qpair disconnect onto the
data thread.

This requires a small change in the way that resets are handled for
pcie. Please see the code in reset.c for that change.

fixes: bb01a089

Change-Id: I8a39e444c7cbbe85fafca42ffd040e929721ce95
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/472749
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-10-31 04:50:59 +00:00
Maciej Szwed
72e079a880 lib/nvme: implement external IO processing
This patch allows to send IO requests from external module to the nvme
device.

External module should call nvme_io_msg_ctrlr_start() to start IO message
producer on the controller and enable sending messages.

nvme_io_msg_send() is used to send IO to NVMe driver thread context,
where passed function will be called. Allowing the external module to
issue IO as needed.

NVMe driver users should poll spdk_nvme_io_msg_process() to move forward,
sending IO from external module and process their completions.

Change-Id: Ie59abac69870c4e4daa50120c747f3b620395921
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/471386
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-10-24 23:43:59 +00:00
Seth Howell
5215fad6ef nvme: Add an API for updating a controller trid.
This can be useful when trying to perform multipath failover at the
application level. However, the controller must be in the failed state
before calling this function.

Change-Id: I5403c0036fed5dd3600ee20592925297494ba8aa
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/470699
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-10-24 17:36:31 +00:00
Seth Howell
e45b619c3d nvme: add function to reconnect qpairs.
While it is unlikely that a single qpair will be failed, it is important
to make it possible to reconnect a single qpair.

This function is also handy at the application layer when going through
a reconnect workflow. If we get -ENXIO from a qpair when we poll, we
will turn around and call this function. If we get -ENXIO from this
function, then we know the whole controller is failed and we need to do
a reset.

Change-Id: I6a8ea0ce27fce2f5fc0a5b3db05834acd68e6a39
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/471417
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-10-24 17:36:31 +00:00
Seth Howell
efc0a86426 test/nvme: add process_completions unit tests.
Change-Id: Idfa3631fca537f90b2d2808d162dbf74e891e862
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/471694
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
2019-10-24 17:36:31 +00:00
Seth Howell
08d4d977e8 nvme: combine qpair->is_connecting and is_enabled
These will form the base of a little state machine for managing the nvme
qpair structure.

Change-Id: If6f6df38cc17221ac8fcb7d8c0d7e2e808897a99
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/470534
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-10-22 21:14:22 +00:00
Seth Howell
5cd7634939 nvme_ctrlr: enable the admin qpair before init.
The driver has historically waited until we have to do a listen
before enabling the admin qpair. That is a very PCIe-centric mindset.
For fabric controllers, a lot of the early initialization operations such
as get_cc and set_cc are handled through the admin qpair so it should be
enabled before we begin the initialization process.

As a side effect of this cahnge, the internal API
nvme_ctrlr_enable_admin_qpair has been removed. It would have turned
into a one-liner.

Change-Id: Icd162657d01a85c227a3f20c295d0208e07ce44d
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/471743
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
2019-10-22 21:14:22 +00:00
yidong0635
2dab6e4ab2 test/nvme: Add test_doorbell_buffer_config for nvme_ctrlr_cmd.
Add test_doorbell_buffer_config in nvme_ctrlr_cmd_ut.c to increase
code coverage rate.

Change-Id: If79cca086800dfe18fa041de9262a2e32ff1ebc1
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/470707
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-10-11 18:47:06 +00:00
yidong0635
729fa9743f test/nvme : Add test io_cmd_raw_no_payload_build to unittest.
This function has not been tested, add test in nvme_ctrlr_cmd_ut.c
to increase code coverage rate.

Change-Id: Ibe5c98924d9e2f3239d782e7a32b828caaddb04a
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/470718
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-10-10 16:21:09 +00:00
Seth Howell
584a630287 nvme: don't fail the ctrlr from ctrlr_process_init
If we are to have multiple reconnect attempts, we have to control
whetehr the controller is placed in the failed state from outside the
reset function itself. This will allow us to fail the controller only
after all of our retries are exhausted.

Change-Id: Ia82e10325272f25b2b8527336dc3bc507c93b401
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/469932
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
2019-10-07 15:05:00 +00:00
Seth Howell
f5d88e46e2 nvme: always set ctrlr->is_failed through API
Use the standard API function to fail the controller in all cases.

This patch, and the several following patches are aimed at creating a
mechanism for reporting up to the application layer that a controller is
failed and or removed. To do this, I use the reset_cb to inform the
upper layer that the controller is failed.
This also requires changes to how we handle a controller reset to
pave the way for doing optional reset retries in the libraries.

Change-Id: I06dfce08326c23472a1caa8f6efbac2fd1a720f2
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/469635
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-10-07 15:05:00 +00:00
yidong0635
46d6711825 test/nvme: Add test sanitize to unittest.
Add test_sanitize in nvme_ctrlr_cmd_ut.c, this will
increase code coverage rate.

Change-Id: I80dec6eed5e2114c91db1406d1aa6c6a5e287293
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/469576
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-09-27 04:23:10 +00:00
Benjamin Saunders
6bcd3588d1 nvme: add support for write uncorrectable command
Change-Id: I9fb7a998f7c13ce53cba630a895e8e11cf5f4a1c
Signed-off-by: Benjamin Saunders <bsaunders@google.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/467559
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-09-26 18:42:57 +00:00
Darek Stojaczyk
c049304a95 env: add spdk_pci_device_unclaim()
spdk_pci_device_claim() could create a file on the
filesystem that couldn't be deleted programatically.
It could only be overwritten - e.g. by another spdk
instance - but this didn't really work if that
another instance had less privileges and hence no
access to the previous file.

This is exactly the case we're seeing on our CI when
running SPDK as non-root. In general it's a good idea
not to leave any leftover files, so now we'll delete
the pci claim file when the spdk process exits.

spdk_pci_device_claim() used to return a file descriptor
that could be simply closed to "un-claim" the device.
It'll now return only a return code. The fd will be
stored inside spdk_pci_device and will be closed either
when user calls the newly introduced spdk_pci_device_unclaim(),
or when the device is detached.

We'll still need to clean up those files somewhere in
our test scripts (probably ./setup.sh cleanup) to
clean up after crashed processes or so - but we don't
necessarily want to run such scripts inside the autotest
whenever a non-root spdk is about to be started.

Change-Id: I797e079417bb56491013cc5b92f0f0d14f451d18
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/467107
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-09-18 20:34:39 +00:00
Ziye Yang
ea5ad0b286 nvme/tcp: Change hdr in nvme_tcp_pdu to pointer
Purpose: Prepare the further optimnization in the
target side whening receving pdu headers, we expect
to use zero copy.

Change-Id: Iae7f9106844736d7160d39d0af1f5941084422ec
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/465380
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
2019-08-28 15:38:02 +00:00
Jim Harris
32e22643ef nvme: add NVME_QUIRK_DELAY_BEFORE_INIT quirk
Currently we *always* wait 2 seconds before starting
controller initialization during attach.  This
works around an issue where some older Intel NVMe SSDs
could not handle MMIO writes too soon after a PCIe
FLR (which would be triggered when VFIO was enabled).

After further discussion with Intel experts, we know
the SSD models that exhibit this issue.  So we can
quirk this so that only the older SSDs incur the extra
delay.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ieb408c24f6afd5bd5147d1c87239aa20f2d13511

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/466064
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
2019-08-26 17:35:06 +00:00
Changpeng Liu
62bb65289d nvme: change retry count can be configured via bdev nvme driver
Also eliminate 'spdk_nvme_retry_count' finally.

Change-Id: I2f3e390e4b8a49208a11b54bb82c4891cf3e1845
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/464473
Reviewed-by: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-08-09 00:44:50 +00:00