Commit Graph

179 Commits

Author SHA1 Message Date
Daniel Verkamp
8a6ba58cb4 scripts/check_format: check for spaces before tabs
Automatically detect more whitespace errors.

All existing cases are fixed; only whitespace change (verify with
diff -w) except for one comment style fixup in include/spdk/nvme.h.

Change-Id: If750e54b9c8e3421ea6feda5f20184a31431631e
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/402360
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-03-05 11:09:13 -05:00
Ehud Naim
c83cd9375b nvme: fixing ctrlr mutex free
adding nvme_ctrlr_destruct_finish because nvme_transport_ctrlr_destruct may
use a destroyed mutex.

nvme_ctrlr_destruct() free "ctrlr_lock" and after that call
nvme_transport_ctrlr_destruct()->nvme_pcie_ctrlr_destruct()(with pci)->
nvme_ctrlr_proc_get_devhandle()->nvme_robust_mutex_lock(&ctrlr->ctrlr_lock);

Change-Id: I55714ea9097d2c9d844a00b5a88fa2d51a3f4469
Signed-off-by: Ehud Naim <ehudn@marvell.com>
Reviewed-on: https://review.gerrithub.io/399605
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-02-20 12:31:34 -05:00
Daniel Verkamp
3839639759 nvme: add API to allocate CMB I/O buffers
Change-Id: I2a3c7a272dc08be5a5ecb4339622816482c4cbb0
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Signed-off-by: Stephen Bates <sbates@raithlin.com>
Reviewed-on: https://review.gerrithub.io/397036
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2018-02-12 19:11:44 -05:00
Pawel Wodkowski
891c12a63c util: add spdk_strerror() wrapper with TLS support
This patch remove need for additional buffer when translating error code
to string.

Change-Id: Iaa60088b5c450581d3cdddbb425119b17d55a44b
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/386114
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-01-04 15:00:09 -05:00
Ziye Yang
b8a5cb99af nvme_rdma: Add rdma_cm_event_str
To make the error message more clear.

Change-Id: I95a9a3e06f4473da30d022134a8da764f165a070
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.gerrithub.io/388880
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2017-12-08 14:07:00 -05:00
Daniel Verkamp
ea1c15791f log: rename SPDK_TRACE_* to SPDK_LOG_*
Disambiguate the log components from the trace functionality
(include/spdk/trace.h).

The internal spdk_trace_flag structure and related functions will be
renamed in a later commit - this is just a find and replace on
SPDK_TRACE_* and SPDK_LOG_REGISTER_TRACE_FLAG().

Change-Id: I617bd5a9fbe35ffb44ae6020b292658c094a0ad6
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/376421
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2017-12-07 12:23:19 -05:00
Daniel Verkamp
1804f3c528 nvme: move init_ctrlrs out of multi-process struct
Each process should manage its own list of controllers that are
initializing; the list doesn't need to be shared between processes.

This is the first step toward preventing non-PCI controllers from being
added into the shared attached_ctrlrs list.

Change-Id: Ia6f85fe89e28a04f0950da5362bb2f49d1b76da9
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/388695
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2017-11-28 16:21:07 -05:00
Ziye Yang
b114103736 nvme_rdma: set retry_count field in rdma_conn_param
For using SoftRoce, there will be packets loss,
so we need to use this value for re-transmission.
retry_count means the maximum number of times that
a data transfer operation should be retried on the
connection when an error occurs.

Change-Id: Ibd81a33d3838bbdf0c054f01666e7ec59bb0a38f
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/382079
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2017-10-11 13:59:10 -04:00
GangCao
6368d6c0f5 nvme: add new spdk_nvme_connect() API to directly attach the ctrlr
With this new API, callers can attach one specific ctrlr identified by
the transport ID directly along with optional ctrlr opts. If connecting
to multiple controllers, it is still suggested to use spdk_nvme_probe()
and filter the requested controllers with the probe callback.

Two primary use cases:
1) connecting to the NVMe-oF discovery controller
2) more straightforward way to connect a specific controller (avoiding
the probe callback)

A typical usage of this API with specific ctrlr_opts:
1. struct spdk_nvme_ctrlr_opts user_opts = {}
2. Call spdk_nvme_ctrlr_get_default_ctrlr_opts(&user_opts, sizeof(user_opts))
3. Modify the content of the initialized user_opts with user required value like
   user_opts.num_io_queues = 8
4. Call spdk_nvme_connect(&trid, &user_opts, sizeof(user_opts))

Change-Id: Idf67ee5966f6753918c12604342c892d2f3bbe3a
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.gerrithub.io/370634
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-10-09 12:31:12 -04:00
GangCao
dc2fb2ed39 nvme: add a public API to rename spdk_nvme_ctrlr_opts_set_defaults
This change is relating to add a new public API spdk_nvme_connect() under
include/spdk/nvme.h. This new spdk_nvme_connect() API will connect the user
specified trid and have a user optional ctlr opts. Rename this API and make
it as public.

A typical usage of this API as following:
1. struct spdk_nvme_ctrlr_opts user_opts = {}
2. Call spdk_nvme_ctrlr_get_default_ctrlr_opts(&user_opts, sizeof(user_opts))
3. Modify the content of the initialized user_opts with user required value like
   user_opts.num_io_queues = 8
4. Call spdk_nvme_connect(&trid, &user_opts, sizeof(user_opts))

Change-Id: Ideec8247365ebf7dd15069e29821be8ea27b08be
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.gerrithub.io/380849
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-10-09 12:31:12 -04:00
GangCao
6f227249fa nvme: add a new opts_size parameter for default ctrlr opts
Add a new parameter for the default ctrlr opts initialization.
This is to make sure future compatibility when SPDK components
are built as a shared library. User's version and SPDK's version
may be in different size.

The change here is to make sure the backward compatibility when
new fields are added in the struct spdk_nvme_ctrlr_opts.

Change-Id: Icfc9640993cb06063b825d4df5835d920dd374e5
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.gerrithub.io/380846
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2017-10-05 14:51:20 -04:00
Daniel Verkamp
8acc4ae5da nvme: add controller option to specify host ID
A random host ID is generated per SPDK application startup if the user
doesn't specify a host ID during controller startup.

This also changes the default host NQN for NVMe-oF connections to a
random UUID NQN based on the host ID.

Change-Id: Ib0f70dd63e53087716842b412a1f134a9991d4da
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/380528
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2017-09-29 11:43:25 -04:00
Changpeng Liu
8181c59293 nvme: override the default queue depth if users specified the value
SPDK already uses DEFAULT_IO_QUEUE_SIZE and MQES to decide the correct
queue depth of NVMe queue pair, hardcoded it to NVME_IO_ENTRIES(512)
does not make sense if users want to set queue depth bigger than 512.

Change-Id: Iaa73fc79e055292ae9bd19af0c8c12f257ae5c46
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/379052
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2017-09-20 19:58:49 -04:00
Daniel Verkamp
6847a679ea nvme/rdma: add option to specify source address
Allow the user to set the source address when connecting to a NVMe-oF
controller.

Change-Id: Ice3add4b2cd3b64fdb8d0d7807d2235f90fd86b1
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/375837
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2017-09-05 13:02:28 -04:00
Daniel Verkamp
932a186b4c log: remove uses of SPDK_TRACE_DEBUG
Replace SPDK_TRACE_DEBUG with component-specific flags.

Change-Id: Iee7eafab5e6ac8713f247323a18552b5afb0e86a
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/375834
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2017-08-29 15:07:13 -04:00
Daniel Verkamp
d92f0f75ca log: rename SPDK_TRACELOG to SPDK_DEBUGLOG
This matches the name to the behavior and prepares for addition of a new
log macro for "info" log level.

Change-Id: I94ccd49face4309d3368e399528776ab140748c4
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/375833
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2017-08-29 13:25:58 -04:00
Seth Howell
4d43844f4d lib: replace strerror with strerror_r
replaces all references to strerror in the spdk lib directory with
references to the thread safe strerror_r

Change-Id: I80d946cce3299007ee10500b93f7e1c8e503ee41
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/374012
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-08-15 16:47:01 -04:00
Dariusz Stojaczyk
b597955f29 env/vtophys: vtophys functions will now return error codes
Change-Id: Ic2b799c7d1dbd3249a890a79f42118fc79a30b94
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/373223
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2017-08-10 13:32:36 -04:00
Jim Harris
002660c4f0 nvme: add transport get_max_sges abstraction
For pcie, this just equals the number of SGLs we can fit
into the per-tracker memory.

For rdma, this is just set to 1 for now since nvme_rdma.c
does not support multiple SGEs yet.  Once that support is
added, this will change to use MSDBD (Maximum SGL Data Block
Descriptors) instead from the controller identify data.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I34a4c546b5ff46918a296a73ed8cbcc6c9879d5a

Reviewed-on: https://review.gerrithub.io/372358
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2017-08-02 17:15:21 -04:00
Jim Harris
75a1b39fcc nvme/rdma: allow for sge length > payload_size
Requests may need to be split in which case a child
request may have a size smaller than the SGE that
contains the child request's payload.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I6c7ab76104d56fa9dde168cfdddb6320c7157f98

Reviewed-on: https://review.gerrithub.io/372347
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2017-08-02 17:15:21 -04:00
Jim Harris
0158f354db nvme: only update cntlid after adminq connect
For ioq, target is supposed to respond with the same
cntlid as specified in the connect capsule, but the
Linux kernel target doesn't actually do that.  It really
only needs to be set on the adminq connect, so just
do that.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I4eec605f856bdad2c8614d505241566a9fe292ab

Reviewed-on: https://review.gerrithub.io/372345
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2017-08-02 17:15:21 -04:00
Daniel Verkamp
48b2e94f66 nvme/rdma: remove newlines from discovery strings
The Linux kernel NVMe-oF target can easily be misconfigured to report
newline-terminated strings from the discovery service, since its
parameters are set by echoing into sysfs files, and echo adds a newline
by default.

Newline characters are not allowed as part of the TRADDR and TRSVCID
fields, since they are defined as ASCII strings in the NVMe specification
sense, which limits the acceptable characters to the 0x20-0x7E range.

However, we can add a workaround for misconfigured targets with trailing
newlines that shouldn't impact any valid configurations.

Change-Id: I5db183d5637128fa8d50c245f4bfa965cc3ce8e2
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/370593
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2017-08-01 13:42:26 -04:00
Daniel Verkamp
ce4fcbce71 nvme: add I/O qpair creation options
Add a new struct spdk_nvme_io_qpair_opts to allow the user to override
controller options on a per-I/O qpair basis.

Existing callers with qprio == 0 can be updated to:

  ... = spdk_nvme_ctrlr_alloc_io_qpair(ctrlr, NULL, 0);

Callers that need to specify a non-default qprio should be updated to:

  struct spdk_nvme_io_qpair_opts opts;
  spdk_nvme_ctrlr_get_default_io_qpair_opts(ctrlr, &opts, sizeof(opts));
  opts.qprio = SPDK_NVME_QPRIO_...;
  ... = spdk_nvme_ctrlr_alloc_io_qpair(ctrlr, &opts, sizeof(opts));

Change-Id: I8ac3ea369535cfde759abbe75e1d974b6450a800
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/369676
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2017-07-18 18:33:20 -04:00
Daniel Verkamp
7538196f30 nvme/rdma: print getaddrinfo() error if it fails
gai_strerror() is documented as thread-safe, so this should be safe to
use from a library.

Change-Id: Ia0ec0b5c387db8993dd1ba3a3029562f25cc210d
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/369642
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2017-07-17 17:58:55 -04:00
GangCao
ec7dad7199 nvme-rdma: probe the discovery ctrlr when the subnqn is set
Change-Id: Ia15d65868773442937ce578f92b842989730804d
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.gerrithub.io/367010
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2017-06-30 12:46:05 -04:00
Daniel Verkamp
ad20a6ddc2 nvme/rdma: check discovery log page RECFMT
Change-Id: I1e272c63a5d863c92f1aa8299a9d98dcb72d0b13
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/365082
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ziye Yang <optimistyzy@gmail.com>
2017-06-14 13:55:07 -04:00
Ziye Yang
fb7ad4ac66 nvme_rdma: Support getting all log page entries
Preivously, we can only get part of the log page
entries due to the limitation of buffer. With this
patch, we can get all.

Change-Id: I81b8afc73c108dac65d1ff5fe8d03bda41fa29a1
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/363323
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2017-06-08 15:44:46 -04:00
John Meneghini
8a44220b1a env: Rename spdk_malloc/zmalloc/realloc/free to spdk_dma_(func)
- rename spdk_malloc_socket to spdk_dma_malloc_socket
  - rename spdk_malloc to spdk_dma_malloc
  - rename spdk_zmalloc to spdk_dma_zmalloc
  - rename spdk_realloc to spdk_dma_realloc
  - rename spdk_free to spdk_dma_free

Change-Id: I52a11b7a4243281f9c56f503e826fd7c4a1fd883
Signed-off-by: John Meneghini <johnm@netapp.com>
Reviewed-on: https://review.gerrithub.io/362604
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2017-05-31 15:30:27 -04:00
Daniel Verkamp
a02c0062bd nvme/host: use spec-compliant PSDT value 01b
Resolves GitHub issue #165.

Change-Id: If214bdfbe5b92faf2a6611d25e36c3c8876e3190
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/362239
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2017-05-23 17:10:50 -04:00
Daniel Verkamp
5f3841f0e3 nvmf: allow NQN to be 223 bytes + null terminator
An ECN to the NVMe 1.3 specification has clarified that the NQN may
contain 223 bytes before the null terminator.  Make all of our NQN
length checks consistently enforce this behavior.

Change-Id: Iebfd57d11abea64964c7a6ad9d886e40efa243c3
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-05-18 14:36:39 -07:00
Ben Walker
b961d9cc12 include: Move the remainder of the code base to stdinc.h
Change-Id: I6a142feeaad3117bd3c75e7c5cb7231a1cfa78ae
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2017-05-08 13:20:36 -07:00
Daniel Verkamp
5742e9b9e7 nvme: allocate requests on a per-queue basis
Change-Id: I7bec816e518a0a6f2e9fb719128c83d4b908d46c
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-04-12 21:05:24 -07:00
Daniel Verkamp
83e556534b nvme/rdma: eliminate bounce buffer copy
Register all spdk_malloc() memory regions as ibv_mr in a spdk_mem_map
so we can look up the RDMA key for the user's buffer and pass it in the SGL
directly, rather than copying through a pre-registered bounce buffer.

Change-Id: I7340bc2020b5256750c95dbd24ba67961404e5e7
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-03-10 11:39:28 -07:00
Ben Walker
ac9b92c853 nvme/rdma: Consolidate send_cq and recv_cq
The send completions must be processed prior to the
recv completions. However, if the completion queues
are separate this leaves a small window where
a send+recv completion arrive between polling
the send_cq and the recv_cq, resulting in the code
seeing the recv completion prior to the send
completion.

By combining the completion queues, this eliminates
any potential gap. The send completion will always
be processed before the recv completion.

Change-Id: I06bfef6af48559d0b9e00524ebc10f1a102e7387
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2017-03-09 15:08:18 -07:00
Daniel Verkamp
6a03451228 nvme/rdma: use spdk_malloc() for DMA data buffers
These allocations need to be from memory registered with the SPDK env
library to allow future work on automatic ibverbs memory registration.

Change-Id: I6ec6999ecd6d6bf6ba4ab159630f7d01f3d46154
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-03-03 10:23:20 -07:00
Ziye Yang
27cf11d04e nvme,rdma: optimize nvme_rdma_post_recv
Move the ibv_recv_wr initialization in
nvme_rdma_alloc_rsps. Thus we can save some
CPU times

Change-Id: Id449b2684290431f8b3ba97ec4058171d34038bf
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
2017-02-22 19:15:41 -07:00
Ziye Yang
b89e758c8d nvme.rdma: move ibv_send_wr init while allocate rdma reqs
We do not need to set it for submission since the contents
are same

Change-Id: I345094e2e8a858b318be73d28f09393566587d95
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
2017-02-22 19:14:54 -07:00
Daniel Verkamp
84d904841f util: move common helper functions to util.h
These were repeated a few different places, so pull them into a common
header file.

Change-Id: Id807fa2cfec0de2e0363aeb081510fb801781985
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-02-10 09:38:52 -07:00
Daniel Verkamp
018e6afff2 nvme: remove ctrlr_get_pci_id transport callback
The PCIe transport initializes the quirks directly, so the generic hook
to get PCI ID is no longer necessary.  This path was dead code.

Change-Id: I25bdaa598db53e4312a264d9d8356d1b416696e5
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-01-31 14:50:40 -07:00
Ben Walker
eb2ec1b07b nvme/rdma: Reap multiple completions per poll
This is more CPU efficient than only grabbing one
completion per call to ibv_poll_cq.

Change-Id: I0c70d33639f0f345482d9e7c810f9c6723937058
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2017-01-30 14:16:05 -07:00
Ziye Yang
47ae3957eb nvme/rdma: Invoke rdma_ack_cm_event in exceptional cases
In our previous code, we did not ack the event in
exceptional cases when we get a event via rdma_get_cm_event.
Thus, the code may block with in this statement:
rdma_destroy_id(rqpair->cm_id);
in some exceptiaonal cases. And this patch will solve this
issue.

Change-Id: Iddb6fb5356a5ee0ed04e261a040ba53042fca302
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
2017-01-26 10:48:23 -07:00
Ben Walker
d77c030172 nvmf: NVMe-oF 1.1 adds cntlid to RDMA private data
Change-Id: I44ec5264fc93fa85706750cb23bbd0ed0587db81
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2017-01-23 16:46:55 -07:00
Ziye Yang
d61ddd3c93 nvme/rdma: Support directly connect via trid
Preivously, we only supports probe the NVMf target
via discovery info, now we can support to directly
to connect it.

Change-Id: I08ce1d95de6744286357e68b48c97b773b902ac8
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
2017-01-13 10:57:03 -07:00
Ziye Yang
90f13aa634 nvme/rdma: Support sgl for readv/writev functions
(1) Add nvme_rdma_build_sgl_request function
(2) Merge nvme_rdma_pre/post_copy_mem to nvme_rdma_copy_mem

Change-Id: I86abab821b32b4da0aa9489a6b9f7dc430333159
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
2017-01-11 12:36:52 -07:00
Daniel Verkamp
f80c0f4fdd nvme: remove transport ctrlr_attach callback
Now that the hotplug code is isolated in nvme_pcie.c, it can call the
PCIe transport attach function directly.

Change-Id: I2df3b9168473b537cc9b13367e06d3d3b6fa22be
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-01-09 11:36:27 -07:00
Daniel Verkamp
df8129fb39 nvme: move num_entries to transport-specific qpairs
The spdk_nvme_qpair::num_entries value is never used in the common code,
so move it to the individual transport qpairs to make it clear that it
is a transport-specific implementation detail.

Change-Id: I5c8f0de4fcd808912ba6d248cf5cee816079fd32
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-01-05 15:49:09 -07:00
Ben Walker
c2d38ee8da nvme/rdma: Improve error message when polling fails
Change-Id: Ia75d62669f3c1f923031f6a33e4797cc56ac1141
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2017-01-05 09:24:02 -07:00
Ben Walker
d9a3a8e50c nvme/rdma: nvme_rdma_recv now takes the response index
Instead of passing the work completion, just pass the
response index. This keeps the work completions localized
to the polling function.

Change-Id: I0e6a1d8564200b5ac3aa43dfd58ae152d439bbd8
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2017-01-05 09:24:02 -07:00
Ben Walker
0ebf93e35c nvme/rdma: Pass qsize to nvme_rdma_ctrlr_create_qpair
This eliminates an if statement, since the two callers
of this function know the desired queue size.

Change-Id: I28fabac8613f7b8fc7d96cf95b085b6e4dcf985f
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2017-01-05 09:24:02 -07:00
Ben Walker
c26c655d59 nvme/rdma: Eliminate nvme_rdma_ctrlr_construct_admin_qpair
Just call the regular qpair create function instead.

Change-Id: Ic35b1eb6fcdf0d82733ea573a493f583dd63d5bd
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2017-01-05 09:24:02 -07:00
Ben Walker
3ad0c336c8 nvme/rdma: Eliminate max_queue_depth
Use the num_entries value in the generic qpair instead. These
values had to match anyway.

Change-Id: Ia6400fbaba97df3ef6db4dc07a2ab95af1e5143f
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2017-01-05 09:24:02 -07:00
Ben Walker
10aed30149 nvme/rdma: Handle requested qsize not matching obtained qsize.
Change-Id: I44d9e01dba87c2f78ef13296b78532255d749432
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2017-01-05 09:24:02 -07:00
Daniel Verkamp
e1b514ce9a nvme/rdma: cache value of bb_mr->rkey
Avoid an extra level of pointer chasing when we are filling out the NVMe
SGL.

Change-Id: I1a40af16fda80f7480c419524876bfb1a1902eb8
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-01-03 16:56:34 -07:00
Ben Walker
bf89b4da10 nvme/rdma: Eliminate _nvme_rdma_ctrlr_create_qpair
This intermediate function is no longer needed.

Change-Id: I3523cc6d8f3b290165a953d42cca8b76eda762c5
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-12-19 12:51:50 -07:00
Ben Walker
30677dc222 nvme/rdma: Move call to send fabric connect into qpair_connect
Sending the fabric connect command is part of establishing
a connection, so move it into the main connection-establishing
function.

Change-Id: I55e7ffdd16b576c81b51d7d3910203f9afc1f4c2
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-12-19 12:51:50 -07:00
Ben Walker
a96a6ecf58 nvme: Rename nvme_qpair_construct to nvme_qpair_init
This function initializes the members of an existing
qpair struct. It doesn't construct one from scratch.

Change-Id: I0b9afac1ad25cfb217efd146702f693c74f5f697
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-12-19 12:51:50 -07:00
Ben Walker
3e5c1d0a2b nvme/rdma: Don't allocate requests/responses until after connected.
No need to allocate all of the requests and responses until
we know a connection can be established.

Change-Id: I072a10aadfd7ced773634448f7d7e788622d0a4c
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-12-19 12:51:50 -07:00
Ben Walker
ee80b31cb7 nvme/rdma: Delete config_rdma_req
The code is clearer if this function is incorporated
into its only caller.

Change-Id: I33901cddf80ae27896b2acfd1b9e7d212f21f5f3
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-12-19 12:51:50 -07:00
Ben Walker
414702426d nvme/rdma: Rename nvme_rdma_bind_addr to resolve_addr
This is resolving the address and route to the target, not
binding a socket to an address.

Change-Id: I80055481ed2e020410a1e186a4e7371b60faaee9
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-12-19 12:51:50 -07:00
Ben Walker
42dc2836bc nvme/rdma: Add a utility for getting the next cm event
Change-Id: Ia7650e729ab233be6464fb0a190c9cd29f462fcf
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-12-19 12:51:50 -07:00
Ben Walker
fa97f35f06 nvme/rdma: Remove unused code in nvme_rdma_connect
Change-Id: I4f63136f57d074a9f7ff6027f2e8f6fb8a9c8dae
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-12-19 12:51:50 -07:00
Daniel Verkamp
b8da09b629 nvme/rdma: the NVMe SGL should use rkey, not lkey
We should be sending the bounce buffer's remote key to the target so it
can put it into an RDMA SGE on the remote side.

Change-Id: Icded155ad2292c67baa722f001c9c07178bc2754
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-12-15 16:39:53 -07:00
Daniel Verkamp
b74979421f nvme/rdma: remove write-only rdma_req->bb_len
Change-Id: Ied1bee7c8db87cca890d94e519ec07ae3fc658b2
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-12-15 16:39:53 -07:00
Daniel Verkamp
e8184c1f25 nvme/rdma: change max queue size from 127 to 128
There is no particular reason for this to be 127; make it 128 to at
least be consistent with the PCIe transport.

Change-Id: I60500e0044d3549ba6350e1f35f09d624848bd21
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-12-15 16:39:28 -07:00
Ben Walker
b2a86421f4 nvme/rdma: Fix specification of R/W queue depth
The meaning of the parameters was swapped.

Change-Id: I3f4938c446fd06afd1852bcf08e779b7b988855d
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-12-15 16:35:00 -07:00
Ben Walker
375da65762 nvme/rdma: Remove nvmf_cm_construct
This function was only called from one place and saved no
lines of code.

Change-Id: If5e653732df57c1f2c93e20cf4f286eac31df91c
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-12-15 16:35:00 -07:00
Daniel Verkamp
d24ea9009a nvme: add offset to spdk_nvme_ctrlr_cmd_get_log_page()
Change-Id: I1af4950995062f892062f29db85da2a8dd793d85
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-12-15 11:16:14 -07:00
Daniel Verkamp
9938985766 nvme/rdma: queue request if no rdma_req available
This matches the behavior of nvme_pcie, which queues a request if no
tracker is available.

Change-Id: Idbf6c951c89451cfea22ec6bc553ff46f988f818
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-12-15 11:13:09 -07:00
Daniel Verkamp
ee82713678 nvme/rdma: clean up nvme_rdma_req_init()
Make the caller pre-allocate an rdma_req and change req_init() so it
only does initialization, not allocation.

This is necessary to distinguish between rdma_req allocation failure and
other types of failures, which will become important in future patches
when requests will be queued if rdma_req allocation fails.

Change-Id: Ie6edebc1b5f05001b42fc959a29ce0ea6875e41e
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-12-15 11:09:50 -07:00
Daniel Verkamp
be92f47fa8 nvme/rdma: merge rdma_qpair_construct into alloc_reqs
Simplify the control flow and match the name of the function to its
purpose.

Change-Id: I65bad7e3b2ef710ca29eff9799b8dcaae3999315
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-12-15 11:09:50 -07:00
Daniel Verkamp
0eb3125531 nvme: remove transport qpair_construct callback
Make the qpair construct functions private to the transports - it
doesn't need to be called from generic code.

Change-Id: I5f730a4bcf60ce231fe27bc8f4c3c39cb647dd2d
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-12-15 11:09:49 -07:00
Daniel Verkamp
988906135c nvme: determine io_queue_size at startup
Add a transport callback to return the maximum queue size, and enforce
it in the generic nvme_ctrlr layer.

This allows the user to tell what io_queue_size was actually selected by
the transport via the ctrlr_opts returned during attach_cb.

Change-Id: I8a51332cc01c6655e2a3a171bb92877fe48ea267
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-12-15 11:09:48 -07:00
Daniel Verkamp
f505f57b36 nvme: add ctrlr function to set initial CAP value
Change-Id: Id7f68f279f87a08844015efcd35a790944b3bfbe
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-12-15 11:09:25 -07:00
Daniel Verkamp
fb31963c4b nvme/rdma: register cmds as a single MR per queue
Equivalent to commit 6ab28a201b except now
for commands instead of responses.

Change-Id: Ibe4382dc0f65c1b90c2cee2ad285bbdd21b96a89
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-12-15 10:58:23 -07:00
Daniel Verkamp
5ab3a71e2a nvme/rdma: remove unnecessary bb_sgl
The only field of bb_sgl that was actually used is lkey, and that is
already stored in bb_mr.

Change-Id: I790369a06ce223f88e356df20a9d9a74a93ff225
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-12-15 10:52:27 -07:00
Daniel Verkamp
5848faab5c nvme/rdma: split discovery probe into a new function
Refactor the discover log page processing into a loop that calls a
function for each log page entry.  This sets us up to add support for
multiple Get Log Page calls to handle larger discovery service lists.

Change-Id: I85676ada375d0dadda2a3f4ab6331123ac7aaf60
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-12-14 11:03:38 -07:00
Daniel Verkamp
32e6dd1573 nvme/rdma: bounds check discovery log page numrec
Change-Id: Ic7bcd6b7f42510be68f70fb3526a344754ecbbc0
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-12-14 11:03:38 -07:00
Daniel Verkamp
23fc6682fb nvme/rdma: use trid.adrfam when resolving address
Change-Id: I3c3c0faa808e6047d1f9297dc6985597cc0c4aaf
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-12-14 10:14:10 -07:00
Daniel Verkamp
0346dd775e nvme: add ADRFAM (address family) to transport ID
This isn't used yet in the NVMe library, but it will be necessary later
for supporting non-IPv4 addresses.

Change-Id: I167ce63ad25b0e0c9aa192b12d764c8d078e67f9
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-12-14 10:14:10 -07:00
Daniel Verkamp
92ee5fc4be nvme: rename queue_size option to io_queue_size
This better describes what the field controls (it does not affect the
admin queue size).

Change-Id: I851ae46fb4ed0fce819af07ae235824e0fc817e6
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-12-13 15:23:13 -07:00
Ziye Yang
c21e9fa5fb nvme/rdma: move ibv_send_wr/ibv_recv_wr usage from stack to heap
It will be more safe to do in this way.

Change-Id: I2bbe97ac72dc9f8b0e32a183f9aec5a628e0830a
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
2016-12-12 17:07:56 -07:00
Daniel Verkamp
4714e34d6e nvme/rdma: reset discovery trid for each entry
Don't leave garbage from previous discover entries in the trid we are
returning to the user.

Change-Id: I60ae5932db4a95cedb8df1ff98a2479220b55ce4
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-12-12 15:31:25 -07:00
Ben Walker
ec0b5d2ba1 nvme: Clean up redundant arguments in transport_ctrlr_construct
The other simplifications to probe_info and trid made the
trtype argument redundant.

Change-Id: Ie7bea4e2204e690dc4909eeacd065e0722b53272
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-12-12 10:49:17 -07:00
Ben Walker
32e838af3c nvme: Remove probe_info, just use transport_id
The probe_info was reduced to just containing a
transport_id, so remove probe_info entirely.

Change-Id: Ica9a22d126cd14e282decd3eea1a0afe0460f099
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-12-12 10:49:17 -07:00
Ben Walker
6c2e170dc7 nvme: Remove pci_id from probe_info
This can be obtained by parsing traddr into a pci_addr,
then getting a handle to the pci_dev and asking for all
of the pci information.

Change-Id: I1948cbd3ec65611293192ef5558ace19dd444d4c
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-12-12 10:49:13 -07:00
Ben Walker
34100924e4 nvme: Rename nvme_probe_one to nvme_ctrlr_probe
Simplify the arguments as well.

Change-Id: I653e1aad4e574cf95d793b41b749cb19e0024c2b
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-12-08 15:22:48 -07:00
Ben Walker
06557b0a4d nvme: Add remove callback to transport scan
Scanning the transport may result in both new
devices and removed devices, so pass the callback
for both operations.

Change-Id: I6f73dbe6fd7cf61575c354b43f8ae3e2a01e2965
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-12-08 15:22:47 -07:00
Ben Walker
d70ff832bf nvme: nvme_transport_ctrlr_scan now takes a transport id
Simplify the arguments to nvme_transport_ctrlr_scan to take
a transport id that identifies the discovery service (or
NULL to scan PCIe).

Further, separate scan into two functions - scan and attach.
Scan is for scanning an entire bus, attach is for a specific
device.

Change-Id: I464f351a02a04bc5a45096dcf5dc8fc5ac489041
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-12-08 15:22:47 -07:00
Ziye Yang
da43f64b17 nvme/rdma: Get right initiatior depth by ibv_device_query
Change-Id: I125e95dd405c21c5aea852e619801f1ef2de94ee
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
2016-12-08 14:18:36 -07:00
Ben Walker
5f78155fde nvme: Embed a transport_id in the discovery_info struct
Instead of repeating the fields, just embed a transport_id.

Change-Id: I282704c9d59784abd5f7c93be4e47c673fcf6dde
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-12-08 11:48:21 -07:00
Ben Walker
a2f35bcc16 nvme: Remove nvme_attach
Change-Id: Ia688e49574d8d80f1473f93f127586ca5aaa7d91
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-12-08 11:48:21 -07:00
Ben Walker
4af9f06c73 nvme: Rename discover_info to transport_id
This is a small step toward making discovery more like
scanning a local PCI bus.

Change-Id: Ie7149ad060f2eeb56939b1241187bdf09681f2aa
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-12-08 11:48:21 -07:00
Ziye Yang
f74d64f07f nvme/rdma: fix the nvme_rdma_pre_copy_mem
Before adding readv/writev support in nvme_rdma,
using this patch.

Change-Id: I25ff0df61d0346f22560d011158d7f80e72007ea
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
2016-12-08 10:33:05 -07:00
Daniel Verkamp
307d1320d8 nvme/rdma: use common Get Log page function for discovery
Change-Id: Ic635dabd039c56dfebdd0da64c4efe672b240013
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-12-07 13:36:24 -07:00
Ziye Yang
efd7ecd5d4 nvme/rdma: fix the kato value assginment in connection
Change-Id: Idc63a8f20a85bc73d48bd622b5a6e18ca4ed50d6
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
2016-12-07 12:34:02 -07:00
Ziye Yang
8eaf533156 nvme/rdma: set IOSQES and IOCQES in cc register
The reason is that kernel nvmf target will check the
value. If not set, it will fail the other commands later.
Even for discovery ctrlr, kernel nvmf target will
check the cc value.

Change-Id: I998327f91ba96281d261952878eb84d648a823da
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
2016-12-07 12:34:02 -07:00
Daniel Verkamp
152a3f0528 nvme/rdma: Connect command HSQSIZE is 0-based
Change-Id: I8e089c515bc6dfdf3088cfbb772d58527c6bd8de
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-12-07 12:34:02 -07:00
Ben Walker
5b8c0c5ace nvme: Rename spdk_nvme_transport to spdk_nvme_transport_type
It's not the whole transport - it's just an enum for the
type of transport.

Change-Id: Ia435a21792f221ddf50ddf4f0923c6152622eccb
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-12-07 09:47:00 -07:00
Ziye Yang
b93d5a7ae5 nvme/rdma: set NUMDL/NUMDU fields in Get Log Page
Change it according to the spec thus we can test
kernel nvmf target

Change-Id: Ica98dd40503a40c0f0de8efaefb1f6f67a89cde8
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
2016-12-06 12:57:57 -07:00
Ziye Yang
10690dedcd nvme/rdma: change the arg to 0 in nvme_rdma_fabric_prop_get_cmd
The buf is NULL, the size of buffer should be 0.

Change-Id: Iba89732ea6abeb65106ae1e158902e1dd9a1204c
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
2016-12-05 13:02:04 +08:00