Commit Graph

558 Commits

Author SHA1 Message Date
Jakub Radtke
b3497d5da7 nvme: add Get/Set Feature wrappers with NSID
Some features are per namespace and require a namespace ID; the existing
get/set feature functions do not allow passing NSID, so add new API
functions that do:
* spdk_nvme_ctrlr_cmd_get_feature_ns
* spdk_nvme_ctrlr_cmd_set_feature_ns

Change-Id: Ia7dc96e57badf1a56489785fb288493592b58b80
Signed-off-by: Jakub Radtke <jakub.radtke@intel.com>
Reviewed-on: https://review.gerrithub.io/414706
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-06-12 17:59:33 +00:00
Jakub Radtke
caac0ed9e8 ocssd: nvme driver public API extension for Open-Channel
This patch adds to nvme driver new public API dedicated for
Open-Channel:
* spdk_nvme_ocssd_ctrlr_cmd_geometry

Change-Id: I69592618d9bcc280346258e771195683d5b33893
Signed-off-by: Jakub Radtke <jakub.radtke@intel.com>
Reviewed-on: https://review.gerrithub.io/412816
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-06-12 17:59:33 +00:00
Xiaodong Liu
5f1c1189fa nvme: add PI to spdk_nvme_ns_cmd_read/writev()
apptag and its mask are required to set proper PI
elements in nvme read/write cmds.

Change-Id: Ibabc4738f637d13ea16246d5e77e7d045f032af8
Signed-off-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/414786
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Young Tack Jin <youngtack.jin@circuitblvd.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-06-12 17:40:32 +00:00
Changpeng Liu
967339f3e5 nvme: factor out qpair destruction function
Factor qpair destruction function so that we can put common
resource release together in future.

Change-Id: I44139947820c2a384b745ae2673799f1b736369c
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/412604
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-06-08 00:26:05 +00:00
Daniel Verkamp
8b158aaacd nvme: factor out request timeout checking
This will be used in other transports as well.

Change-Id: I05026b0dfea2647d61a173379aca368ca48a2f52
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/413864
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-06-07 15:46:50 +00:00
Daniel Verkamp
cdb3a594a8 nvme: ignore requests submitted before timeout set
If requests are in flight at the time a timeout callback is configured,
we can't retroactively get the submission time of those requests; treat
them as not having a timeout.

Change-Id: Ic589e874a0f2c8c7f4ce352afa7c3aea33b01fae
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/413863
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-06-07 15:46:50 +00:00
Daniel Verkamp
e7b29a14c0 nvme: move timeout information to nvme_request
This is the first step toward timeout handling for other transports.

Change-Id: I386dd990f667d449e94ba4bcedaa3435743755fd
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/413862
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-06-07 15:46:50 +00:00
Daniel Verkamp
1681a055f9 nvme/pcie: restore timeout checking on admin queue
This was partially fixed in commit ddeaeeec19 ("nvme: Only check
timeouts on requests from the same process"), but the function that
calls nvme_pcie_qpair_check_timeout() was also erroneously filtering out
the admin queue.  Restore the original behavior of checking all queue
types.

Change-Id: I26a44ff5eb772735d314ce7b8322ba9222675911
Fixes: 31bf5d795e ("nvme: make timeout function per process")
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/411628
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-06-07 15:46:50 +00:00
Young Tack Jin
f52d800864 nvme: add metadata to spdk_nvme_ns_cmd_read/writev()
verified on QEMU NVMe for OCSSD 2.0 and RFC will be shared soon

Change-Id: I01c513e30768ac1961a5c5af20de3a2490da7729
Signed-off-by: Young Tack Jin <youngtack.jin@circuitblvd.com>
Reviewed-on: https://review.gerrithub.io/413854
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-06-06 23:20:43 +00:00
Changpeng Liu
4c4c8ca269 nvme: don't reset controller when processing NS mangement commands
For attach/detach command, the identify namespace data should be
updated during the namespace attribute notice event callback, while
here, in case of the driver may not set aer callback, so update the
namespace identify same with attach command.

Change-Id: Ie594b2ff646a67488d03af2771c00f9947395aba
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/412884
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-06-06 20:02:16 +00:00
Daniel Verkamp
008c111b48 nvme/pcie: only set mmio_ctrlr around actual MMIO
When the shadow doorbell feature is enabled, no actual MMIO takes place
when a shadow update is sufficient; tighten the bounds of the updates to
g_thread_mmio_ctrlr in the two doorbell update locations so that we only
need to touch the thread-local variable when actual MMIO access is
required.

Change-Id: Ida974bec33f56cbb9f7d3611f483c6975ec773ab
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/413856
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-06-06 17:10:09 +00:00
Daniel Verkamp
e83a62bf09 nvme: rename quirk from LIGHTNVM to OCSSD
This is consistent with the Open Channel naming we are using elsewhere.

Change-Id: Ib088359bed29a958f8b50e41cf34143a23429f54
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/413840
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-06-06 01:45:36 +00:00
zkhatami88
0cdb08b0e0 env: add size parameter to spdk_mem_map_translate
Change-Id: I808101edaf4d75613baf19a950915f1d8e75b1af
Signed-off-by: zkhatami88 <z.khatami88@gmail.com>
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/413154
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Zahra Khatami <zahra.k.khatami@oracle.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-06-05 18:36:00 +00:00
Xiaodong Liu
c6ae008db5 lnvm: add a quirk for identify LightNVM device
Change-Id: I841d7b47bc85498abb608944587e7b7585138263
Signed-off-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/411588
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-06-04 03:31:46 +00:00
Jim Harris
13ed998697 nvme: make basic request functions static inline
This reduces submission+completion time by 10-15
core clocks per IO on an Intel Xeon Platinum
processor.  Similar improvements should be seen
on other processors as well.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I3241ba53ef5f21a8eef930b523a951525922e6b8

Reviewed-on: https://review.gerrithub.io/413284
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2018-06-04 03:28:19 +00:00
Jim Harris
9c4679bc77 nvme: rename g_pid to g_spdk_nvme_pid
This variable will no longer be static in a future patch.
So make the name a bit more verbose since it will now
be visible outside of the module that defines it.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I5e720ef44aa5c5f38e0fe91de091a89b9970fcb7

Reviewed-on: https://review.gerrithub.io/413283
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2018-06-04 03:28:19 +00:00
Jim Harris
d76cd984aa nvme: optimize nvme_allocate_request memsets
The memset was zeroing a lot of bytes that get
initialized either later in this function or elsewhere
in the submission code path.  Eliminating these
extra memsets saves a few nanoseconds of CPU overhead
in the NVMe submission path.

Note: one use of the cpl data member depended on
the nvme_allocate_request memset.  Since this use
case is not in the primary I/O path, just memset it
in that specific location before using it.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ife483a4d9c24c033cc7d26d94ec1700905a936f4
Reviewed-on: https://review.gerrithub.io/413153
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-06-01 21:01:42 +00:00
Daniel Verkamp
7dff719f7e nvme: optimize layout of struct nvme_payload
Rather than storing nvme_payload::type explicitly, use the SGL reset
function pointer as an indicator: if reset_sgl_fn is non-NULL, then the
payload is an SGL type; otherwise it is a contiguous buffer type.

This eliminates the one-byte type member from struct nvme_payload,
making it an even 32 bytes instead of 33, allowing the removal of the
awkward packing inside struct nvme_request.

Change-Id: If2a32437a23fe14eb5287e096ac060067296f1dd
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/413175
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-06-01 21:01:42 +00:00
Daniel Verkamp
caf85d8f23 nvme: add macros to populate nvme_payload
The definitions of these macros will change in an upcoming patch that
modifies the way nvme_payload is laid out.

Change-Id: Ic6edc18928542b07be7519a72bdbf6babbeb0131
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/413174
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-06-01 21:01:42 +00:00
Daniel Verkamp
5c2ccd0628 nvme: add helper function to get payload type
This will simplify upcoming patches that change the way nvme_payload
stores its type.

Change-Id: Idf0a5b8dfd7d66a10f89254d2c5c54fee2968a43
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/413173
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-06-01 21:01:42 +00:00
Changpeng Liu
be1c82b7a8 nvme: update the NS identify data according to AER
Number of Namespaces of controller identify structure
defines the maximum number of namespaces supported by
this controller, for physical NVMe controllers, the NN
is a fixed number, while here, we set the same rule for
NVMeoF controllers.

After NVMe driver got namespace notice event, it should
update the namespace identify data structure for NS
attach/detach commands.

Change-Id: Id72a2600a2ce9492fa2d6e09924667acbb77ae43
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/412883
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-05-30 17:28:03 +00:00
Piotr Pelplinski
fbf06aca1a nvme: fail when request is too long
Fail when number of child requests is >= NVME_MAX_CHILD_REQUESTS.

Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Change-Id: I8c370053847c9f623b861137da8d2387a66fc030

Reviewed-on: https://review.gerrithub.io/408850
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-05-30 17:22:11 +00:00
Changpeng Liu
266f37a5a0 nvme: make response callback after moving completion queue head
For some cases, especially for Admin commands, there maybe has
recursive commands, e.g.: in AER callback we may send a new AER
request, in such case, the current code can't process such
case.  While here, move the completion queue head to next before
any response callback will fix such issue.

Change-Id: Ide56701d94615881790cf025ede2f07420b9b16e
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/412766
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-05-29 18:09:30 +00:00
Daniel Verkamp
c4bb0ea600 nvme: add helper to wait for internal commands
Factor out the common pattern of waiting for an internally-submitted
command to complete.  This will give us a convenient central place to
add error checking.

Change-Id: I65334d654d294cfb208fc86d16fa387ac5432254
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/412545
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-05-28 01:43:55 +00:00
Ben Walker
99c8c6d8e1 nvme: Allow users to request which I/O command set they'd like to use
As of NVMe 1.3b, there is only one command set. But pipe
this through the driver per-spec anyway.

Change-Id: I4faf8596f5ce638e5e2a500b424e00ceb6e89edc
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/412102
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-05-24 22:42:38 +00:00
Changpeng Liu
65e56caa17 nvme: wrapper request completion callback into a function
When IO is finished SPDK will trigger callback at controller layer,
while here, wrapper the completion callback into a function so
that we can add error injection at this function in following patch.

Change-Id: I7b7a6d278d87fd09a05f51f688398fdf2e9c4e05
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/411630
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-05-24 18:03:25 +00:00
Daniel Verkamp
955b295a58 nvme: make AER callback per-process
For the same reason as commit 31bf5d795e ("nvme: make timeout function
per process"), the AER callback also needs to be stored in the
per-process controller data structure.

Change-Id: I41425d81a2ab16c06ef9b900bef6a6128117fcb0
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/410953
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-05-22 18:17:43 +00:00
Daniel Verkamp
3148c48079 nvme/rdma: create per-process controller struct
The RDMA transport was not correctly registering new controllers on
creation.  RDMA doesn't support multi-process, but it still needs a
single per-process structure for the owning process.

Change-Id: I337ab9d00b468671e7c7a21175682beed4cfdd2e
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/410958
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-05-21 23:19:54 +00:00
Daniel Verkamp
943c7c69c3 nvme: hold ctrlr_lock when setting timeout callback
The per-process controller data may only be touched while holding the
ctrlr_lock.

Change-Id: I18c8c4e43db4d58e6b86f0c0fd222f6d30830b85
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/410952
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-05-21 18:22:50 +00:00
Daniel Verkamp
cbd9c241dc nvme: factor out process lookup into a function
Change-Id: I7598222db5d76c1a1578fbb5935d4348f7c62f54
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/410951
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-05-21 18:22:50 +00:00
Daniel Verkamp
f750c4ebb5 nvme: apply Identify CNS quirk to NS ID Descriptor
Avoid sending CNS values other than 0 and 1 to controllers with the
Identify CNS quirk added in commit 6442451b10 ("nvme: add a quirk for
identify 0x2").

This probably doesn't affect either of the controllers with the quirk in
practice, since this command is already only sent to controllers that
report NVMe version 1.3 or newer, but let's add the check anyway for
consistency.

Change-Id: I858c28ab7d43714c501dd22d6797259054f0f0a8
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/410941
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-05-17 23:11:35 +00:00
Young Tack Jin
6442451b10 nvme: add a quirk for identify 0x2
Change-Id: I9cfc237a8514a1d323313851e14576ba2ba69077
Signed-off-by: Young Tack Jin <youngtack.jin@circuitblvd.com>
Reviewed-on: https://review.gerrithub.io/410529
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-05-10 19:38:54 +00:00
Daniel Verkamp
ace321df22 nvme: add API to get maximum data transfer size
Change-Id: I8573732b3049e2a5b471e5a0313e39019fdaad5f
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/410518
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-05-09 17:20:11 +00:00
Daniel Verkamp
f7b58aea2b nvme: capture VS register at init time
This will be used later instead of retrieving VS (potentially via a Get
property command for Fabrics) multiple times.

The Active NS List code was previously depending on the VER field of the
Identify Controller data, but this was only added with NVMe 1.2, so we
can't rely on it to detect NVMe 1.1 controllers; it is changed to use
the new cache VS value instead.

Change-Id: Iba9ed5ecbc82b4654973438d119daba0c4cf0724
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/408895
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-04-30 17:19:07 +00:00
Daniel Verkamp
b8edc465e2 nvme: check Optional Asynchronous Events Supported
In addition to checking for the required version, also use the OAES bits
to determine which features to request.

Change-Id: Idb07e4175cca5609904876bd17d912b50bc6b62a
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/409352
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2018-04-27 23:11:20 +00:00
Changpeng Liu
f0f3a48f40 nvme: set AER configuration bits based on NVMe version
Change-Id: Ia2fd4d31560c9117b167cbf12d2749c303e4cb8c
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/408772
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-04-25 22:24:29 +00:00
Pawel Wodkowski
f9fae3f3a9 nvme: change num_children to be wider type
uint8_t is too small to handle huge payloads. 32M payload already
overflow this.

Change-Id: I083ba7d3ded25b99571d422b7a3a4e7653a8d231
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/408677
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-04-23 18:20:17 +00:00
Ben Walker
ddeaeeec19 nvme: Only check timeouts on requests from the same process
Change-Id: I80521c4c01daf033319f88cf273255387a7b5248
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/408403
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-04-20 16:23:36 -04:00
Daniel Verkamp
8e098e4190 nvme: verify remove_cb is non-NULL before calling
This fixes the VFIO hot-remove path, which called remove_cb without
checking to see if it had been specified by the user.  The normal uevent
removal path already checked for remove_cb.

Change-Id: I0ad8d2c90a77b16800a8b505cb69ea05b0706d70
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/408392
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-04-19 18:47:51 -04:00
Daniel Verkamp
3fa7c33ac1 nvme: require trid to be valid in nvme_ctrlr_probe
This is an internal NVMe driver function, so we don't need to allow for
the case where trid is NULL.  All callers already passed an address of a
local variable except the unit tests, which can be trivially fixed.

Fixes a static analyzer warning about trid being dereferenced in
nvme_transport_ctrlr_construct() before being checked for NULL in the
caller.

Change-Id: I2bfeb5c92a302093b7c7f2949adcd18baa11855a
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/408395
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-04-19 17:51:14 -04:00
Ben Walker
37ef0cca91 nvme: Don't expose admin qpair in timeout callbacks
This was broken by commit 31bf5d795

Change-Id: I8c81c7b76cd47db347ce9c3f8a0e8296b690cb49
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/408240
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-04-19 15:48:36 -04:00
Ben Walker
63fb2f772c nvme: Close file descriptor if controller fails to initialize
Change-Id: I9fee91ece79b204962a70fc49d9032abe2c55090
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/408218
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-04-18 15:35:12 -04:00
Jonathan Richardson
ce70f29662 nvme: Remove calls to getpid() when submitting nvme requests
As of glibc version 2.3.4 onwards getpid() is no longer cached. SPDK
makes calls to it in nvme_allocate_request() which is called for each
nvme request received. This results in a system calls up to millions of
times per second which slows down nvme submissions. Since the pid never
changes, it only needs to be called once in initialization per process.
This improves the performance of nvme_allocate_request() signficantly.

Change-Id: Idee0f06484d459906b9ce1d9b7360a33119c7e56
Signed-off-by: Jonathan Richardson <jonathan.richardson@broadcom.com>
Signed-off-by: Scott Branden <scott.branden@broadcom.com>
Reviewed-by: Ray Jui <ray.jui@broadcom.com>
Reviewed-on: https://review.gerrithub.io/407599
Reviewed-by: Scott Branden <sbranden@gmail.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-04-16 16:19:44 -04:00
Daniel Verkamp
fa6f7a166d nvme: improve error messages in set_num_qpairs
The functions they were referring to have been renamed; rather than
fixing up the function names, use the spec-defined NVMe command names so
it's more understandable. (The second message was also incorrectly
referring to "set" instead of "get", which is fixed as well.)

Change-Id: Id140a91c837d8c913760d2f55318472689c00f45
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/407593
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-04-16 12:31:26 -04:00
Daniel Verkamp
453c804d1b nvme: continue initialization even if NN=0
Similar to the previous patch, some controllers may report 0 namespaces;
we should still be able to send admin commands in this situation.

Change-Id: Ia14e3ce773c63e645199e1c40ba5b37095da2473
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/407497
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-04-13 15:59:00 -04:00
Daniel Verkamp
c9ef7642c9 nvme: make set_num_qpairs failure non-fatal
A controller that fails the Set Features/Get Features - Number of Queues
won't be able to create I/O queues, but it may still accept admin
commands.

Change-Id: Iec79d641f7d460448a8d8e1295764f1f03f98594
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/407378
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-04-13 15:59:00 -04:00
Ziye Yang
31bf5d795e nvme: make timeout function per process
Change-Id: I6e58baaeb09580b5f70e1acf5323376ca0b26bbf
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/407382
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-04-13 12:30:51 -04:00
Ben Walker
aedbb3b81a nvme: Rmove all uses of strncpy
strncpy is going to be added to the list of banned functions because
it does not guarantee strings are null terminated.

Change-Id: Ic18623c281cca7c3d87732bc7677b284d57685c8
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/407023
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-04-12 13:45:04 -04:00
Changpeng Liu
2d192cf8fb nvme: use AER configuation structure when starting controller
Change-Id: I79dad84d1dc58e61eb36b461b41fbd7ee73631fc
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/406899
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-04-10 15:43:38 -04:00
Daniel Verkamp
be4ac912a1 nvme/rdma: improve nvme_rdma_get_event() message
Add the event status code to assist in debugging, and reorder the
message so it is easier to read.

Change-Id: I2a1fb900471a482707fbfb6db84bc81cb3b0cdad
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/406636
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-04-06 18:30:58 -04:00