Commit Graph

68 Commits

Author SHA1 Message Date
Daniel Verkamp
d63a30e39d nvme/pcie: return 1 when PCI address doesn't match
If the user asked for a specific PCI address in spdk_nvme_probe(), we
need to return 1, not 0, for the other PCI addresses that don't match
when enumerating.  0 means to attach the PCI driver, whereas 1 means to
continue enumerating.

With the previous behavior of returning 0, all NVMe devices would be
attached to the DPDK PCI driver, even if the user did not request for
them to be probed, and further calls to spdk_nvme_probe() would not find
any devices.

Change-Id: Ifbbcd7d1abe8ab535b6957855172e66a3e69fbe4
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-01-20 17:07:30 -07:00
Ziye Yang
0a573526b6 nvme/pcie: Add the support to probe nvme by pci_addr
Currently we use the pci functions provided by DPDK,
it identifies the device by class id related
info but not by pci bdf info, so we can add the filering
by pci_addr in pcie_nvme_enum_cb function.

Change-Id: I5942e98853f00fc10fa6aae5c113517653d1b357
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
2017-01-18 15:30:45 -07:00
Jim Harris
86e8a920bf nvme: split non-compliant SGLs into multiple requests
Since nvme_ns_cmd.c now walks the SGL, some of the test code
needs to also be updated to initialize and return correct values
such as ctrlr->flags and sge_length.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I521213695def35d0897aabf57a0638a6c347632e
2017-01-17 07:51:09 -07:00
Daniel Verkamp
f80c0f4fdd nvme: remove transport ctrlr_attach callback
Now that the hotplug code is isolated in nvme_pcie.c, it can call the
PCIe transport attach function directly.

Change-Id: I2df3b9168473b537cc9b13367e06d3d3b6fa22be
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-01-09 11:36:27 -07:00
Daniel Verkamp
df8129fb39 nvme: move num_entries to transport-specific qpairs
The spdk_nvme_qpair::num_entries value is never used in the common code,
so move it to the individual transport qpairs to make it clear that it
is a transport-specific implementation detail.

Change-Id: I5c8f0de4fcd808912ba6d248cf5cee816079fd32
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-01-05 15:49:09 -07:00
Cunyin Chang
6152d5bf8f nvme: move hot-plug logic to nvme pcie layer.
Change-Id: I2042b34e4284a9c59aa3092ccd061c075748880c
Signed-off-by: Cunyin Chang <cunyin.chang@intel.com>
2017-01-03 16:56:13 -07:00
Ben Walker
a96a6ecf58 nvme: Rename nvme_qpair_construct to nvme_qpair_init
This function initializes the members of an existing
qpair struct. It doesn't construct one from scratch.

Change-Id: I0b9afac1ad25cfb217efd146702f693c74f5f697
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-12-19 12:51:50 -07:00
Daniel Verkamp
0eb3125531 nvme: remove transport qpair_construct callback
Make the qpair construct functions private to the transports - it
doesn't need to be called from generic code.

Change-Id: I5f730a4bcf60ce231fe27bc8f4c3c39cb647dd2d
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-12-15 11:09:49 -07:00
Daniel Verkamp
988906135c nvme: determine io_queue_size at startup
Add a transport callback to return the maximum queue size, and enforce
it in the generic nvme_ctrlr layer.

This allows the user to tell what io_queue_size was actually selected by
the transport via the ctrlr_opts returned during attach_cb.

Change-Id: I8a51332cc01c6655e2a3a171bb92877fe48ea267
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-12-15 11:09:48 -07:00
Daniel Verkamp
f505f57b36 nvme: add ctrlr function to set initial CAP value
Change-Id: Id7f68f279f87a08844015efcd35a790944b3bfbe
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-12-15 11:09:25 -07:00
Daniel Verkamp
92ee5fc4be nvme: rename queue_size option to io_queue_size
This better describes what the field controls (it does not affect the
admin queue size).

Change-Id: I851ae46fb4ed0fce819af07ae235824e0fc817e6
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-12-13 15:23:13 -07:00
Ben Walker
ec0b5d2ba1 nvme: Clean up redundant arguments in transport_ctrlr_construct
The other simplifications to probe_info and trid made the
trtype argument redundant.

Change-Id: Ie7bea4e2204e690dc4909eeacd065e0722b53272
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-12-12 10:49:17 -07:00
Ben Walker
32e838af3c nvme: Remove probe_info, just use transport_id
The probe_info was reduced to just containing a
transport_id, so remove probe_info entirely.

Change-Id: Ica9a22d126cd14e282decd3eea1a0afe0460f099
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-12-12 10:49:17 -07:00
Ben Walker
6c2e170dc7 nvme: Remove pci_id from probe_info
This can be obtained by parsing traddr into a pci_addr,
then getting a handle to the pci_dev and asking for all
of the pci information.

Change-Id: I1948cbd3ec65611293192ef5558ace19dd444d4c
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-12-12 10:49:13 -07:00
Ben Walker
3da43e64e4 nvme: Remove pci_addr from probe_info
This can be obtained by parsing the traddr.

Change-Id: Idaf35066cbf900c87e771a44934de99fb5420001
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-12-12 10:46:50 -07:00
Isaac Otsiabah
7c60aec01e nvme: add I/O timeout callback
Change-Id: Ie6220590467b88fe1e63b4b0f8a01221fc0c1206
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-12-10 16:17:18 -07:00
Isaac Otsiabah
a4cc346b17 nvme: convert nvme_tracker list to a TAILQ
Change-Id: I822529b28b683f707604c07721040f22689961c4
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-12-10 15:45:53 -07:00
Ben Walker
8a9c1d4011 nvme: Make striping a quirk
Use the standard quirk mechanism to specify which devices
need software assisted striping.

Change-Id: Id8156876a90b4caf9d687637e14c7ad4a66ceda6
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-12-09 14:38:42 -07:00
Ben Walker
34100924e4 nvme: Rename nvme_probe_one to nvme_ctrlr_probe
Simplify the arguments as well.

Change-Id: I653e1aad4e574cf95d793b41b749cb19e0024c2b
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-12-08 15:22:48 -07:00
Ben Walker
06557b0a4d nvme: Add remove callback to transport scan
Scanning the transport may result in both new
devices and removed devices, so pass the callback
for both operations.

Change-Id: I6f73dbe6fd7cf61575c354b43f8ae3e2a01e2965
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-12-08 15:22:47 -07:00
Ben Walker
d70ff832bf nvme: nvme_transport_ctrlr_scan now takes a transport id
Simplify the arguments to nvme_transport_ctrlr_scan to take
a transport id that identifies the discovery service (or
NULL to scan PCIe).

Further, separate scan into two functions - scan and attach.
Scan is for scanning an entire bus, attach is for a specific
device.

Change-Id: I464f351a02a04bc5a45096dcf5dc8fc5ac489041
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-12-08 15:22:47 -07:00
GangCao
6bdcf5abe8 nvme: use nvme_robust_mutex related operations
Change-Id: I35416506dbafe5e9d21861e207e295e114bdb3db
Signed-off-by: GangCao <gang.cao@intel.com>
2016-12-07 13:46:03 -07:00
Ben Walker
5b8c0c5ace nvme: Rename spdk_nvme_transport to spdk_nvme_transport_type
It's not the whole transport - it's just an enum for the
type of transport.

Change-Id: Ia435a21792f221ddf50ddf4f0923c6152622eccb
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-12-07 09:47:00 -07:00
Daniel Verkamp
f93fd72680 env: split PCI drivers into individual files
Change the PCI enumeration API to individual functions per device type
so that only the drivers that are actually in use get linked into the
final executable.  All of the common code is still shared internally in
the env_dpdk library.

Change-Id: I2ba83afe59202a510f999a0674e23e60b6581221
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-12-06 09:30:55 -07:00
GangCao
bb726d516b nvme: add multi-process support
This version of multi-process support needs to have DPDK 16.11 builtin.

Change-Id: I3352944516f327800b4bd640347afc6127d82ed4
Signed-off-by: GangCao <gang.cao@intel.com>
2016-12-02 12:24:22 -07:00
Cunyin Chang
e8c63cdde0 nvme: Add support for hotplug.
Change-Id: Iac504ce15e4ea3100e5afa31764fcfff7f979dbb
Signed-off-by: Cunyin Chang <cunyin.chang@intel.com>
2016-12-01 10:21:59 +08:00
Cunyin Chang
d1a37e98e6 nvme: Make the nvme_transport_ctrlr_scan() could scan a specific device.
Change-Id: I74f7582f5175742e96268dc9c0cc83db2e1eaea0
Signed-off-by: Cunyin Chang <cunyin.chang@intel.com>
2016-12-01 10:17:41 +08:00
Daniel Verkamp
a9e436911f nvme: remove transport qpair_destroy() function
Each transport should handle its own qpair cleanup internally.

Change-Id: I7dd737be820ea6bad686f4aad7d74044fad58a47
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-11-30 17:10:39 -07:00
Daniel Verkamp
ba16e46349 nvme: pass opts and probe_info to nvme_attach()
Let the transport access the controller options during
ctrlr_construct().

Change-Id: I83590c111e75c843685dd9315f0f08416168356d
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-11-30 17:10:39 -07:00
GangCao
02a142abee nvme: move callback check only for the owning process
Change-Id: I05b5540fe8b1e22e8aebe23598b8cd44c8990861
Signed-off-by: GangCao <gang.cao@intel.com>
2016-11-30 17:04:48 -07:00
Ziye Yang
2e6ec373bf nvme/rdma: fix the I/O qpair creation issue
Change-Id: I357f4a83736977d319175445b4d65443f57b2c6f
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
2016-11-30 16:57:18 -07:00
Cunyin Chang
0e807fae27 nvme: Handle SIGBUS generated when a device is hot removed
Catch SIGBUS and handle it by remapping new memory into the
location where the BAR previously was.

Change-Id: Ie8d00a60a0bbe7f7ec57a5c39c0a63c5d9443206
Signed-off-by: Cunyin Chang <cunyin.chang@intel.com>
2016-11-30 14:25:04 -07:00
Cunyin Chang
56765aa39f env_dpdk: Add spdk_pci_device_[attach|detach]
These functions will attach or detach from a PCI device. Attaching
typically means mapping the BAR.

Change-Id: Iaaf59010b8a0366d32ec80bb90c1c277ada7cfe7
Signed-off-by: Cunyin Chang <cunyin.chang@intel.com>
2016-11-30 14:10:52 -07:00
Ziye Yang
88ae5a7629 nvme: move devhandle from spdk_nvme_ctrlr to nvme_pcie_ctrlr
This variable is only for nvme_pcie

Change-Id: I2bb8e65c3e6c26ef5919915b95d006bc92ebe750
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
2016-11-29 16:13:10 -07:00
GangCao
494b0c32c2 nvme: update the handling of pending admin requests
Change-Id: I035c59b6c4e4fdbbc8665c74541274e9d86ca8af
Signed-off-by: GangCao <gang.cao@intel.com>
2016-11-28 15:11:14 -07:00
Changpeng Liu
277ca8ddd0 nvme: free associated memory when deleting the IO queue pair
Change-Id: Ibc139c5fd2e0ef72b6dbd5a8416aec916759430f
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
2016-11-23 10:27:44 -07:00
GangCao
3c11422403 nvme: acquire ctrlr_lock when submitting and completing admin IOs
Change-Id: I19f395d1859f75a665a1a0cd7bef1d88ebb15631
Signed-off-by: GangCao <gang.cao@intel.com>
2016-11-22 16:56:38 -07:00
GangCao
022b8a6df4 nvme: increase number of NVME_ADMIN_TRACKERS
Change-Id: I9f5ef187b70a61eda8f0d16677dd5cbe4e4fa3c2
Signed-off-by: GangCao <gang.cao@intel.com>
2016-11-22 14:38:05 -07:00
Daniel Verkamp
7d5bcb4893 nvme: remove spdk_nvme_transport_type from API
Use the NVMe over Fabrics spec definitions for TRTYPE rather than the
internal library transport type.

Change-Id: Idead559a8f8d95274fc580d10e82033822e6eda8
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-11-22 13:42:00 -07:00
Daniel Verkamp
16ae39419d nvme: untangle internal enum_cb complexity
Change-Id: I73ddb996a652b78534ce49e5d4e43b269940ec4d
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-11-22 13:41:38 -07:00
Ziye Yang
246c39a7ee nvme: Add nvme over fabrics support
Change-Id: I6f6259e77baa5dc5861f31ec4a9034e15297d333
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
2016-11-18 13:58:44 -07:00
GangCao
af9eca84d6 nvme: rename struct spdk_nvme_controller_process
Change-Id: If8f29409b86281532eae6d87d056a2f02f6609d4
Signed-off-by: GangCao <gang.cao@intel.com>
2016-11-18 12:53:52 -07:00
GangCao
84b7670dff nvme: use spdk_zmalloc for IO qpair creation
Change-Id: I1a9b324605069b5fc1a5a7a23e87933ad3b2b3ca
Signed-off-by: GangCao <gang.cao@intel.com>
2016-11-09 15:03:45 -07:00
Daniel Verkamp
0aec36be1f nvme: return virtual address in SGL callback
Instead of the next_sge callback returning the physical address
directly, make it return the virtual address and convert to physical
address inside the NVMe library.

This is necessary for NVMe over Fabrics host support, since the RDMA
userspace API requires virtual addresses rather than physical addresses.
It is also more consistent with the normal non-SGL NVMe functions that
already take virtual addresses.

Change-Id: I79a7af64ead987535f6bf3057b2b22aef3171c5b
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-11-09 08:57:40 -07:00
Daniel Verkamp
1ffec5d53a nvme: convert transport type to an enum
Function pointers will not work for the DPDK multi-process model (they
can have different addresses in different processes), so define a
transport enum and dispatch functions that switch on the transport type
instead.

Change-Id: Ic16866786eba5e523ce533e56e7a5c92672eb2a5
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-11-07 10:42:18 -07:00
Daniel Verkamp
fcb00f3780 nvme: expand probe information to a struct
spdk_nvme_probe() will now provide a struct spdk_nvme_probe_info to the
probe and attach callbacks in place of the PCI device pointer.

This struct contains the useful information that could be retrieved from
the PCI device during probe.

The goal of this change is to allow expansion of the probe information
in the future when other transports (specifically, NVMe over Fabrics)
are added that do not necessarily use PCI addressing or device IDs.

Change-Id: I59a2a9e874e248ce5fa1d7f4b57c8056962ff3cd
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-11-02 14:15:02 -07:00
Changpeng Liu
9f860cc4b4 nvme: replace trace log with error log for error conditions
Change-Id: I634573f8d9b5ed05d850960f8c45c10d96fa3a7a
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
2016-11-02 10:31:23 -07:00
GangCao
bfc8bc87fb nvme: add the per process admin cpl queue for multi-process case
Change-Id: Ie67e3414db807160092bb10812a586b7230e0a89
Signed-off-by: GangCao <gang.cao@intel.com>
2016-11-01 16:48:21 -07:00
Daniel Verkamp
ed1e7ca2d7 env: add struct spdk_pci_id and getter function
Use the new public PCI ID structure in the NVMe library to replace the
previously private struct pci_id.

Change-Id: I267d343917f60bdae949a824bc0fe67457cbbc0d
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-11-01 09:40:36 -07:00
GangCao
075ba523bd nvme: set the qprio when constructing the qpair
Change-Id: Id5b35c023b975cc07fe721e4749bac6a0c803719
Signed-off-by: GangCao <gang.cao@intel.com>
2016-10-31 13:41:22 -07:00