bdev: add a getting started guide

For now, this contains the documentation for configuring
block devices in configuration files.  This file can then
be used a common reference for other getting started guides -
iscsi, vhost and nvmf targets.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ie6f6c0b3f36dd3fdf418b904462c81a1696b9694

doc/Doxyfile

@@ -762,17 +762,19 @@ INPUT                  = ../include/spdk \
                          index.md \
                          directory_structure.md \
                          porting.md \
+                         bdev/index.md \
+                         bdev/getting_started.md \
                          event/index.md \
                          ioat/index.md \
+                         iscsi/index.md \
+                         iscsi/getting_started.md \
                          nvme/index.md \
                          nvme/async_completion.md \
                          nvme/fabrics.md \
                          nvme/initialization.md \
                          nvme/io_submission.md \
                          nvmf/index.md \
-                         nvmf/getting_started.md \
+                         nvmf/getting_started.md
-                         iscsi/index.md \
-                         iscsi/getting_started.md
 
 # This tag can be used to specify the character encoding of the source files
 # that doxygen parses. Internally doxygen uses the UTF-8 encoding. Doxygen uses

doc/bdev/getting_started.md

@@ -0,0 +1,89 @@
+# SPDK bdev Getting Started Guide {#bdev_getting_started}
+
+Block storage in SPDK applications is provided by the SPDK bdev layer.  SPDK bdev consists of:
+
+* a driver module API for implementing bdev drivers
+* an application API for enumerating and claiming SPDK block devices and performance operations
+(read, write, unmap, etc.) on those devices
+* bdev drivers for NVMe, malloc (ramdisk), Linux AIO and Ceph RBD
+* configuration via SPDK configuration files or JSON RPC
+
+# Configuring block devices {#bdev_config}
+
+SPDK block devices are typically configured via an SPDK configuration file.  These block devices
+can then be associated with higher level abstractions such as iSCSI target nodes, NVMe-oF namespaces
+or vhost-scsi controllers.  This section will describe how to configure block devices for the
+SPDK bdev drivers included with SPDK.
+
+## NVMe
+
+The SPDK nvme bdev driver provides SPDK block layer access to NVMe SSDs via the SPDK userspace
+NVMe driver.  The nvme bdev driver binds only to devices explicitly specified.  These devices
+can be either locally attached SSDs or remote NVMe subsystems via NVMe-oF.
+
+~~~
+[Nvme]
+  # NVMe Device Whitelist
+  # Users may specify which NVMe devices to claim by their transport id.
+  # See spdk_nvme_transport_id_parse() in spdk/nvme.h for the correct format.
+  # The devices will be assigned names in the format <YourName>nY, where YourName is the
+  # name specified at the end of the TransportId line and Y is the namespace id, which starts at 1.
+  TransportID "trtype:PCIe traddr:0000:00:00.0" Nvme0
+  TransportID "trtype:RDMA subnqn:nqn.2016-06.io.spdk:cnode1 traddr:192.168.100.1 trsvcid:4420" Nvme1
+~~~
+
+This exports block devices for all namespaces attached to the two controllers.  Block devices
+for namespaces attached to the first controller will be in the format Nvme0nY, where Y is
+the namespace ID.  Most NVMe SSDs have a single namespace with ID=1.  Block devices attached to
+the second controller will be in the format Nvme1nY.
+
+## Malloc
+
+The SPDK malloc bdev driver allocates a buffer of memory in userspace as the target for block I/O
+operations.  This effectively serves as a userspace ramdisk target.
+
+Configuration file syntax:
+~~~
+[Malloc]
+  NumberOfLuns 4
+  LunSizeInMB  64
+~~~
+
+This exports 4 malloc block devices, named Malloc0 through Malloc3.  Each malloc block device will
+be 64MB in size.
+
+## Linux AIO
+
+The SPDK aio bdev driver provides SPDK block layer access to Linux kernel block devices via Linux AIO.
+Note that O_DIRECT is used and thus bypasses the Linux page cache. This mode is probably as close to
+a typical kernel based target as a user space target can get without using a user-space driver.
+
+Configuration file syntax:
+
+~~~
+[AIO]
+  # normal file or block device
+  AIO /dev/sdb
+  AIO /dev/sdc
+~~~
+
+This exports 2 aio block devices, named AIO0 and AIO1.
+
+## Ceph RBD
+
+The SPDK rbd bdev driver provides SPDK block layer access to Ceph RADOS block devices (RBD).  Ceph
+RBD devices are accessed via librbd and librados libraries to access the RADOS block device
+exported by Ceph.
+
+Configuration file syntax:
+
+~~~
+[Ceph]
+  # The format of provided rbd info should be: Ceph rbd_pool_name rbd_name size.
+  # In the following example, rbd is the name of rbd_pool; foo is the name of
+  # rbd device exported by Ceph; value 512 represents the configured block size
+  # for this rbd, the block size should be a multiple of 512.
+  Ceph rbd foo 512
+~~~
+
+This exports 1 rbd block device, named Ceph0.

doc/bdev/index.md

@@ -0,0 +1,3 @@
+# Block Device Abstraction Layer {#bdev}
+
+- @ref bdev_getting_started

doc/index.md

@@ -24,3 +24,4 @@ which avoids kernel context switches and eliminates interrupt handling overhead.
 - @ref nvmf
 - @ref ioat
 - @ref iscsi
+- @ref bdev

doc/iscsi/getting_started.md

@@ -21,107 +21,20 @@ TCP ports to use as iSCSI portals; general iSCSI parameters; initiator names and
 access to iSCSI target nodes; number and types of storage backends to export over iSCSI LUNs; iSCSI
 target node mappings between portal groups, initiator groups, and LUNs.
 
-The SPDK iSCSI target supports several different types of storage backends. These backends will create
+Each LUN in an iSCSI target node is associated with an SPDK block device.  See @ref bdev_getting_started
-SCSI LUNs which can be exported via iSCSI target nodes.
+for details on configuring SPDK block devices.  The block device to LUN mappings are specified in the
+configuration file as:
 
-The storage backends can be configured in the iscsi.conf configuration file to specify the number or
+~~~~
-size of LUNs, block device names (for exporting in-kernel block devices), or other parameters.
-
-Currently there are 3 types of storage backends supported by the iSCSI target:
-
-## Malloc
-
-Configuration file syntax:
-~~~
-[Malloc]
-  NumberOfLuns 4
-  LunSizeInMB  64
-~~~
-
-Other TargetNode parameters go here (TargetName, Mapping, etc.):
-~~~
 [TargetNodeX]
   LUN0 Malloc0
-~~~
+  LUN1 Nvme0n1
+~~~~
 
 This exports a malloc'd target. The disk is a RAM disk that is a chunk of memory allocated by iscsi in
 user space. It will use offload engine to do the copy job instead of memcpy if the system has enough DMA
 channels.
 
-## Block Device
-
-AIO devices are accessed using Linux AIO. O_DIRECT is used and thus unaligned writes will be double buffered.
-This option also bypasses the Linux page cache. This mode is probably as close to a typical kernel based
-target as a user space target can get without using a user-space driver.
-
-Configuration file syntax:
-
-~~~
-[AIO]
-  #normal file or block device
-  AIO /dev/sdb
-~~~
-
-Other TargetNode parameters go here (TargetName, Mapping, etc.):
-~~~
-[TargetNodeX]
-  LUN0 AIO0
-~~~
-
-## Ceph RBD
-
-Ceph RBD devices are accessed via librbd and librados libraries to access the RADOS block device
-exported by Ceph.
-
-Configuration file syntax:
-
-~~~
-[Ceph]
-  # The format of provided rbd info should be: Ceph rbd_pool_name rbd_name size.
-  # In the following example, rbd is the name of rbd_pool; foo is the name of
-  # rbd device exported by Ceph; value 512 represents the configured block size
-  # for this rbd, the block size should be a multiple of 512.
-  Ceph rbd foo 512
-~~~
-
-Other TargetNode parameters go here (TargetName, Mapping, etc.):
-~~~
-[TargetNodeX]
-  LUN0 Ceph0
-~~~
-
-## NVMe
-
-The SPDK NVMe driver by default binds to all NVMe controllers which are not bound to the kernel-mode
-nvme driver. Users can choose to bind to fewer controllers by setting the NumControllers parameter.
-Then the NVMe backend controls NVMe controller(s) directly from userspace and completely bypasses
-the kernel to avoid interrupts and context switching.
-
-~~~
-[Nvme]
-  # NVMe Device Whitelist
-  # Users may specify which NVMe devices to claim by their transport id.
-  # See spdk_nvme_transport_id_parse() in spdk/nvme.h for the correct format.
-  # The second argument is the assigned name, which can be referenced from
-  # other sections in the configuration file. For NVMe devices, a namespace
-  # is automatically appended to each name in the format <YourName>nY, where
-  # Y is the NSID (starts at 1).
-  TransportID "trtype:PCIe traddr:0000:00:00.0" Nvme0
-  TransportID "trtype:PCIe traddr:0000:01:00.0" Nvme1
-
-  # The number of attempts per I/O when an I/O fails. Do not include
-  # this key to get the default behavior.
-  NvmeRetryCount 4
-
-[TargetNodeX]
-  # other TargetNode parameters go here (TargetName, Mapping, etc.)
-  # nvme with the following format: NvmeXnY, where X = the controller ID
-  # and Y = the namespace ID
-  # Note: NVMe namespace IDs always start at 1, not 0 - and most
-  #  controllers have only 1 namespace.
-  LUN0 Nvme0n1
-~~~
-
 You should make a copy of the example configuration file, modify it to suit your environment, and
 then run the iscsi_tgt application and pass it the configuration file using the -c option. Right now,
 the target requires elevated privileges (root) to run.