doc: Convert iscsi doc to markdown

Change-Id: I70da378361bd82da7b42f4ad5a5560baf70361f2
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
This commit is contained in:
Ben Walker 2017-01-04 14:53:20 -07:00
parent 4aba7b498d
commit 0c01cc5664
5 changed files with 66 additions and 141 deletions

View File

@ -768,8 +768,8 @@ INPUT = ../include/spdk \
nvme/io_submission.md \ nvme/io_submission.md \
nvmf/index.md \ nvmf/index.md \
nvmf/getting_started.md \ nvmf/getting_started.md \
iscsi/index.txt \ iscsi/index.md \
iscsi/getting_started.txt iscsi/getting_started.md
# This tag can be used to specify the character encoding of the source files # This tag can be used to specify the character encoding of the source files
# that doxygen parses. Internally doxygen uses the UTF-8 encoding. Doxygen uses # that doxygen parses. Internally doxygen uses the UTF-8 encoding. Doxygen uses

View File

@ -20,4 +20,4 @@ which avoids kernel context switches and eliminates interrupt handling overhead.
- @ref nvme - @ref nvme
- @ref nvmf - @ref nvmf
- @ref ioat - @ref ioat
- \ref iscsi - @ref iscsi

View File

@ -1,43 +1,9 @@
/*- # Getting Started Guide {#iscsi_getting_started}
* BSD LICENSE
*
* Copyright (c) Intel Corporation.
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
*
* * Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in
* the documentation and/or other materials provided with the
* distribution.
* * Neither the name of Intel Corporation nor the names of its
* contributors may be used to endorse or promote products derived
* from this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
* "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
* LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
* A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
* OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
* SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
* LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
* DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
* THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/** The Intel(R) Storage Performance Development Kit iSCSI target application is named `iscsi_tgt`.
* \page iscsi_getting_started iSCSI Target Getting Started Guide
The Intel(R) Storage Performance Development Kit iSCSI target application is named "iscsi_tgt".
This following section describes how to run iscsi from your cloned package. This following section describes how to run iscsi from your cloned package.
\section iscsi_prereqs Prerequisites # Prerequisites {#iscsi_prereqs}
This guide starts by assuming that you can already build the standard SPDK distribution on your This guide starts by assuming that you can already build the standard SPDK distribution on your
platform. The SPDK iSCSI target has been known to work on several Linux distributions, namely platform. The SPDK iSCSI target has been known to work on several Linux distributions, namely
@ -45,7 +11,7 @@ Ubuntu 14.04, 15.04, and 15.10, Fedora 21, 22, and 23, and CentOS 7.
Once built, the binary will be in `app/iscsi_tgt`. Once built, the binary will be in `app/iscsi_tgt`.
\section iscsi_config Configuring iSCSI Target # Configuring iSCSI Target {#iscsi_config}
A `iscsi_tgt` specific configuration file is used to configure the iSCSI target. A fully documented A `iscsi_tgt` specific configuration file is used to configure the iSCSI target. A fully documented
example configuration file is located at `etc/spdk/iscsi.conf.in`. example configuration file is located at `etc/spdk/iscsi.conf.in`.
@ -61,30 +27,28 @@ SCSI LUNs which can be exported via iSCSI target nodes.
The storage backends can be configured in the iscsi.conf configuration file to specify the number or The storage backends can be configured in the iscsi.conf configuration file to specify the number or
size of LUNs, block device names (for exporting in-kernel block devices), or other parameters. size of LUNs, block device names (for exporting in-kernel block devices), or other parameters.
Currently there are 3 types of stroage backends supported by iSCSI target: Currently there are 3 types of storage backends supported by the iSCSI target:
Malloc ## Malloc
======
Configuration file syntax: Configuration file syntax:
\verbatim ~~~
[Malloc] [Malloc]
NumberOfLuns 4 NumberOfLuns 4
LunSizeInMB 64 LunSizeInMB 64
\endverbatim ~~~
Other TargetNode parameters go here (TargetName, Mapping, etc.): Other TargetNode parameters go here (TargetName, Mapping, etc.):
\verbatim ~~~
[TargetNodeX] [TargetNodeX]
LUN0 Malloc0 LUN0 Malloc0
\endverbatim ~~~
This exports a malloc'd target. The disk is a RAM disk that is a chunk of memory allocated by iscsi in This exports a malloc'd target. The disk is a RAM disk that is a chunk of memory allocated by iscsi in
user space. It will use offload engine to do the copy job instead of memcpy if the system has enough DMA user space. It will use offload engine to do the copy job instead of memcpy if the system has enough DMA
channels. channels.
Block Device ## Block Device
============
AIO devices are accessed using Linux AIO. O_DIRECT is used and thus unaligned writes will be double buffered. AIO devices are accessed using Linux AIO. O_DIRECT is used and thus unaligned writes will be double buffered.
This option also bypasses the Linux page cache. This mode is probably as close to a typical kernel based This option also bypasses the Linux page cache. This mode is probably as close to a typical kernel based
@ -92,50 +56,48 @@ target as a user space target can get without using a user-space driver.
Configuration file syntax: Configuration file syntax:
\verbatim ~~~
[AIO] [AIO]
#normal file or block device #normal file or block device
AIO /dev/sdb AIO /dev/sdb
\endverbatim ~~~
Other TargetNode parameters go here (TargetName, Mapping, etc.): Other TargetNode parameters go here (TargetName, Mapping, etc.):
\verbatim ~~~
[TargetNodeX] [TargetNodeX]
LUN0 AIO0 LUN0 AIO0
\endverbatim ~~~
Ceph RBD ## Ceph RBD
========
Ceph RBD devices are accessed via librbd and librados libraries to access the RADOS block device Ceph RBD devices are accessed via librbd and librados libraries to access the RADOS block device
exported by Ceph. exported by Ceph.
Configuration file syntax: Configuration file syntax:
\verbatim ~~~
[Ceph] [Ceph]
# The format of provided rbd info should be: Ceph rbd_pool_name rbd_name size. # The format of provided rbd info should be: Ceph rbd_pool_name rbd_name size.
# In the following example, rbd is the name of rbd_pool; foo is the name of # In the following example, rbd is the name of rbd_pool; foo is the name of
# rbd device exported by Ceph; value 512 represents the configured block size # rbd device exported by Ceph; value 512 represents the configured block size
# for this rbd, the block size should be a multiple of 512. # for this rbd, the block size should be a multiple of 512.
Ceph rbd foo 512 Ceph rbd foo 512
\endverbatim ~~~
Other TargetNode parameters go here (TargetName, Mapping, etc.): Other TargetNode parameters go here (TargetName, Mapping, etc.):
\verbatim ~~~
[TargetNodeX] [TargetNodeX]
LUN0 Ceph0 LUN0 Ceph0
\endverbatim ~~~
NVMe ## NVMe
====
The SPDK NVMe driver by default binds to all NVMe controllers which are not bound to the kernel-mode The SPDK NVMe driver by default binds to all NVMe controllers which are not bound to the kernel-mode
nvme driver. Users can choose to bind to fewer controllers by setting the NumControllers parameter. nvme driver. Users can choose to bind to fewer controllers by setting the NumControllers parameter.
Then the NVMe backend controls NVMe controller(s) directly from userspace and completely bypasses Then the NVMe backend controls NVMe controller(s) directly from userspace and completely bypasses
the kernel to avoid interrupts and context switching. the kernel to avoid interrupts and context switching.
\verbatim ~~~
[Nvme] [Nvme]
# NVMe Device Whitelist # NVMe Device Whitelist
# Users may specify which NVMe devices to claim by their PCI # Users may specify which NVMe devices to claim by their PCI
@ -173,47 +135,46 @@ the kernel to avoid interrupts and context switching.
# Note: NVMe namespace IDs always start at 1, not 0 - and most # Note: NVMe namespace IDs always start at 1, not 0 - and most
# controllers have only 1 namespace. # controllers have only 1 namespace.
LUN0 Nvme0n1p0 LUN0 Nvme0n1p0
\endverbatim ~~~
You should make a copy of the example configuration file, modify it to suit your environment, and You should make a copy of the example configuration file, modify it to suit your environment, and
then run the iscsi_tgt application and pass it the configuration file using the -c option. Right now, then run the iscsi_tgt application and pass it the configuration file using the -c option. Right now,
the target requires elevated privileges (root) to run. the target requires elevated privileges (root) to run.
\verbatim ~~~
app/iscsi_tgt/iscsi_tgt -c /path/to/iscsi.conf app/iscsi_tgt/iscsi_tgt -c /path/to/iscsi.conf
\endverbatim ~~~
\section iscsi_initiator_config Configuring iSCSI Initiator # Configuring iSCSI Initiator {#iscsi_initiator}
The Linux initiator is open-iscsi. The Linux initiator is open-iscsi.
Installing open-iscsi package Installing open-iscsi package
Fedora: Fedora:
\verbatim ~~~
yum install -y iscsi-initiator-utils yum install -y iscsi-initiator-utils
\endverbatim ~~~
Ubuntu: Ubuntu:
\verbatim ~~~
apt-get install -y open-iscsi apt-get install -y open-iscsi
\endverbatim ~~~
Setup ## Setup
=====
Edit /etc/iscsi/iscsid.conf Edit /etc/iscsi/iscsid.conf
\verbatim ~~~
node.session.cmds_max = 4096 node.session.cmds_max = 4096
node.session.queue_depth = 128 node.session.queue_depth = 128
\endverbatim ~~~
iscsid must be restarted or receive SIGHUP for changes to take effect. To send SIGHUP, run: iscsid must be restarted or receive SIGHUP for changes to take effect. To send SIGHUP, run:
\verbatim ~~~
killall -HUP iscsid killall -HUP iscsid
\endverbatim ~~~
Recommended changes to /etc/sysctl.conf Recommended changes to /etc/sysctl.conf
\verbatim ~~~
net.ipv4.tcp_timestamps = 1 net.ipv4.tcp_timestamps = 1
net.ipv4.tcp_sack = 0 net.ipv4.tcp_sack = 0
@ -226,63 +187,64 @@ net.core.rmem_max = 524287
net.core.wmem_max = 524287 net.core.wmem_max = 524287
net.core.optmem_max = 524287 net.core.optmem_max = 524287
net.core.netdev_max_backlog = 300000 net.core.netdev_max_backlog = 300000
\endverbatim ~~~
Discovery ### Discovery
Assume target is at 192.168.1.5 Assume target is at 192.168.1.5
\verbatim ~~~
iscsiadm -m discovery -t sendtargets -p 192.168.1.5 iscsiadm -m discovery -t sendtargets -p 192.168.1.5
\endverbatim ~~~
Connect to target ### Connect to target
\verbatim ~~~
iscsiadm -m node --login iscsiadm -m node --login
\endverbatim ~~~
At this point the iSCSI target should show up as SCSI disks. Check dmesg to see what At this point the iSCSI target should show up as SCSI disks. Check dmesg to see what
they came up as. they came up as.
Disconnect from target ### Disconnect from target
\verbatim ~~~
iscsiadm -m node --logout iscsiadm -m node --logout
\endverbatim ~~~
Deleting target node cache ### Deleting target node cache
\verbatim ~~~
iscsiadm -m node -o delete iscsiadm -m node -o delete
\endverbatim ~~~
This will cause the initiator to forget all previously discovered iSCSI target nodes. This will cause the initiator to forget all previously discovered iSCSI target nodes.
Finding /dev/sdX nodes for iSCSI LUNs ### Finding /dev/sdX nodes for iSCSI LUNs
\verbatim ~~~
iscsiadm -m session -P 3 | grep "Attached scsi disk" | awk '{print $4}' iscsiadm -m session -P 3 | grep "Attached scsi disk" | awk '{print $4}'
\endverbatim ~~~
This will show the /dev node name for each SCSI LUN in all logged in iSCSI sessions. This will show the /dev node name for each SCSI LUN in all logged in iSCSI sessions.
### Tuning
After the targets are connected, they can be tuned. For example if /dev/sdc is After the targets are connected, they can be tuned. For example if /dev/sdc is
an iSCSI disk then the following can be done: an iSCSI disk then the following can be done:
Set noop to scheduler Set noop to scheduler
\verbatim ~~~
echo noop > /sys/block/sdc/queue/scheduler echo noop > /sys/block/sdc/queue/scheduler
\endverbatim ~~~
Disable merging/coalescing (can be useful for precise workload measurements) Disable merging/coalescing (can be useful for precise workload measurements)
\verbatim ~~~
echo "2" > /sys/block/sdc/queue/nomerges echo "2" > /sys/block/sdc/queue/nomerges
\endverbatim ~~~
Increase requests for block queue Increase requests for block queue
\verbatim ~~~
echo "1024" > /sys/block/sdc/queue/nr_requests echo "1024" > /sys/block/sdc/queue/nr_requests
\endverbatim ~~~
*/

3
doc/iscsi/index.md Normal file
View File

@ -0,0 +1,3 @@
# iSCSI Target {#iscsi}
- @ref iscsi_getting_started

View File

@ -1,40 +0,0 @@
/*-
* BSD LICENSE
*
* Copyright (c) Intel Corporation.
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
*
* * Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in
* the documentation and/or other materials provided with the
* distribution.
* * Neither the name of Intel Corporation nor the names of its
* contributors may be used to endorse or promote products derived
* from this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
* "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
* LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
* A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
* OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
* SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
* LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
* DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
* THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/*!
\page iscsi iSCSI Target
- \ref iscsi_getting_started
*/