From 7347136f8b725778e3e5338b657c70b2252aaec4 Mon Sep 17 00:00:00 2001 From: Ben Walker Date: Fri, 30 Jun 2017 10:01:01 -0700 Subject: [PATCH] fio_plugin: Update to 2.21, SPDK configure script Documentation update - the plugin just works with 2.21 with no changes. Change-Id: If1fb2367a00065bfb5627d0d2af9683e5c909dc1 Signed-off-by: Ben Walker Reviewed-on: https://review.gerrithub.io/367734 Tested-by: SPDK Automated Test System Reviewed-by: Jim Harris Reviewed-by: Daniel Verkamp --- examples/nvme/fio_plugin/README.md | 59 ++++++++++++++++++------------ 1 file changed, 36 insertions(+), 23 deletions(-) diff --git a/examples/nvme/fio_plugin/README.md b/examples/nvme/fio_plugin/README.md index c6447b31a..a75a3cca0 100644 --- a/examples/nvme/fio_plugin/README.md +++ b/examples/nvme/fio_plugin/README.md @@ -1,54 +1,67 @@ -Compiling ----------- +# Compiling fio First, clone the fio source repository from https://github.com/axboe/fio git clone https://github.com/axboe/fio -Then check out the fio 2.18 tag +Then check out the fio 2.21: - cd fio && git checkout fio-2.18 + cd fio && git checkout fio-2.21 -Finally, compile the code with +Finally, compile the code: - ./configure && make + make -Next, edit the CONFIG file located in the root of the SPDK repository and set CONFIG_FIO_PLUGIN -to y and FIO_SOURCE_DIR to the location of the fio repository that was just created. +# Compiling SPDK -Further, you'll need to build DPDK with -fPIC set. You can do this by modifying your -DPDK config file (i.e. config/defconfig_x86_64-native-linuxapp-gcc) to include the line +First, clone the SPDK source repository from https://github.com/spdk/spdk + + git clone https://github.com/spdk/spdk + git submodule update --init + +Then, run the SPDK configure script to enable fio (point it to the root of the fio repository): + + cd spdk + ./configure --with-fio=/path/to/fio/repo + +Finally, build SPDK: + + make + +**Note to advanced users**: These steps assume you're using the DPDK submodule. If you are using your +own version of DPDK, the fio plugin requires that DPDK be compiled with -fPIC. You can compile DPDK +with -fPIC by modifying your DPDK configuration file and adding the line: EXTRA_CFLAGS=-fPIC -At this point, build SPDK as per normal. The fio plugin will be placed in the same directory -as this README. - -Usage ------- +# Usage To use the SPDK fio plugin with fio, specify the plugin binary using LD_PRELOAD when running fio and set ioengine=spdk in the fio configuration file (see example_config.fio in the same directory as this README). - LD_PRELOAD= fio + LD_PRELOAD=/examples/nvme/fio_plugin/fio_plugin fio -To select NVMe devices, you simply pass an identifier as the filename in the format +To select NVMe devices, you pass an SPDK Transport Identifier string as the filename. These are in the +form: filename=key=value [key=value] ... ns=value -Do not have any ':' in filename, otherwise it will be spilt into several file names. Also the -NVMe namespaces start at 1, not 0! The namespace must also be specified at the end. - -Local PCIe NVMe devices will be specified using a PCI domain.bus.slot.func traddr format. For example: +Specifically, for local PCIe NVMe devices it will look like this: filename=trtype=PCIe traddr=0000.04.00.0 ns=1 -Remote devices accessed via NVMe over Fabrics will be specified based on the following example: +And remote devices accessed via NVMe over Fabrics will look like this: filename=trtype=RDMA adrfam=IPv4 traddr=192.168.100.8 trsvcid=4420 ns=1 -Currently the SPDK fio plugin is limited to thread usage model, so fio jobs must also specify thread=1 + +**Note**: The specification of the PCIe address should not use the normal ':' +and instead only use '.'. This is a limitation in fio - it splits filenames on +':'. Also, the NVMe namespaces start at 1, not 0, and the namespace must be +specified at the end of the string. + +Currently the SPDK fio plugin is limited to the thread usage model, so fio jobs must also specify thread=1 when using the SPDK fio plugin. fio also currently has a race condition on shutdown if dynamically loading the ioengine by specifying the