When overriding with igb_uio you used to have to manually probe uio
before calling setup.sh. I feel this is a common enough use case that we
should check it in setup.sh directly.
Change-Id: I8a31f90bab6960eade468816ffe960820f83d2fe
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/818
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Correct shellcheck rule SC2089: Quotes/backslashes will be treated literally. Use an array.
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I40d4eef334cd030294669f8e8494f9e7a563d495
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476466
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
It fixes change 474989.
"for blkname in $blknames; do"
was changed to
"for blkname in "${blknames[@]}"; do"
if blknames=" vda" then first blkname="" and
linux_bind_driver is called for vda although it has active mountpoint
Change-Id: I16d4379db274faa5703230072cfcf8d18a8ef5e6
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476061
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Correct shellcheck rule SC2128: Expanding an array without an index only gives the first element.
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I0e7c335af678114dc78dfb12a02369a69158e435
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/474989
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Seth Howell <seth.howell@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Correct shellcheck rule SC2166: Prefer [ p ] && [ q ]
as [ p -a q ] is not well defined.
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I3fb2fb0e790bbdd296b64673c301324632d7c37b
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/472173
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Correct shellcheck rule SC2013: To read lines rather than
words, pipe/redirect to a 'while read' loop.
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I01c0dcf045fc131ce0cfa672e5ede0b881a47406
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/471433
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Correct shellcheck rule SC2145: Argument mixes string and
array. Use * or separate argument.
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I45102ead2e97cc1d6b11c21f269e58a235055c35
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/470926
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Variable was used as an array but is now assigned a string.
Change-Id: I79e59d80be96657a9c2fc66f9d205458f9db67b2
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/468366
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
In the event that our fuzz test passes, we should manually remove the
trace files from the /dev/shm directory. This will prevent us from
failing to allocate /dev/shm files when a machine consistently passes
the tests and doesn't reboot.
Change-Id: I2ccde6aba7d61fda3a28ed24e009c6f8f81c20b4
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/465996
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
SPDK/DPDK startup time can be very important - especially
in secondary processes. If DPDK cannot figure out the
TSC via cpuid or rdmsr, it will instead use a 100ms sleep
to calculate the TSC. On Fedora kernels, the msr module
is linked into the kernel, enabling rdmsr and bypassing
this 100ms sleep. But on Ubuntu kernels, the user must
explicitly load the msr module, otherwise it will incur
the 100ms sleep.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I779f023bb1681570da81c99cefa66debf3e1adc6
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/465711
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Changing according to styling check done by ShellCheck.
Removing from check_format.sh exclusion list:
SC2236 - Use -n instead of ! -z
SC2070 - -n doesn't work with unquoted arguments. Quote or use [[ ]]
Change-Id: Ia9d645b9d0ce31b67c4de682395cf36f4ddc8d1f
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/463180
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
On multiple development systems (including mine), this results
in new error messages when running scripts/setup.sh. This has
the potential of causing lots of questions when users upgrade
to SPDK 19.07.
I understand the desire for the error messages, but I think we have
to come up with a smarter check than just against "unlimited".
This reverts commit 0abee610f0.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I2854af556b67a9a4e5f686c72c4407962808f964
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/463357
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Currently we do not switch driver for NVMe that is mounted.
The problem is that we take into consideration only the
first namespace, but any other namespace can be still mounted.
In such case we should not switch driver.
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: Idd13edccd0929574f2914a71e841e67871dd2e9f
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457762
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Scripts were using a mix of two approaches, lets unify that so
just dollar-parenthesis syntax is used.
Also update two scripts from spdk/test which we previously missed.
Change-Id: I429f9bc158076462b419fae597f716c329f9b7aa
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/460344
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
To bind VMD device it need to be specified
in PCI_WHITELIST.
Change-Id: If19c47941278e4df9c6157024bba8b76c90695c6
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Signed-off-by: Orden Smith <orden.e.smith@intel.com>
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455635
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
On non NUMA architectures the numa_node sysfs entry doesn't exist.
Check if it exists before reading to prevent an error causing 'status'
to fail.
Reviewed-by: Qingmin Liu <qingmin.liu@broadcom.com>
Tested-by: Jonathan Richardson <jonathan.richardson@broadcom.com>
Reviewed-by: Jonathan Richardson <jonathan.richardson@broadcom.com>
Reviewed-by: Scott Branden <scott.branden@broadcom.com>
Signed-off-by: Jonathan Richardson <jonathan.richardson@broadcom.com>
Change-Id: Iacf5b062a4f5db5e049d2cfcff7c991d4ac5d3c3
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/448576
Reviewed-by: Scott Branden <sbranden@gmail.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Qingmin Liu <calmarrow@gmail.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This utility used to be called "bdevtest" so that's
what opts.name was set to for spdk_app_opts_init().
Change it to the new name "bdevio".
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Id8e6614c6fe82651952f46ec56de14ef0a8961f0
Reviewed-on: https://review.gerrithub.io/c/445800
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
This function should be changed as well but was overlooked.
Fixing this here.
Change-Id: I2f7c8e475d89cfe177441e1843d3713397f6fe7b
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/444079
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
On some platforms, especially AWS, uio_pci_generic is not readily
available, and we don't have access to an IOMMU, so using the DPDK
igb_uio driver is a valid workaround.
Change-Id: Ic96776f925d9bbbcab625f5adb7642ca6bd4033d
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/444122
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
The current state for the setup script is that the vfio kernel driver
will be preferred against the uio_pci_generic driver only if an IOMMU is
present in the system. This is checked by looking for any IOMMU groups
in sysfs.
In case of vfio no-IOMMU driver, there are no IOMMU groups when loading
the vfio modules. The IOMMU groups are created when the PCI devices get
bound to the vfio-pci driver. Thus, even though the vfio driver is
loaded, the setup script will prefer to use the uio_pci_generic kernel
driver. This patch changes this behavior.
In order to support vfio no-IOMMU mode, the setup script will be
checking if the vfio module is loaded with parameter
"enable_unsafe_noiommu_mode". In other case, it will be falling back to
the uio_pci_generic driver.
Change-Id: I1e8317bc4e3d6af4ba8a9e0c51175c9f4190f47b
Signed-off-by: Nikos Dragazis <ndragazis@arrikto.com>
Reviewed-on: https://review.gerrithub.io/c/441062
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
iter_pci_dev_id abd iter_pci_dev_id functions should
not return BDF for devices that are not ment to be used
in tests.
Note that not all tests are ready for this change as they
discover functions on its own. Lets this changed in
separate patch.
Change-Id: I45a59ec121aa81e9f981acae7ec0379ff68e520a
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/443767
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Add PCI blacklist so we can skip only some devices.
Change-Id: I8600307dd53f32acb4dfeb3f57845e0b9d29fdb9
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/442977
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Bash interprets everything after command as additional
function arguments. To not confuse user just remove this part
and replace by '!'.
Change-Id: I44228003a1f96324271e726df4f5033f3258523c
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/442976
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
The original mechanism to identify files to cleanup
relied on glob matching from the output of 'echo'
piped to a grep. This yields a case where some
objects can appear and picked up as matching because
other items on the same line matched the grep
string. Changing this to use 'ls -1' which will
restrict the grep string matching to individual lines
thereby only picking up the entities the script
intended to do.
Change-Id: I020c80236fa68bcabeca0299fe7a27f3320de97b
Signed-off-by: Lance Hartmann <lance.hartmann@oracle.com>
Reviewed-on: https://review.gerrithub.io/437380
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Grepping `mount` output for /dev/sdX does not guarantee
the device has no mountpoints. If /dev/sdX happens to
have an LVM built on top, then it could be
/dev/mapper/something that appears in `mount`.
Fix this by checking mountpoints of /dev/sdX and all its
children as reported by `lsblk`.
Change-Id: Id2fa6939645584d009bc87c7341a97f6948ebde9
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/434209
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
The PCI devices used for SPDK are bound with vfio-pci or
uio_pci_generic kernel drivers. In setup.sh, if the path /sys/kernel
/iommu_groups is not empty, vfio-pci kernel driver is the only choice;
otherwise uio_pci_generic is selected.
In system, IOMMU can be enabled but set to pass through. It means
IOMMU will not affect the DMA transmission although IOMMU groups has
been configured. In this case both two kernel drivers are workable. The
script cannot deal with the case now.
The new option DRIVER_OVERRIDE is introduced in the patch and allow
user selects the kernel driver for PCI devices. With the patch the above
case can be handled correctly.
Change-Id: I540d8750bf837ce67b8bc8b516a1a3acb72c502c
Signed-off-by: tone.zhang <tone.zhang@arm.com>
Reviewed-on: https://review.gerrithub.io/427297
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
A recent patch changed bdevperf app name from 'bdevtest' to 'bdevperf'.
Change-Id: I6f89cb20560b819f8c451b34ee5afbf5fca68939
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/430343
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
SPDK deletes those files automatically for single-process
applications and the multi-process' ones can now be freed
with `./setup.sh cleanup`.
The script clears so called DPDK runtime directories, which
are defined as follows (for DPDK 18.05+):
* If DPDK is running as root, /var/run/dpdk/<per-pid-prefix>/
* If DPDK is not running as root:
* If $XDG_RUNTIME_DIR is set, ${XDG_RUNTIME_DIR}/dpdk/<per-pid-prefix>/
* Otherwise, /tmp/dpdk/<per-pid-prefix>/
Since DPDK 18.08, even the shared config file is placed in
the runtime dir, making it the only dir we have to clear.
Change-Id: I13f58a98f6a07e609b888b4e816dd45eac10f49c
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/422485
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
This was most likely overlooked in the original implementation.
Change-Id: Ib3efbcf4a3abcbd6ad6bf9ced7280879b73f86bf
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/422081
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
From now on remove any SPDK artifacts just use:
./scripts/setup.sh cleanup
Fixes#302
Change-Id: I5ebe522fecfcb8a867a96ab10bacda6083c8c224
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/419575
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
When running "setup.sh config" (or setup.sh without
specify a mode - which defaults to config), do not
unload the contigmem driver and then reload it. Over
time, memory can get fragmented and contigmem will fail,
crashing the system.
But still check if the requested HUGEMEM matches what was
previously specified. If a different amount was requested,
unload contigmem as before and then reload with the new
amount.
This patch brings FreeBSD behavior in line with Linux. On
Linux, we do not release all of the hugepages and then
reallocate them.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I18cb6fd4bdc1e1da0382423455ebce58c13bee7a
Reviewed-on: https://review.gerrithub.io/412238
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
The get_nvme_name_from_bdf function uses the variable name $bdf, which
is also used as a loop iterator in its caller; since variables are
global by default in bash, this overwrites the original $bdf and may
cause unexpected behavior like skipping setup of some NVMe devices.
The long-term fix should be to declare all variables local in all
functions, but a minimal, targeted fix is to just rename the variable in
the inner function.
Change-Id: I53a877606a585415e8af525c6162ed7b0f67ba45
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/403370
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
setup.sh uses lsmod to detect if a module is present but this does
not work when modules are built-in. We add a second check (on
/sys/module/<module>) and place the check in a function.
We also change the sense of driver_loaded so it is more sane and also
allows us to return different positive values depending on if the
driver is a module or built-in.
Change-Id: Iccc4dca212a6f04fb2ac9bd4768935f8b2bb240a
Signed-off-by: Stephen Bates <sbates@raithlin.com>
Reviewed-on: https://review.gerrithub.io/402178
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Briefly tested with SPDK target via bdevio
through both vhost-user and QEMU's virtio-pci.
vhost-blk driver is not mentioned in any doc yet.
A virtio-blk bdev can be created as following:
`construct_virtio_user_blk_bdev /tmp/virtio.0 MyVirtioBlk0 --vq-count 16`
`construct_virtio_pci_blk_bdev 0000:01:00.0 MyVirtioBlk1`
or
```
[VirtioUser0]
Path /tmp/vhost.0
Type Blk
Queues 16
[VirtioPci]
Enable Yes
```
If `Type` field is not present, the VirtioUser
section will be parsed as a SCSI one, preserving
backward compatibility.
Change-Id: I3eae9f3b90570fbb0177df4951e5eed86fe07c66
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/393056
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
It's apparently unavailable on some systems.
We used it to grep all Virtio-PCI block devices
before matching them against our desired PCI bdf
in get_virtio_names_from_bdf(). If we were unbinding
a Virtio-PCI device and lsblk failed, the script
assumed there were no block devices and hence no
active mountpoints for that Virtio device. The initial
grep is not necessary and, since it's causing us
troubles, is now being removed.
Change-Id: I5a621391dc4772b2bd67eaa832e318d9a0d25948
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/401780
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Instead, PCI_WHITELIST="none" can now be used.
SKIP_PCI continues to work, but is undocumented.
Change-Id: Id4a7959716f6bca92be0da761258c84f32aca8b5
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/395946
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
It is now possible to reset only
particular PCI devices.
Change-Id: Ic45be5e7e16cf4736c742b14fc0d709788a3b3f1
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/395945
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Change-Id: I0f6e1d8df9480538e2dc39cf658396d4ecd14daa
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/393724
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
NVME_WHITELIST param remains to work, but is now
undocumented. PCI_WHITELIST will apply to all PCI
devices - NVMe, IO/AT and Virtio.
Change-Id: I782f48bea68079c63e4a2794e4a4eb8f9a7226c9
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/395944
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
We used to crash when `mount` returned
multiple hugetlb mountpoints.
Change-Id: I75717c2c394204bdfec730edabb9ff6386047742
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/395926
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Patch da7e3bb8 [1] disabled chown for vfio
groups and hugepage mountpoints. Chown code
was never being executed.
[1] da7e3bb8 ("setup.sh: replace username param with TARGET_USER env
variable")
Change-Id: Ie5c9ac42fca5b60c72ced501d4065842a5bef9cd
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/395925
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Make it configurable the same way as other options.
setup.sh will configure hugepages and vfio for the
current user by default. To specify other user,
set TARGET_USER variable.
Change-Id: Ib98178a70a007bec761f7465689c2150e8844ffb
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/394806
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
In some cases we may not want to assign all PCIe NVMe controllers in a
system to SPDK. Add a new input to the setup.sh script called
NVME_WHITELIST which whitelists (via PCIe slot ID) the NVMe
controllers you wish to add to SPDK.
If the NVME_WHITELIST input argument is not defined then all PCIe NVMe
controllers will be added. The values in the whitelist whould be
white-space seperated and the entire list should be enclosed in double
quotes ("").
To blacklist all PCIe NVMe devices use a non-valid PCIe slot ID
(e.g. the string "none" would work very well).
Examples:
NVME_WHITELIST="0000:02:00.0" ./setup.sh
NVME_WHITELIST="0000:08:00.0 0000:06:00.1" ./setup.sh
NVME_WHITELIST="none" ./setup.sh
Change-Id: If6ebb04307180cbac11fc41cd9a70749640bc598
Signed-off-by: Stephen Bates <sbates@raithlin.com>
Reviewed-on: https://review.gerrithub.io/394303
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Very old kernels (e.g. 3.13 from Ubuntu 14.04) don't have the nvme
directory in /sys/bus/pci/devices/$bdf/, so check for its existence
before trying to list it.
Change-Id: I1f1ca04c71de6359de2b924fb9437ad03c56523b
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/393725
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This makes us iterate through all devices
on a controller and check if any of those
is mounted. Previously we only checked the
very first device, which is obviously
insufficient.
Change-Id: I776846b931480d1e0a059137950c21f057ecb710
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/393749
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Add a new file, scripts/common.sh, that can be shared between not only
autotest scripts, but also scripts/setup.sh, scripts/gen_nvme.sh, etc.,
and move the PCI iteration functions there.
The iterators are also expanded to work identically for both dev_id and
class_code on Linux and FreeBSD.
Change-Id: I98423cd06242e78535f5da4fce82166812ea96a8
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/393416
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
When looking up nvme devices by name in older linux kernel versions, setup.sh
was throwing errors while trying to follow a symlink on
/sys/block/. This patch adds a fallback to the proper location of this
symlink in those versions of the kernel.
Change-Id: I13b5c1ce6f7505251789b64c740aba04d67f39a9
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/391019
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
User may only want to configure hugepage and no configure
on the PCI devices. Add the SKIP_PCI parameter to offer
this option.
If user only wants to configure hugepage, can run:
SKIP_PCI=1 ./scripts/setup.sh
The default behavior is that PCI devices will be configured.
Change-Id: I8302efd5a09f2885e00fa8ed00d398219c11dde7
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.gerrithub.io/389473
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Introduce several new functions configure_linux_pci, reset_linux_pci and
configure_freebsd_pci for the PCI devices relatd operations.
Change-Id: Ifdaddd314ba01fd73ff700d231a6aeae5ec7c480
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.gerrithub.io/387265
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
The setup.sh can now be invoked as following:
```
HUGENODE=0 NRHUGE=5 ./scripts/setup.sh
HUGENODE=1 NRHUGE=3 ./scripts/setup.sh
```
This will allocate 5 hugepages on node0,
and 3 hugepages on node1. If HUGENODE
param is not specified, the setup.sh will
operate just like it used to.
Change-Id: Ib198f5f32203abcc646af3c8d823f2b3e9bce362
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/384086
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
The previous defines have been
removed by patch 80b44ea.
Fixes: 80b44ea ("rte_virtio: remove unused PCI ID #defines")
Change-Id: I0e90e7f91a5e91efca3e5562b94cb1f6a36e7428
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/382935
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Use lsblk to detect if an NVMe namespace or any partition on the
namespace has an active mountpoint. If it does, do not unbind
the NVMe device associated with that NVMe namespace.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I1ab7540d640baa201efac49bc9515fd861dd8f8c
Reviewed-on: https://review.gerrithub.io/382479
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
If the user did not have the ioatdma driver loaded for example,
when running "setup.sh reset", we should not load ioatdma so that
we can bind to it. Instead just unbind the devices from the
uio/vfio driver.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I953941eb4918105b97ad78987b47b33f4372ae01
Reviewed-on: https://review.gerrithub.io/382474
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
This allows to use same command
for allocating hugepages on
systems with different hugepage
sizes.
The old NRHUGE variable can be
still set on Linux machines,
but is currently undocumented.
Change-Id: I1fba315d95e9aae8b6a8c9c445deb447fecc65dc
Signed-off-by: Hailiang Wang <hailiangx.e.wang@intel.com>
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/369546
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
When a DPDK app closes, it leaves behing configuration files that can
cause latent failures in the build pool machines if they are not
removed.
Change-Id: I80caa1b595e6586a109c013b92dfcf306d7a0150
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/379874
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Supports both PCI mode (for usage in guest VMs) and
vhost-user mode (for usage in host processes). The rte_virtio
subdirectory contains a lot of code lifted from the DPDK
virtio-net driver. Most of the PCI and vhost-user code is
reused almost exactly as-is, but the virtio code is drastically
rewritten as the DPDK code was very network specific.
Has been lightly tested with both the bdevio and bdevperf
applications in both PCI and vhost-user modes.
Still quite a bit of work needed - a list of todo
items is included in a README in the module's directory.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I85989d3de9ea89a87b719ececdb6d2ac16b77f53
Reviewed-on: https://review.gerrithub.io/374519
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Another atempt to fix setup.sh On Ubuntu 16.04 that use GNU Awk 4.1.3.
Eg when subsystem vendor name and subsystem name are empty:
$ lspci -mm -n -D
...
0000:00:05.0 "0880" "8086" "0e28" -r04 "8086" "3582"
0000:00:05.1 "0880" "8086" "0e29" -r04 "" ""
0000:00:05.2 "0880" "8086" "0e2a" -r04 "8086" "3582"
...
tr -d '"' part is brekaing whole awk script. So lets try to escape
vendor and device ID to comapre them as whole string.
Change-Id: Ic41e82c41347c6e0a8246b445f86eddf56780089
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/374462
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Device IDs starting with 0e get treated as 0 due to numerical
interpretation (0 * 10^x = 0). So use ~ to do a string regexp
comparison instead.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I4b5558c0127b0c4f021daf8151bf4d3f514e44da
Reviewed-on: https://review.gerrithub.io/374507
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
1) Create separate functions for checking the PCI class code
vs. checking the PCI device/vendor IDs
2) Instead of a simple grep, check the specific fields to
ensure we avoid accidental matching of a class code
against a device ID (or vice versa)
This will make it a bit cleaner to add some new device
IDs (i.e. virtio devices) in some future commits.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I36d4e2dec2f0f15dc0d603fc2512ea03a64dd560
Reviewed-on: https://review.gerrithub.io/374014
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
On some systems, hugetlbfs is reported as being mounted on 'nodev'
rather than 'hugetlbfs'; use the type instead of the device to match the
hugetlbfs filesystem.
Change-Id: I56498b7cf01de9759669fe313048b43b2319b933
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/368228
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Adding group write permission is needed in case of using vfio
driver and using libvirtd for vm provisioning to avoid
permission errors.
Change-Id: I5348e74c65bd8639071c1a2a164f28849dde9678
Signed-off-by: Karol Latecki <karolx.latecki@intel.com>
If we created /mnt/huge due to no hugetlbfs mount point existing, we
should also use that path in the chown step for VFIO.
Change-Id: I11614a67b16dbda6626b9adf58d8faea10205cab
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
This scripts adds new parameter "status", which shows extra
information about devices connected to the system:
- Numa node on which device is located
- Currently bound driver
- Device name in OS if available
Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Change-Id: If77d5fc20330b2492afd87bf832664c450705624
Default is still 1024 2MB huge pages, but user can
override this by setting a different value in the
environment.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I8d189d3a71f474469a44b6eb41b4443591863f3c
In some circumstances, no utmp entry is available, so logname fails.
In particular, gnome-terminal no longer creates a utmp entry:
https://bugzilla.gnome.org/show_bug.cgi?id=747046
As a workaround, try $SUDO_USER - the use case here is to determine
the (unprivileged) user name so we can give them ownership of
certain files, so this is usually the right thing to use anyway.
If we are not running under sudo, the caller should have passed the
username as a parameter to scripts/setup.sh anyway, since we can't
reliably determine which user is intended.
Also check if username is actually set before using it to run chmod - it
is possible that the scripts/setup.sh caller does not want to provide
access to an unprivilieged user and just wants to run everything as
root.
Change-Id: I20631c325b52884a378029dcf38568a2b311b457
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Fix typo in checking for mounted hugepages and replace expr with
arithmetic context as expr is returning error when expression is zero.
Change-Id: I2433fa938b3b2e758e1a4267414c10c5a3a5caff
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
This patch also drops support for automatically unbinding
devices from the kernel - run scripts/setup.sh first.
Our generic pci interface is now hidden behind include/spdk/pci.h
and implemented in lib/util/pci.c. We no longer wrap the calls
in nvme_impl.h or ioat_impl.h. The implementation now only uses
DPDK and the libpciaccess dependency has been removed. If using
a version of DPDK earlier than 16.07, enumerating devices
by class code isn't available and only Intel SSDs will be
discovered. DPDK 16.07 adds enumeration by class code and all
NVMe devices will be correctly discovered.
Change-Id: I0e8bac36b5ca57df604a2b310c47342c67dc9f3c
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Allocate larger contiguous regions (but the same amount of total
memory).
Change-Id: I54d09c11afdc7bdf232cc3b0864719c063a74b1a
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
This increases the largest contiguous region available for any single
object (e.g. DPDK mempools) from 32 MB to 64 MB.
Change-Id: I46633b246bcf7843053ab9d0d92993a784287a57
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
For now, drop the option to configure the number of hugepages (just use
the default 1024).
Change-Id: I686b6151e4a32ffa4ae9a9cf2694693369e03f49
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
cleanup.sh and unbind.sh have been combined into a single
setup.sh that takes one optional parameter (reset). If no
parameter is given, the script will automatically bind
all NVMe and IOAT devices to either uio_pci_generic
or vfio-pci, as appropriate based on IOMMU settings. If
the reset parameter is given, the devices will be bound back
to the appropriate kernel drivers.
Change-Id: I25db3234f1ecfb352a281e5093f4c1aa455152ae
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
