The setup.sh can now be invoked as following:
```
HUGENODE=0 NRHUGE=5 ./scripts/setup.sh
HUGENODE=1 NRHUGE=3 ./scripts/setup.sh
```
This will allocate 5 hugepages on node0,
and 3 hugepages on node1. If HUGENODE
param is not specified, the setup.sh will
operate just like it used to.
Change-Id: Ib198f5f32203abcc646af3c8d823f2b3e9bce362
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/384086
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
The previous defines have been
removed by patch 80b44ea.
Fixes: 80b44ea ("rte_virtio: remove unused PCI ID #defines")
Change-Id: I0e90e7f91a5e91efca3e5562b94cb1f6a36e7428
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/382935
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Use lsblk to detect if an NVMe namespace or any partition on the
namespace has an active mountpoint. If it does, do not unbind
the NVMe device associated with that NVMe namespace.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I1ab7540d640baa201efac49bc9515fd861dd8f8c
Reviewed-on: https://review.gerrithub.io/382479
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
If the user did not have the ioatdma driver loaded for example,
when running "setup.sh reset", we should not load ioatdma so that
we can bind to it. Instead just unbind the devices from the
uio/vfio driver.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I953941eb4918105b97ad78987b47b33f4372ae01
Reviewed-on: https://review.gerrithub.io/382474
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
This allows to use same command
for allocating hugepages on
systems with different hugepage
sizes.
The old NRHUGE variable can be
still set on Linux machines,
but is currently undocumented.
Change-Id: I1fba315d95e9aae8b6a8c9c445deb447fecc65dc
Signed-off-by: Hailiang Wang <hailiangx.e.wang@intel.com>
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/369546
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
When a DPDK app closes, it leaves behing configuration files that can
cause latent failures in the build pool machines if they are not
removed.
Change-Id: I80caa1b595e6586a109c013b92dfcf306d7a0150
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/379874
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Supports both PCI mode (for usage in guest VMs) and
vhost-user mode (for usage in host processes). The rte_virtio
subdirectory contains a lot of code lifted from the DPDK
virtio-net driver. Most of the PCI and vhost-user code is
reused almost exactly as-is, but the virtio code is drastically
rewritten as the DPDK code was very network specific.
Has been lightly tested with both the bdevio and bdevperf
applications in both PCI and vhost-user modes.
Still quite a bit of work needed - a list of todo
items is included in a README in the module's directory.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I85989d3de9ea89a87b719ececdb6d2ac16b77f53
Reviewed-on: https://review.gerrithub.io/374519
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Another atempt to fix setup.sh On Ubuntu 16.04 that use GNU Awk 4.1.3.
Eg when subsystem vendor name and subsystem name are empty:
$ lspci -mm -n -D
...
0000:00:05.0 "0880" "8086" "0e28" -r04 "8086" "3582"
0000:00:05.1 "0880" "8086" "0e29" -r04 "" ""
0000:00:05.2 "0880" "8086" "0e2a" -r04 "8086" "3582"
...
tr -d '"' part is brekaing whole awk script. So lets try to escape
vendor and device ID to comapre them as whole string.
Change-Id: Ic41e82c41347c6e0a8246b445f86eddf56780089
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/374462
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Device IDs starting with 0e get treated as 0 due to numerical
interpretation (0 * 10^x = 0). So use ~ to do a string regexp
comparison instead.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I4b5558c0127b0c4f021daf8151bf4d3f514e44da
Reviewed-on: https://review.gerrithub.io/374507
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
1) Create separate functions for checking the PCI class code
vs. checking the PCI device/vendor IDs
2) Instead of a simple grep, check the specific fields to
ensure we avoid accidental matching of a class code
against a device ID (or vice versa)
This will make it a bit cleaner to add some new device
IDs (i.e. virtio devices) in some future commits.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I36d4e2dec2f0f15dc0d603fc2512ea03a64dd560
Reviewed-on: https://review.gerrithub.io/374014
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
On some systems, hugetlbfs is reported as being mounted on 'nodev'
rather than 'hugetlbfs'; use the type instead of the device to match the
hugetlbfs filesystem.
Change-Id: I56498b7cf01de9759669fe313048b43b2319b933
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/368228
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Adding group write permission is needed in case of using vfio
driver and using libvirtd for vm provisioning to avoid
permission errors.
Change-Id: I5348e74c65bd8639071c1a2a164f28849dde9678
Signed-off-by: Karol Latecki <karolx.latecki@intel.com>
If we created /mnt/huge due to no hugetlbfs mount point existing, we
should also use that path in the chown step for VFIO.
Change-Id: I11614a67b16dbda6626b9adf58d8faea10205cab
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
This scripts adds new parameter "status", which shows extra
information about devices connected to the system:
- Numa node on which device is located
- Currently bound driver
- Device name in OS if available
Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Change-Id: If77d5fc20330b2492afd87bf832664c450705624
Default is still 1024 2MB huge pages, but user can
override this by setting a different value in the
environment.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I8d189d3a71f474469a44b6eb41b4443591863f3c
In some circumstances, no utmp entry is available, so logname fails.
In particular, gnome-terminal no longer creates a utmp entry:
https://bugzilla.gnome.org/show_bug.cgi?id=747046
As a workaround, try $SUDO_USER - the use case here is to determine
the (unprivileged) user name so we can give them ownership of
certain files, so this is usually the right thing to use anyway.
If we are not running under sudo, the caller should have passed the
username as a parameter to scripts/setup.sh anyway, since we can't
reliably determine which user is intended.
Also check if username is actually set before using it to run chmod - it
is possible that the scripts/setup.sh caller does not want to provide
access to an unprivilieged user and just wants to run everything as
root.
Change-Id: I20631c325b52884a378029dcf38568a2b311b457
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Fix typo in checking for mounted hugepages and replace expr with
arithmetic context as expr is returning error when expression is zero.
Change-Id: I2433fa938b3b2e758e1a4267414c10c5a3a5caff
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
This patch also drops support for automatically unbinding
devices from the kernel - run scripts/setup.sh first.
Our generic pci interface is now hidden behind include/spdk/pci.h
and implemented in lib/util/pci.c. We no longer wrap the calls
in nvme_impl.h or ioat_impl.h. The implementation now only uses
DPDK and the libpciaccess dependency has been removed. If using
a version of DPDK earlier than 16.07, enumerating devices
by class code isn't available and only Intel SSDs will be
discovered. DPDK 16.07 adds enumeration by class code and all
NVMe devices will be correctly discovered.
Change-Id: I0e8bac36b5ca57df604a2b310c47342c67dc9f3c
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Allocate larger contiguous regions (but the same amount of total
memory).
Change-Id: I54d09c11afdc7bdf232cc3b0864719c063a74b1a
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
This increases the largest contiguous region available for any single
object (e.g. DPDK mempools) from 32 MB to 64 MB.
Change-Id: I46633b246bcf7843053ab9d0d92993a784287a57
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
For now, drop the option to configure the number of hugepages (just use
the default 1024).
Change-Id: I686b6151e4a32ffa4ae9a9cf2694693369e03f49
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
cleanup.sh and unbind.sh have been combined into a single
setup.sh that takes one optional parameter (reset). If no
parameter is given, the script will automatically bind
all NVMe and IOAT devices to either uio_pci_generic
or vfio-pci, as appropriate based on IOMMU settings. If
the reset parameter is given, the devices will be bound back
to the appropriate kernel drivers.
Change-Id: I25db3234f1ecfb352a281e5093f4c1aa455152ae
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
