This will enable us to create a helper applciation to print memory
statistics for a given SPDK application.
Change-Id: I0ad5a8163166a5d67d9a5533303ec0858297ba6b
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/477511
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Add registration and unregistration of block zoned bdev. Attach it to the
underlying bdev during creation and unattach at deletion.
Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: I773aff6c7609952f28c02dd1794f0529a781b2e1
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/468033
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Adding new bdev module - zoned block device virtual bdev. It should be
possible to build on top of a regular (ie. non-OCSSD) bdev and surface
a zoned device API instead.
Added create and deletion functions for this bdev.
Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: Ia383483007117d1c826298fd391467a51fa2fe4e
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/468030
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This patch adds support for hotplug / hotremove detection for devices
behind the VMD. The detection acts similarly to the one implemented for
regular PCIe devices, that is user has to periodically call probe
function. Additionally, for applications not using SPDK's event
framework, spdk_vmd_hotplug_monitor has to be called periodically as
well.
Change-Id: I9f6839560efcf16c839b01976639d835f119cb47
Signed-off-by: orden smith <orden.e.smith@intel.com>
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/472741
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
These will be used from the posix sock layer.
Change-Id: I6427d9e9aee889e11ba7e36876112a5aba449e31
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/471751
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
NVMe character device implementation. This patch adds implementation
of IO producer using CUSE library. It allows to create nvme device
nodes in linux kernel for controller as well as for namespace and
process ioctl requests as usual from linux environment.
Both devices (controller and namespaces) are exposed as character
devices.
To compile NVMe CUSE module use "./configure --with-nvme-cuse".
Names for created CUSE devices can be retrieved using
spdk_nvme_cuse_get_ctrlr_name() and spdk_nvme_cuse_get_ns_name().
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I0fc9a9a1ef3c9c2b3112d07c2b4b1f8d49665ee1
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/466917
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Add module blobfs_bdev as a general module to simplify
the operations of blobfs on bdev. Then its functions
can be utilized by other libraries or apps.
blobfs_detect can be used to detect whether there is
one blobfs on given bdev.
Change-Id: Ib425574816061dc945fb652b539f791a44097a43
Signed-off-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/466486
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Instead of having two places in the code that do the
syscalls to set up the recvbuf size, just call the
function we already have.
Change-Id: I7d098fc7a39d0593c58fbe05b1f0b14f8b6a360d
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/470509
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
This is useful for testing partial sends in the socket layer.
Change-Id: Ib5bb62d641ad8d6a23a662261f7c72f5ba55c813
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/470520
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
This isn't really needed, and actually hinders
things like Seastar which use more recent C++
variants.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I7ce3d0578d6832a5d2b26c661bf7a7e2119bcac5
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/470079
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
We previously were failing for all make targets except for clean if the
mk/config.mk file didn't exist, but we should also be failing to
perform make clean if we don't have this file. It turns out you end up
erroring out in the next line anyways. Also, if you make that include
optional, make still fails because we don't have any kind of environment
specified and spdk.common.mk depends on that variable being defined.
There may be some benefit to being able to run make clean if we don't
have a valid configuration file, but right now, there are a lot of
things that depend on the configuration file, such as which environment
file to make or clean and which directories need cleaned.
Change-Id: I2a9c1e982c01ba76b67b153ee7336402300b76b0
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/466346
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
The unit test mock macro function doesn't mesh well with the scan-build
static analysis tool. So we need to reset cc to the user provided value
or default before running the unit tests.
Change-Id: Ib636f6e396b09f9bf1cb8f930e6befa56ce99848
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/466816
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Everywhere we use this, we increase the size to 2MB.
Just make that the default behavior.
Change-Id: Ie174419a09df1792a0c7311eddd0c2dcefa267b5
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/466991
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
The dependencies between vhost and rte_vhost were not added during
earlier changes. This change moves the rte_vhost directory up to the
level of the other libraries and adds the proper dependencies for when
it is linked.
Change-Id: I089de1cd945062b64975a0011887700c0e38bb0f
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/467700
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
The SPDK application framework defines a list of event subsystem
dependencies. When linking against individual shared libraries, it is
useful for this dependency structure to be codified in the shared
libraries themselves.
For example, when linking a bdev based application against
libspdk_bdev.so, one might wish to only specify this shared object at
link time. However, when you actually run the application, it will fail
to start because it is not linked to the copy and vmd subsystems.
However, once thesedependencies are added, one can effectively link
against only the exact subsystems they need and any dependent subsystems
will be linked in automatically.
Change-Id: Ic986281a162ac20b523486e9f8cccf4a0787afd7
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/466081
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
The OCF build was broken by some of the recent changes
to the Makefiles. This change aims to fix that by separating out the ocf
environment from the ocf bdev.
Change-Id: Id445340033898e9ae70a4bcfc799951110762d55
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/465808
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This will allow us to link individually against dpdk module libraries
without having to define those libraries dependencies.
Change-Id: Ief2140ec0fadd970aba990dab333d603dfb46317
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/465704
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This will make it much easier to link against some subset of spdk
libraries when building SPDK applications.
Change-Id: I8ad95a001965288a8b5e38eb252391fef68d7138
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/465194
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
By adding this file and modifying the way we define a couple of
variables in the makefiles, we can actually avoid having to redefine
the library dependencies on a per file basis.
Change-Id: Ieab4aa1021b0341fc21e3b65677a9ad7f70559c2
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/466063
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
This is more accurate to what they are, and will make defining library
dependencies much simpler. This change in directory does not affect the
final placement of naming of libraries at the end of time.
Change-Id: Ic48a9233dff564e39ce357a9ea0a111ea2b6414b
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/465454
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
The subdirectories of libraries are currently listed as dependencies of
the library. If you look at the actual symbols they contain, the
opposite is true. For all of the bdevs, event subsystems and other
library directories, the subdirectories actually rely on symbols
contained in the parent directory.
If we want to create the proper inter-library dependencies in SPDK at
compile/link time, then we need to respect the order of these symbol
dependencies when executing make (e.g. make libspdk_event.so before
libspdk_event_bdev.so). Otherwise we create a loop of dependencies in
the make system.
Change-Id: Idcbb050b2709f575d0cec46b481824d6313612b4
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/465191
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
When building shared libs, we specifically don't set the --as-needed
flag so that we still link against libraries that have constructor
functions. LD reports on its man page that the default behavior is
equivalent to --no-as-needed.
For RHEL based distros like fedora and centos this works fine. While
the LD man page shipped with ubuntu distributions also states that
--no-as-needed is the default, they don't respect that behavior and do
--as-needed linking unless you force them with --no-as-needed.
Change-Id: I914ab849323de198af5c5e53fffb1f57fcaff5fe
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/464621
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
We need to know this up front.
Change-Id: I3a9ceb90cf62eacbf3fdf518a9ccb4c4978b3a05
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/463014
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Previously it always forced armv8-a+crc. Now it only does that
if not otherwise specified.
Change-Id: I7c1d73fca8e89779cc3e8780a38f52975488d97a
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/463012
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
This patch prepares environment to allow compile applications and
libraries with VPP.
Change-Id: Icc067c17b57932062afa5e6a67791bd58de3a2cc
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456461
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
New flag -V was added for VMD enabling.
Change-Id: Ie0feae1bdfce39ad246f57411d00587c6417d21d
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/450926
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This module simply sits on another virtual bdev
and adds a simulated average and p99 latency to that drive.
Change-Id: Ie9fc91e27585fd0636cb7dc845cb41744bf24625
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453594
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Signed-off-by: Orden Smith <orden.e.smith@intel.com>
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Change-Id: I4a3c4d1ca8d0b18201edf99a67a3d75ca7ab8153
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449499
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Add uninstall target to makefiles to be able to perform
reverse of install target.
Fixes#464
This patch adds 'uninstall' target to makefile.
'make uninstall' will remove spdk_tgt app, headers, libraries
and shared libraries from system directories defined by $DESTDIR.
Additionaly, if there will be any empty directories left after
this operation, they will be removed as well.
Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Change-Id: I7b07fb4b81081d3914ff09165991fbe3a26b9067
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/431471
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Instead of linking to the real sock library in the unit tests,
link to a mocked up version like we do with the env.
Change-Id: I9607f135c6af86e09fa52badf44897f90ce94521
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/454492
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Not only 4KB page size is supported on AArch64. The most common is 4KB
and 64KB. 16KB is supported too. We can get it by "getconf PAGESIZE".
Change-Id: I6f09741372d35e868228528d976fe3b5a65ff96c
Signed-off-by: Richael Zhuang <richael.zhuang@arm.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/454447
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
ARM system doesn't provide definition of PAGE_SIZE. The patch adds the definition of that symbol to the compiler flags for ARM system
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Signed-off-by: Sasha Kotchubievsky <sashakot@mellanox.com>
Change-Id: Iae0a528569aedaea7df95d9f0d9256cb23b9798b
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453657
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
FTL doesn't have any kind of special package requirements.
It is getting pretty good traction in the community, so
let's enable it by default.
Note that we will disable FTL on FreeBSD. FTL uses
CIRCLEQ which is not available on FreeBSD. Let's not
spend time trying to get FTL to work on FreeBSD until
there's a demand to do so.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I15525b6c4e6ee52f49adf74d55f9484fe08a6dcc
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452752
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Just the starting point, base functions and structs.
Change-Id: I2ff2b672aa675a19b89e4449f1cc4aa664007f6f
Signed-off-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/435747
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Don't assume detection of LD by the detect_cc script suceeded. If it
wasn't detected then LD_TYPE won't be set and the gcc command line
option shouldn't be set either. Compilation will fail if -fuse-ld is
specified with no value.
Change-Id: If4a0ede8cdc5a71cf92de2beb0f24e4650f58f3a
Signed-off-by: Jonathan Richardson <jonathan.richardson@broadcom.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/447376
Reviewed-by: Scott Branden <sbranden@gmail.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This is a beginning of series that will provide
common code for all NVMe based bdevs.
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I097b4a03ef79277659f45d217c7e4e58236c98bf
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/445893
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Add a CLEAN_FILES macro that shared_lib/Makefile can use
to add to the list of files to be cleaned.
Fixes#663.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I12982e0989e02a69aaea4e470777301280090096
Reviewed-on: https://review.gerrithub.io/c/444427
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
./configure --enable-pgo-capture
make
<Run sample workload>
./configure --enable-pgo-use
make
Output has used the captured profile from the sample workload to improve
performance.
Change-Id: Ie5690b873b05d11b4ea6c7d44021e564d4e4d170
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/444110
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
In SPDK, we will build isa-l with no shared option
and then integrate it into SPDK. And we do not need
to install isal in the system libaries.
Note: ocf build in autobuild.sh now needs to build
include/spdk/config.h before building the ocf library,
to ensure that header is available in a clean build
environment.
Change-Id: I3f0ce6932b386de17a77cf5bfdfd738b22417e2d
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Signed-off-by: paul luse <paul.e.luse@intel.com>
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Reviewed-on: https://review.gerrithub.io/c/441279
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Chunyang Hui <chunyang.hui@intel.com>
Add OCF module based on OCF meta-library
Open CAS Framwework (OCF) is high performance block storage
caching meta-library
It is open-source, published at https://github.com/Open-CAS/ocf
With this patch OCF-enabled device is represented in SPDK
as virtual bdev having core and caching devices as its base devices
This patch includes implementation of:
* OCF top adapter (vbdev_ocf.c)
* OCF bottom adapter (dobj.c, data.c)
* Adaptation layer for OCF (env/)
* OCF context abstractions (ctx.c)
Adaptation layer and context abstractions are not dependent on SPDK bdev
OCF bdev supports reads and writes, configured at startup
Other features will be added with separate patches
Change-Id: Ic2dcab378c8238d16f1e4b64d4374bdf257565bc
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.gerrithub.io/c/435708
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This patch adds FTL bdev. RPC scripts have been updated to allow for
creation and removal of FTL bdevs.
Change-Id: I82a5c5033b65bbeb67c238cae969a68cff767dcc
Signed-off-by: Jakub Radtke <jakub.radtke@intel.com>
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/431329
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This allows us to remove the requirement to install intel-ipsec-mb to
system directories.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I579655a98b515cf148b7cd17823a9bb541ea6ad7
Reviewed-on: https://review.gerrithub.io/c/440785
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Implement spdk_crc32c_update() with ARM CRC32 intrinsics.
Change-Id: I1daf7f21012aab02290f88a65bbae619eedf5087
Signed-off-by: Richael Zhuang <richael.zhuang@arm.com>
Reviewed-on: https://review.gerrithub.io/c/437218
Reviewed-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
SPDK threads are no longer mapped 1:1 to system threads. They are
instead identified by the memory address of the spdk_thread object.
Change-Id: I417f8376cebd2ee94f624f4436e6394b51486063
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/437999
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Added -ffunction-sections / --gc-sections to allow the linker to remove
unused functions when linking test executables. This reduces the number
of required mock definitions to those actually required by tested code.
Change-Id: I22725ce82e22675a7c2d28c8227e586f2cd65023
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/436319
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This patch adds core FTL (flash translation layer) submodules.
On regular SSDs the FTL is part of firmware, whereas Open Channel
enables moving it to the host and allows for having full controll
of data placement on the device.
Main functionalities added:
* logical to physical address map
* read / write buffer cache
* wear-leveling
* bad block management
Change-Id: I5c28aa277b212734bd4b1f71ae386b3d6f8c3715
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Signed-off-by: Jakub Radtke <jakub.radtke@intel.com>
Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/431322
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Modifed scripts/detect_cc.sh to take additional parameter specifing
the linker to use. Default to LLD on FreeBSD systems.
Change-Id: Idf97e9676a144028c0803d272ae6f0e903b0dd1f
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/438801
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This removes a ton of duplicate from a lot of
Makefiles. Note that spdk.unittest.mk still keeps
its own app rules, and now just includes
spdk.app_vars.mk instead. Fixing Makefiles that
include this .mk file requires more work that is
deferred for now. We do need to add the
"install: all" rule explicitly to this .mk files.
Note that Makefiles need to include spdk.app.mk
now at the end of the file, so that things like
APP and SPDK_LIB_LIST are defined before including
it.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: If6823cda135180c37690ffc96c91660b7f0718e3
Reviewed-on: https://review.gerrithub.io/434283
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This can just be replaced with ENV_LIBS - similar
to other application Makefiles.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Icad14e6f74264389972c7798aac493e584f7c8c3
Reviewed-on: https://review.gerrithub.io/434280
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This is just a copy of spdk.app.mk for now. But later
patches will modify spdk.app.mk in a "C-specific" way,
so add this cxx variant to use now in the C++ apps.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I0d4d28c6119364d4c3cc7fa5274598b69f4ee6b1
Reviewed-on: https://review.gerrithub.io/434279
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This is a step towards moving a lot of the boilerplate
rules in various application Makefiles into mk/spdk.app.mk.
We will need to add a mk/spdk.app_cxx.mk for our C++
applications, but that will need to share these same
variables.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iaee73a4fa47ff2ee54f1a1ea3c0d2097900df218
Reviewed-on: https://review.gerrithub.io/434278
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
currently they are saved in mk/config.mk but this break config.h
Change-Id: Iecb037f1aea91469f4093724cdf100dc122d21a5
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/427222
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Introduces new macro INSTALL_EXAMPLE and two example apps,
examples/nvme/perf and example/nvme/identify that make
use of it to install them while at the same time
renaming them in the target directory based on the
source directory path relative to examples.
Change-Id: I2d850458bb2589f80e0af6fb7a9d00aa3bbc6907
Signed-off-by: Lance Hartmann <lance.hartmann@oracle.com>
Reviewed-on: https://review.gerrithub.io/429963
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Application Makefiles can now just add xx_MODULES_LIB_LIST
to SPDK_LIB_LIST. This is possible now since all
SPDK libraries are linked with --whole-archive, so there
is no need to differentiate between "modules" libraries
and other SPDK libraries.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iabf81a388b72d3b2a2f48287a8491ddc977722ac
Reviewed-on: https://review.gerrithub.io/434277
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
This allows a lot of simplification to SPDK application
makefiles.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I5fa463f6369834b84a8d92e79fa7768082209d7a
Reviewed-on: https://review.gerrithub.io/434274
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
This is a step towards simplifying the Makefiles for
SPDK applications. After this change, all of the
xxx_MODULES_LINKER_ARGS variables are just a list
of SPDK libraries wrapped in --whole-archive.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I0cf069c2f74e8998d42dea9598db55f333c861b7
Reviewed-on: https://review.gerrithub.io/434146
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
This allows us to remove $(ENV_LINKER_ARGS) from all of the
application Makefiles.
Modify spdk.unittest.mk to explicitly set ENV_LINKER_ARGS
to nothing, to avoid DPDK libraries from getting linked
into unit tests. This seems better than creating separate
LINK_C rules for unit tests. Provide a UNIT_TEST_LINK_ENV
flag to allow select unit tests to override this behavior -
it's needed for some of the env unit tests.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I647f2c848cf434023cf5a1f3a45869bbc4fc8102
Reviewed-on: https://review.gerrithub.io/434145
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
SPDK uses constructor functions for registration functionality
in several different areas - RPCs, bdev modules, event tracing,
etc. Since these functions are not explicitly referenced, the
linker will remove these constructor functions from the linked
application unless their library is included using the
--whole-archive linker option.
SPDK has tried to minimize the number of libraries linked with
--whole-archive to only those that are necessary. But there
are still a lot of exceptions that we need to work around. The
benefit to trying to restrict it is minimal - for example,
an nvmf_tgt binary built before this patch is 4746320 bytes
on my system, but linking all of the libraries with
--whole-archive only increases it to 4784080 bytes - a difference
of less than 1%.
Note that DPDK also links all libraries with --whole-archive by
default.
This will also simplify some upcoming changes around shared
library dependencies - we will be able to blindly use
--whole-archive when linking the shared library as well.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iab0dbbace06a3d15c835491e55683ab4cf9e53f0
Reviewed-on: https://review.gerrithub.io/434107
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
The various *_MODULES_LIST are intended for the modules
that implement the specific interface - not for the
associated core library itself.
For example, BLOCKDEV_MODULES_LIST doesn't contain bdev,
it contains all of the bdev modules. Similar for
COPY_MODULES_LIST and copy.
So make SOCK_MODULES_LIST consistent. This is a real
pain because we have to add sock explicitly to a bunch
of individual Makefiles, but I have plans to clean that
up in some future patches.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I0d293372cfb38bfa46c8478a8d9ca9ce23924747
Reviewed-on: https://review.gerrithub.io/434125
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
There was an implicit merge conflict between this patch
and e956be96e which added NVMe/TCP support including adding
"sock" as an nvme.libtest.mk depdendency.
This reverts commit 7962ce312d.
Change-Id: Ie7e58e89128f28080a986e12cd014de7df4752e5
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Reviewed-on: https://review.gerrithub.io/434103
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This matches the lib/ut_mock directory name, and also
avoids the final library being named spdk_spdk_mock.a.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I80ec09568d0455d1f131e45694b6e2d536e6ca91
Reviewed-on: https://review.gerrithub.io/433909
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
SPDK uses constructor functions for registration functionality
in several different areas - RPCs, bdev modules, event tracing,
etc. Since these functions are not explicitly referenced, the
linker will remove these constructor functions from the linked
application unless their library is included using the
--whole-archive linker option.
SPDK has tried to minimize the number of libraries linked with
--whole-archive to only those that are necessary. But there
are still a lot of exceptions that we need to work around. The
benefit to trying to restrict it is minimal - for example,
an nvmf_tgt binary built before this patch is 4746320 bytes
on my system, but linking all of the libraries with
--whole-archive only increases it to 4784080 bytes - a difference
of less than 1%.
Note that DPDK also links all libraries with --whole-archive by
default.
This will also simplify some upcoming changes around shared
library dependencies - we will be able to blindly use
--whole-archive when linking the shared library as well.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iab0dbbace06a3d15c835491e55683ab4cf9e53fe
Reviewed-on: https://review.gerrithub.io/432917
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
It is the first patch to follow the NVMe over fabrics
spec and implmenent the NVMe/TCP transport. It can be
divided into work in the host and target sides:
Host side: Add the TCP/IP transport in nvme lib (lib/nvme).
Target side: Add the TCP/IP transport in nvmf lib (lib/nvmf).
Change-Id: Idc4f93750df676354f6c2ea8ecdb234e3638fd44
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/425191
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
blob and blob_bdev are already included via the lvol bdev module
dependencies in mk/spdk.modules.mk. This patch adds blobfs there
too.
These changes are needed for some upcoming patches which will
apply --whole-archive to all libraries linked into the application.
That patch will require that we never include the same library
twice.
Shared library builds *require* that blobfs be listed before
blob on the link line, hence adding blobfs here even though it's
not technically associated with a bdev module.
We can also simplify spdk.modules.mk while we're here - just
filter out bdev_lvol to disable lvol for applications like
blob_cli that need to disable it.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I01e5e6b91d947463749f942f54c12f48229bbe5a
Reviewed-on: https://review.gerrithub.io/432916
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Let's just make them consistent and all with a
bdev_* format.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ia3dec7f4be2ecc765b4aa095e6b675f13d326063
Reviewed-on: https://review.gerrithub.io/429289
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Ziye Yang <optimistyzy@gmail.com>
Currently only the iscsi => scsi dependency is defined to
demonstrate how the framework behaves.
Eventually, lib/Makefile will be populated with more lines
of the form:
DEPDIRS-iscsi : scsi log conf event
This line will ensure that the iscsi subdirectory is built
after the specified dependency directories.
Then the lib/iscsi/Makefile also defines:
DEP_LIBNAMES = scsi log conf event
This line adds dependencies from the iscsi.so to the specified
libraries.
Ideally we could avoid this duplication by passing the DEPDIRS
value to the lower level Makefile, but that breaks down when we
start considering event/subsystems/* libraries. So for now we'll
need to duplicate it. Incidentally, DPDK also duplicates the
dependency information like this.
While here, remove use of $(MAKESUBDIRFLAGS) in spdk.subdirs.mk -
it was no longer used anywhere in the tree.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: If63cbc10ffa18ac6361e4210bd6196ffe88bec43
Reviewed-on: https://review.gerrithub.io/429287
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Lance Hartmann <lance.hartmann@oracle.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
We have hairy configuration method. First configure script uses CONFIG,
environment and arguments to produce CONFIG.local and config.h then
makefile uses CONFIG, environment and CONFIG.local to produce config.h
again. First time config.h can't be generated reliably as config.h
target in Makefile is passing MAKEFLAGS and they are unknown at
configuration phase.
Lets remove this mess by producing final mk/config.mk file and get rid
of CONFIG.local. This will also generate full build configuration in one
file instead scattering them in CONFIG and CONFIG.local.
In next patch scripts/genconfig.py can be removed.
Change-Id: I5696813312edff7e493207f7aa8569234a365ec0
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/426364
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
While here, change spdk_lib_list_to_files to
spdk_lib_list_to_static_libs to differentiate it from
the new spdk_lib_list_to_shared_libs.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I6e5913addfbdd556fae2451d4e2b2c43feaf33ab
Reviewed-on: https://review.gerrithub.io/429286
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Lance Hartmann <lance.hartmann@oracle.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This prevents the SPDK build from requirng unnecessary dependencies when
not compiling certain features. Also, fixes github issue #434
Change-Id: I7d0520474f3656ae32670313f2290e6b741c5ca8
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/426131
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Lance Hartmann <lance.hartmann@oracle.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This will allow us to specify more granular system dependencies for each
.so file.
Change-Id: I759876e09ffb36380a503962b4be8fc9d7c71b63
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/426129
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Lance Hartmann <lance.hartmann@oracle.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Also fix options that doesn't start with CONFIG_ prefix.
Change-Id: I32a24373328bab8f513d56c0c09b5a86589f690c
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/427767
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Each file that need to check SPDK_CONFIG_* options need to include
spdk/config.h explicitly.
Change-Id: If9f2a91ac4c2b1a300dcf88ec3e2a12714ad344a
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/427221
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
We need to include it in devel package.
Change-Id: I823200632e8bcb9fdb86c8cb5fbf3a651a710b78
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/426839
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
As a side effect, if SPDK_LOG_DISABLED is used in spdk_log() as log
level then no message will be printed.
Change-Id: I2d57b60a5a310a9ef2a1187a81088d0acf828742
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/425105
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
We kept this disabled by default originally because it lacked
multiple iov support needed for vhost and nvmf in-capsule data.
Now that support has been added, there is no reason to not
enable it automatically - it has no extra package dependencies,
so requiring users to specify --with-raid to configure is just
an extra unneeded step.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ia9796370e12a6f93993e70083de36114a37851bd
Reviewed-on: https://review.gerrithub.io/426003
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Add dependency $(ENV_LIBS) for test/env/*
as part of new variable $(ADDITIONAL_LIBS) which is direct dependency
of $(APP) in mk/spdk.unittest.mk
This solves the problem of
make not triggering LINKing for test/env/*
after changes made to lib/env_dpdk/*.c
Issue was that Makefiles in test/env/*/ classified tests as unittests
by including mk/spdk.unittest.mk
But unittests are not using env_dpdk which means that
unittests don't relink on env_dpdk update
Change-Id: I91b6690dc92d68bc69bbf11ac5336d3a7f56e4e4
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.gerrithub.io/425665
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Dariusz Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Initial support for softare AESNI_MB DPDK driver only.
Have tested (both aesni and QAT seprately and concurrrently) on underlying NVMe devices
with bdevio and a bdevperf script that runs IOs from 512B to 128K each with Q depths from
1 to 512 in powers of 2 for 30 seconds each run.
QAT can be included in the code (but not makefile) and marked as experimental
until we are ready to test in CI. It works well on 2 systems but is a big PITA to get
the hardware setup and configured for use with DPDK (IMHO).
Change-Id: If518c3df8e74e00efa18afdf194824c5e69778fc
Signed-off-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/403107
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This linker flag is really more important when dealing with external SO
files such as libuuid. It shoudl follow those linked options around
instead of all of our internal dependencies.
This series is aimed at fixing github issue 434
Change-Id: Iee65470632b5ab0adeb361005edc894e05c43ea4
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/425441
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Instead of adding the linker flags just before application compilation
on linux, move that linking to earlier in the process. This will also be
useful for the shared library build.
This series is aimed at fixing github issue 434
Change-Id: I84bcc6bd922ddb6e48adb2dfe0b154700773cf27
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/425435
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Lance Hartmann <lance.hartmann@oracle.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This allows us to better distinguish between external and internal libs.
This series is aimed at fixing github issue 434
Change-Id: I2ed141f909e7c4a800df02061007b0d23da25380
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/425434
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Lance Hartmann <lance.hartmann@oracle.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Unit tests define mock functions in the unit test code
itself. This links fine with static libraries, but
doesn't work if trying to link with shared libraries.
So change the unit tests to always explicitly link
with the static libraries.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I3a8a60162fcab6c73a216c4bb364175767fdc62c
Reviewed-on: https://review.gerrithub.io/422441
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
When we build the SPDK_FILTER_LIB_LIST, we should
maintain the dependencies between libraries. Primary
culprit is vhost depending on scsi. So add vhost
to SPDK_FILTER_LIB_LIST before scsi, and then stop
sorting the list unnecessarily before using it.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ib22a5016aa492c30a389a498ae0360fbcec7c12b
Reviewed-on: https://review.gerrithub.io/422439
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Updates build and install of shared lib to conform with
standard naming conventions for shared libraries, including
the setting of the lib's soname.
Change-Id: Ib46d298ab5f77fd8c32b7c04022b546446189428
Signed-off-by: Lance Hartmann <lance.hartmann@oracle.com>
Reviewed-on: https://review.gerrithub.io/421936
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Raid module:
============
- SPDK raid bdev module is a new bdev module which is
responsible for striping various NVMe devices and expose the raid bdev
to bdev layer which would enhance the performance and capacity.
- It can support theoretically 256 base devices (currently it is being
tested max upto 8 base devices)
- Multiple strip sizes like 32KB, 64KB, 128KB, 256KB, 512KB etc is
supported. Most of the current testing is focused on 64KB strip size.
- New RPC commands like "create raid bdev", "destroy raid bdev" and "get raid bdevs"
are introduced to configure raid bdev dynamically in a running
SPDK system.
- Currently raid bdev configuration parameters are persisted in the
current SPDK configuration file for across reboot support. DDF will be
introduced later.
High level testing done:
=======================
- Raid bdev is created with 8 base NVMe devices via configuration
file and is exposed to initiator via existing methods. Initiator is
able to see a single NVMe namespace with capacity equal to sum of the
minimum capacities of 8 devices. Initiator was able to run raw
read/write workload, file system workload etc (tested with XFS file
system workload).
- Multiple raid bdevs are also created and exposed to initiator and
tested with file system and other workloads for read/write IO.
- LVS / LVOL are created over raid bdev and exposed to initiator.
Testing was done for raw read/write workloads and XFS file system
workloads.
- RPC testing is done where on the running SPDK system raid bdevs
are created out of NVMe base devices. These raid bdevs (and LVOLs
over raid bdevs) are then exposed to initiator and IO workload was
tested for raw read/write and XFS file system workload.
- RPC testing is done for delete raid bdevs where all raid bdevs
are deleted in running SPDK system.
- RPC testing is done for get raid bdevs where existing list of
raid bdev names is printed (it can be all raid bdevs or only
online or only configuring or only offline).
- RPC testing is done where raid bdevs and underlying NVMe devices
relationship was returned in JSON RPC commands
Change-Id: I10ae1266f8f2cca3c106e4df8c1c0993ddf435d8
Signed-off-by: Kunal Sablok <kunal.sablok@intel.com>
Reviewed-on: https://review.gerrithub.io/410484
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
This will be necessary when the NVMe/TCP transport is added, which will
use the sock abstraction library.
Change-Id: I6561ccdad1ff575db2afa418eab2f2cd24806cb9
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/416477
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Also remove libspdk_net from the list and add it to each app that
originally linked it via NET_MODULES.
This will allow the use of SOCK_MODULES in application Makefiles without
pulling in the RPC libraries required by spdk_net.
Change-Id: Iad68ddd25bea24bba53a681c78d41e6e703f866d
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/416476
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
This separates the network interface management code (interface.c and
net_rpc.c) from the socket abstraction layer, which allows users that
only want the socket abstraction to avoid pulling in the JSON RPC
libraries.
Change-Id: I9b00285a70bac0c74c73353cfa900d4f3b2e465f
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/416475
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This is already required to build the shared library and iSCSI
components; simplify these Makefiles by adding -lcrypto to the central
SYS_LIBS in mk/spdk.common.mk.
Change-Id: I8fdaffbccc1294e24e32559387b0ed99d1deb0ce
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/415351
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
