In case someone clones bare SPDK repo and attempts to execute the
RPM build, it will fail due to missing submodules. Since it's cheap,
just make sure submodules are in place before the build starts.
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I910949ae5fc5ef19c40f4abfc6f39ebb3fe8929a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16307
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Kamil Godzwon <kamilx.godzwon@intel.com>
per Intel policy to include file commit date using git cmd
below. The policy does not apply to non-Intel (C) notices.
git log --follow -C90% --format=%ad --date default <file> | tail -1
and then pull just the year from the result.
Intel copyrights were not added to files where Intel either had
no contribution ot the contribution lacked substance (ie license
header updates, formatting changes, etc)
For intel copyrights added, --follow and -C95% were used.
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I2ef86976095b88a9bf5b1003e59f3943cd6bbe4c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15209
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Add librbd and librados install and build dependencies when "with-rbd"
config option is set.
This will be used by the Ceph NVMe-oF gateway.
Signed-off-by: Mykola Golub <mykola.golub@clyso.com>
Change-Id: I1534e224cdfc166061a166ceef18261eef2d8cd8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11135
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
rpmbuild has a built-in variable that enables tracing for each script
template. This generates unecessary noise throughout the build part so
disable it there.
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I1a0010d518915d11af4a628b929edbc3b670d56e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14803
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Latest rpmbuild has an enhanced set of macros which define different
compiler flags inside the build environment. Their defaults, however,
may impact the overall SPDK build (and its submodules) in a bit
unpredictable manner. With that in mind, make sure that defaults are
not use and only flags specified by the user are kept in use.
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: Ie3a3123011a43be78f20f805ea6f8cad6571beec
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14802
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I0c8b5549080c1617a5eb0c1cc6145af6f1b0845d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14801
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
In case --with-shared build bundles with --with-vfio-user the vfio .so*
files must be also included for the applications to link properly. To
that end, create a separate package that will ship with these files on
board.
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I2c7e933b531b67a00e5af1e3b4dfbfaf8759d7ad
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14800
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: Ic53bd1733e2462c8d5c5d0ff057405fbcb20c19b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14799
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
make install already does that for us - from build/examples some
subset of binaries is selected but in the rpm scope it should be
enough.
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I3045711fc3daead9c255fc586600abe5796bc783
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14798
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: <sebastian.brzezinka@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
This can speed up these commands by passing, .e.g, -j N args as per
default defined in rpm.sh.
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: Ia64d3c0a0eef18e449e3020f31eaae8dd4caf483
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14797
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: <sebastian.brzezinka@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I7b7e4118e3573d3d1ad83a50d4a980bd69c8c0f4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14480
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: <sebastian.brzezinka@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
dpdk packages are versioned differently across different distros. In
particular, packages under fedora use Epoch in the spec. This requires
the condition to include this value, otherwise rpmbuild assumes it's 0
failing the check.
Also, add extra log to build_rpm_with_rpmed_dpdk test to indicate
when rpmbuild fails due to not resolved build dependencies.
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: Ia4bab6e6486e20c24f7386a085b499462fe5fe0d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14662
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This is done mainly to align the final location with the one listed
in the .pc files created for each lib.
Signed-off-by: Michal Berger <michallinuxstuff@gmail.com>
Change-Id: I99d6875c9e26f8202723dd6d73bd865b6478bcf9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12978
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Up until now, importing an SPDK RPC python module was just a matter of
`import rpc`. It's fine until there's another module called `rpc`
installed on the system, in which case it's impossible to import both of
them. Therefore, to avoid this problem, all of the modules were moved
to a separate directory under the "spdk" namespace.
The decision to move to a location under a separate directory was
motivated by the fact that a directory called scripts/spdk would look
pretty confusing. Moreover, it should make it also easier to package
these scripts as a python package.
Other than moving the packages, all of the imports were updated to
reflect these changes. Files under python now use relative imports,
while those under scripts/ use the "spdk" namespace and have their
PYTHONPATH extended with python directory.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ib43dee73921d590a551dd83885e22870e72451cf
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9692
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This step was accidently removed by 47ee30d3e3.
Fixes issue #2332.
Signed-off-by: Michal Berger <michallinuxstuff@gmail.com>
Change-Id: I7ad9490715567b9a3a4c48ce67f3b038956654cd
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11174
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
USE_DEFAULT_DIRS can be used together with GEN_SPEC to not hardcode
our custom paths into the .spec and instead use default set of dirs -
this should allow users to not define custom dir macros for their
own rpmbuild instances and simply run the build against the .spec:
$ GEN_SPEC=yes USE_DEFAULT_DIRS=yes ./rpmbuild/rpm.sh > foo.spec
# .. prepare source ...
$ rpmbuild -ba foo.spec
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Ia50ac303dfe9090fbd424e63e9eee7d939415ac2
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8390
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Part of #2256
* configured
Change-Id: Id63410c990522e74263f6612e43996237a9ce0ee
Signed-off-by: Josh Soref <jsoref@gmail.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10407
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Details of the changes here:
918fd2f146.
They mainly target the aesni_mb driver which was moved to ipsec_mb and
bump the minimal supported version of the ipsec to v1.0.
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Ica3b4fd66684751939159511845eb6ac6f7d5205
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10003
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
This suite can be used to deploy containers with the following
functionality (more details in README.md):
- storage-target
- proxy-container
- traffic-generator
This will run simple fio test as per fio.conf against nvmf controller
provided by initiator-container. Similar task can be performed directly
from initiator-container as well.
Each container includes SPDK installation with most common tools, e.g.
rpc.py, available under $PATH. This allows for something like:
docker-compose exec storage-target rpc.py nvmf_get_subsystems
Note that SPDK environment heavily depends on a running kernel hence all
the containers need to be privileged. That said, to make sure containers
are not affecting the host too much, some tasks must be done prior running
them. This includes:
- loading proper kernel modules (like nvme-fabrics, etc.)
- allocating hugepages and having at least one hugetlbfs mount
available under /dev/hugepages
base_build is created as docker multi-stage build.
This is done in order to decrease the size of the final image. The
SPDK RPMs are built inside a base image and then copied over to the
main image (+ fio binary) - this leaves all the dependencies inside
the intermediate image instead of the final one.
The resulted difference in size may look similar to the following
(it may differ depending on the docker version etc.):
no multi-stage build: spdk_base == 1.04GB
multi-stage build: spdk_base == 261MB
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I825bd0d0bb4071bd9d44b6a0749c033894899ae0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9055
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
As per os-release(5), if $ID is not recognised, $ID_LIKE can be used
to identify similar systems in preference order. Modify the script to try both
variables.
Also do the same for the RPM build scripts.
Signed-off-by: John Levon <john.levon@nutanix.com>
Change-Id: I4ed924df01ec678aab232e114d8c9980463c38e0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9260
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
This is done in order to detect if user wants to build spdk RPMs
against DPDK RPMs that might have been installed on the system.
This boils down to the following:
- if --with-dpdk, with no argument, is detected don't build
separate RPM holding DPDK libs since user in this case is
most likely interested only in packaging the SPDK so it
can coexist with separate DPDK packaging workflow
- define install and build requirements for the SPDK RPMs
to depend on dpdk-devel RPM
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I4dd587009da282a114524c74d833fd35ebc5b985
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8349
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This particular section must be defined in one place hence the
%{requirements_list} cannot be passed down later on. This requires
reordering of the .spec a bit. Most notably, all global macros are
now defined at the very beginning so then Requires section can be
build properly.
Also, use %() to expand the macro which holds the requirements as
bare "Requires: %{foo}" is not valid.
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I69430f73d10204325fc825ecc392506f261ebaf1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8347
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Alternative for the default of $HOME/rpmbuild.
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Id1feb7207926b518deb87045fc17bb3d1d4c374e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8159
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
For now this is kept at its very basics. Dependencies are handled
via pgkdep, they are not explicitly defined by the .rpm itself.
Currently, up to four .rpm packages are being built:
spdk
spdk-devel
spdk-libs
spdk-dpdk-lib
Together they include all binaries|libs|header files + some setup
scripts which are commonly used throughout the repo. Installation
paths are hardcoded to:
/usr/local/{bin,lib{,/dpdk},include}:
- binaries
- libraries
- header files
/usr/libexec/spdk:
- scripts
/etc:
- configuration files
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Ic5f067c4e7b8da3d697ee469bc9c794d5a0a035b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6436
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
