Commit Graph

83 Commits

Author SHA1 Message Date
Alexey Marchuk
0ab4de566b pkgdep/debian: Install python3-dev
This package is required to install grpcio:

./scripts/pkgdep.sh --developer-tools
...
  Could not find <Python.h>. This could mean the following:
    * You're on Ubuntu and haven't run `apt-get install python3-dev`.
    * You're on RHEL/Fedora and haven't run `yum install python3-devel` or
      `dnf install python3-devel` (make sure you also have redhat-rpm-config
      installed)
    * You're on Mac OS X and the usual Python framework was somehow corrupted
      (check your environment variables or try re-installing?)
    * You're on Windows and your Python installation was somehow corrupted
      (check your environment variables or try re-installing?)

  ----------------------------------------
  ERROR: Failed building wheel for grpcio
Failed to build grpcio

Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: I1a9d6154de9ab86ba8062b53ba56002e226daa94
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15751
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: <qun.wan@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-12-08 14:15:52 +00:00
paul luse
eb53c23236 add (c) and SPDX header to bash files as needed
per Intel policy to include file commit date using git cmd
below.  The policy does not apply to non-Intel (C) notices.

git log --follow -C90% --format=%ad --date default <file> | tail -1

and then pull just the year from the result.

Intel copyrights were not added to files where Intel either had
no contribution ot the contribution lacked substance (ie license
header updates, formatting changes, etc)

For intel copyrights added, --follow and -C95% were used.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I2ef86976095b88a9bf5b1003e59f3943cd6bbe4c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15209
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-11-29 08:27:51 +00:00
Michal Berger
485f9f974a pkgdep/freebsd: Update name of the ceph package
Under latest 13.1 builds ceph package is now available as "ceph14"
so try to install it when the first attempt fails.

Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I4301f357d2c04881a1e8839c644cb000ba74e4e6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15495
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Kamil Godzwon <kamilx.godzwon@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2022-11-21 12:02:48 +00:00
Michal Berger
66c590e0b4 pkgdep/rhel: Upgrade pip under centos8, rocky8
Do this prior installing any pip package to make sure they are stable.

Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I842252fe33b62dddd16858806a7d334cf2139fac
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15292
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Kamil Godzwon <kamilx.godzwon@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2022-11-10 13:20:20 +00:00
Mykola Golub
8a5d487d7e scripts/pkgdep: add RBD support
Adds "--rbd" option to install build dependencies for the rbd bdev.

This will be used by the Ceph NVMe-oF gateway.

Signed-off-by: Mykola Golub <mykola.golub@clyso.com>
Change-Id: I7926433aaa398873133dfffea3c401ce3c1f7813
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11134
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2022-11-10 08:29:10 +00:00
Michal Berger
b35a6f39a6 pkgdep/rhel: Make sure pip3 is installed in first instance
There was a code that attempted to to use pip3 before we try to
actually install it. Change the order.

Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I5dc74b79b3adf132589bbf9ef10360d2eeed51ba
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15293
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-11-08 08:21:59 +00:00
Simon A. F. Lund
3bf8f8a39b pkgdep: bump liburing to v2.2
liburing v2.2 provides support for initialization of rings with "big"
submission and completion entries. Big entries allow embedding NVMe
submission and completion entries in the io_ring sqe/cqe and thereby an
enabler of "io_uring_cmd".

xNVMe utilizes the above. xNVMe v0.4.0 "vendored"/bundled liburing 2.2.
However, the bundling of liburing inside the xNVMe library is removed in
v0.5.0 to avoid linker issues. Instead, xNVMe v0.5.0 links liburing on
the system with discovery via pkg-config.

Thus, this change to update the liburing pkgdep to v2.2, in preparation
for upgrading the xNVMe submodule to v0.5.0.

Signed-off-by: Simon A. F. Lund <simon.lund@samsung.com>
Change-Id: Ibbacbcfa675f82257d41790b68c39dd7b1a82908
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14676
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-10-03 08:43:53 +00:00
Xinliang Liu
ecbdca493e pkgdep: add support for openEuler Linux
OpenEuler Linux although is not compatible with RHEL, but it uses the
same package manager DNF as RHEL and references the package naming of
RHEL.

OpenEuler Linux is becoming the mainstream linux distro in China. So
adding support for it makes sense. For more details about it see:
https://www.openeuler.org/en/.

Fixes: #2713
Change-Id: Ife38a1f897bf6170a0c3ed0a5238ceda018a41f2
Signed-off-by: Xinliang Liu <xinliang.liu@linaro.org>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14639
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: wanghailiang <hailiangx.e.wang@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Michal Berger <michal.berger@intel.com>
2022-09-27 10:20:53 +00:00
Michal Berger
a342847df7 pkgdep: Add clang packages to rhel-based distros
Currently these packages are installed through (sometimes weak)
dependencies. Install them explicitly instead.

Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I5d8f7c92be4a74028ce028eef33ec4ac6d2ad792
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14447
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Kamil Godzwon <kamilx.godzwon@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2022-09-20 16:00:28 +00:00
Karol Latecki
7dfb090f95 pkgdep: silence hash in markdownlint installation
Do not print messages in case rake or bundler
are not found.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I92145a409af0ae95bc9ae194c7830feabd3a4641
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14155
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Kamil Godzwon <kamilx.godzwon@intel.com>
Reviewed-by: Michal Berger <michal.berger@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-08-29 11:54:08 +00:00
Kamil Godzwon
bcf9753b15 scripts/pkgdep: DAOS dependency repository has changed
After recent DAOS 2.0.3 release (2022-07-14) repository has
changed its location.

Signed-off-by: Kamil Godzwon <kamilx.godzwon@intel.com>
Change-Id: I4ad97844fb529c8dde5e39b751c3d92c69b8c26d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13683
Reviewed-by: Michal Berger <michal.berger@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
2022-07-22 07:28:20 +00:00
Kamil Godzwon
eb341f672c pkgdep: do not install grpcio modules on Centos7
grpcio, grpcio-tools modules have compilation
problem on Centos7 and are not necessary under
this OS.

Signed-off-by: Kamil Godzwon <kamilx.godzwon@intel.com>
Change-Id: I2b48a3b1b2deafa983adf83424940ffa6ce7d20e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13070
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-06-21 12:59:16 +00:00
Denis Barakhtanov
386af793fd scripts/pkgdep.sh: Suse install DAOS bdev build dependencies
Signed-off-by: Denis Barakhtanov <denis.barahtanov@croit.io>
Change-Id: I851240a8c24c137aaf9c088ad294e96e349d5e97
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12688
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: <qun.wan@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Kamil Godzwon <kamilx.godzwon@intel.com>
2022-06-21 07:53:12 +00:00
Kamil Godzwon
e8945f926c scripts/pkgdep: adjusting packages for sles
Disabled installing markdownlint due to ruby
version issue.
Changed packages name according to packages
available from a repository.

Signed-off-by: Kamil Godzwon <kamilx.godzwon@intel.com>
Change-Id: Iea22d78dedad5386ab30d6bdd79cd7e961eaee72
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12756
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: <qun.wan@intel.com>
2022-06-21 07:53:12 +00:00
Karol Latecki
777afdf853 scripts/pkgdep.sh: install DAOS bdev build dependencies
Change-Id: I9b4032656839b5ebc5202b26b97b7882664263ab
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12393
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-06-21 07:53:12 +00:00
Alexey Marchuk
9bb64b6b6e pkgdep/debian: Fix python installation on Ubuntu 22.04
There is no 'python' package, 'python3' should be used
instead

Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: I8d55bf579038b9316e26e762b69e6c150869a66b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12804
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Michal Berger <michallinuxstuff@gmail.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2022-06-08 07:46:48 +00:00
John Levon
927f1fd57b libvfio-user: update to use meson
libvfio-user has switched from cmake to meson; update the submodule for
this, and switch over to building with meson.

Signed-off-by: John Levon <john.levon@nutanix.com>
Change-Id: Ia0749590271d0b74260c9118692c21e2a82dde23
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12640
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-05-20 09:21:46 +00:00
Konrad Sztyber
d2db395987 sma: load configuration from file
This patch adds support for loading configuration from a yaml-formatted
file.  The command-line options takes precedence over those from the
config file.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Iafb5b4aec0a64afe358fce2d6c55361fc4093223
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11712
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
2022-05-20 09:16:54 +00:00
Konrad Sztyber
509241cef3 configure: add switch for enabling SMA
The SMA requires some extra python packages during the build to generate
the protobuf interface, so this option will select whether it's enabled
or not.  Additionally, the dependencies are added to `pkgdep.sh` in this
patch.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ib0d75c86d4d8575ec1245b219b4e96edb773b366
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10974
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2022-05-18 07:24:06 +00:00
Kamil Godzwon
9cdadbea09 scripts/pkgdep: add support for Rocky Linux
Add Rocky Linux as another supported OS
Adjust package dependencies

Signed-off-by: Kamil Godzwon <kamilx.godzwon@intel.com>
Change-Id: Ide4f5075a224e9eb37cb99ae5e138ebf91e4ad5b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11994
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Michal Berger <michallinuxstuff@gmail.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
2022-05-09 09:14:36 +00:00
Boris Glimcher
8ccb3c10c0 pkgdep/centos: Add openssl11-devel for centos7
By default centos7 uses openssl-devel version 1.0.2k
The newer version 1.1.1k is available with EPEL repo

Install and use newer version 1.1 for centos7

The new 1.1 version is needed for TLSv1.3 support.

Change-Id: I489f38dd519276741ab0bb2cf7bc2d368a16387f
Signed-off-by: Boris Glimcher <Boris.Glimcher@emc.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12511
Reviewed-by: Michal Berger <michallinuxstuff@gmail.com>
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-05-06 08:12:42 +00:00
Kamil Godzwon
ac7b9b7c66 pkgdep/centos: fix centos routines for installing the packages
Centos7: ndb package version update - version 3.14-2
of the nbd package is no longer available in
epel repository.
Updated for version 3.24-1.

Centos8: repository URL change - old ceph
repository has Centos version mismatch.

Centos8: no need to install NBD package. It is available
on the system by default.

Signed-off-by: Kamil Godzwon <kamilx.godzwon@intel.com>
Change-Id: Ifae0833822b270127b31fe95c14f6803f688bfc6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12156
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-04-26 07:47:27 +00:00
Konrad Sztyber
f6dbaa0686 pkgdep: add python3-devel dependency
The following patch will need it to expand the %{python3_sitelib} macros
when building the RPMs.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I2782b228066fa0524d675c8d964217a7c9d11e36
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11656
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2022-03-08 07:54:23 +00:00
Josh Soref
1ff3715d38 spelling: scripts
Part of #2256

* available
* classes
* coalesce
* configuration
* controller
* currently
* dependencies
* different
* displaying
* does not
* environment
* explicitly
* hugepages
* ignoring
* inflight
* initialize
* parameters
* priority
* properties
* recovery
* subsystem
* subsystems
* template
* the same
* underlying
* usable
* values

Change-Id: Ibc8567af288c9f4641563835e16bf88949ba6a71
Signed-off-by: Josh Soref <jsoref@gmail.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10408
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-12-03 08:13:04 +00:00
Michal Berger
7e0fc66786 scripts/pkgdep: Disable mdl installation on centos
Versions of centos we support - 7 and 8 - don't ship with ruby which
can handle installation of the mdl gem due to unmet dependencies -
it's simply too old.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Id0a3843bcab12b8de332d1d3f64956f043c347e3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9483
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2021-11-22 08:34:00 +00:00
Richael Zhuang
fe09e7d494 scripts/pkgdep: support shfmt for arm64
Get shfmt package according to arch.

Change-Id: Ib51feb64368e1ef559c886d9899dd7a1f28cff95
Signed-off-by: Richael Zhuang <richael.zhuang@arm.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9996
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-11-22 08:33:29 +00:00
Konrad Sztyber
5d5d9cbb3e scripts/trace: use trace library for parsing traces
Added trace provider based on the trace library.  It uses ctypes to
define the trace objects and execute library calls.  It significantly
(~2x) speeds up the execution time on large trace files.

The script automatically detects the type of the trace file and will use
the appropriate engine (either JSON or the trace library) using
python-magic for file detection.  When piping a file through stdin, JSON
format will be assumed.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Icfa216edee4662714266968eeb72f0097b3779b4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9439
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-10-20 07:22:00 +00:00
Michal Berger
1670d1ddf1 scripts/pkgdep: Add Software Collections repo to centos7
This one provides devtoolset packages which can be used for adding
alternatives for some older packages like gcc for instance. Useful
for the CI to bring newer stuff into VMs.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Ia82d712e670177f37256a999f5dc2a07784d2218
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9500
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2021-09-21 07:52:48 +00:00
Michal Berger
e2faef472b pkgdep: Install liburing regardless if it's in the ld cache
Some distros may already provide liburing (e.g. fedora34) as a
package. In case given system has it installed, without the devel
portion, we would skip building our own version leaving system
without proper include files needed for the tests.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I7b07ba8d1c3c6b97d6b114935dfe472cc7902bf6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9543
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-09-20 10:49:19 +00:00
Michal Berger
095df6b25a pkgdep: Copy the bash-completion file instead of creating a symlink
Symlink requires repo to be around, having a hard copy allows it to
be used standalone.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: If5aa354b10762805394418e9c9c0d171623ba093
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8446
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
2021-09-03 08:08:28 +00:00
Michal Berger
3220a05311 pkgdep: Make sure gem is installed on FreeBSD prior installing mdl
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I2ce2ce7294eff1c0cd66a5bf101f37d64ee4645d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9165
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-08-24 07:18:44 +00:00
Maciej Wawryk
6792ae70f1 scripts/pkgdep: add ruby-devel to INSTALL_DEV_TOOLS
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I367b42a84f3af4ca456ea8c2e1263a05a92bbae7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9145
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-08-19 11:24:07 +00:00
Maciej Wawryk
83440f1ded scripts/pkgdep: move markdownlint install to pkgdep scripts
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: If2cc3cf797602f90b26407d35f64840bb2d2697c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9108
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
2021-08-10 08:13:00 +00:00
Ziye Yang
6434aaa274 scripts/pkgdep: Add the libpmem object library dependency.
Since we will create new bdev relied on libpmem object
library, so need add it.

Change-Id: I7ddcc694fd2ec5a0536abc9813b13fa429fcf068
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9049
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-08-04 11:05:30 +00:00
Karol Latecki
84867c8eaf scripts: remove gdb from Python requirements file
This package is no longer available in PyPI.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I3e68b03eabeb4aff14453c90846fd03452c93ae2
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8956
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-07-28 16:19:35 +00:00
Michal Berger
1963d3185c pkgdep/freebsd: Use globbing to match on package names
Some packages include versioning in their name which is not very
friendly in case given package is upgraded - the name of the package
will always change in such a case. To avoid that, use globbing to
match on package names in a more robust way.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I3e23fc8a06f93471f47835e945c24c4a95f80795
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8741
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-07-20 08:09:36 +00:00
Konrad Sztyber
597688b2b1 scripts/trace: use ijson to parse the traces
Since the trace files can get very large (several GBs), parsing them
using python's json module might require an unfeasible amount of memory,
as it needs to load the whole file first.  The ijson [1] library
provides interfaces for parsing files iteratively, only loading a small
portion of a file at a time.

It requires the input JSON to have the tsc_rate and the definitions of
the tracepoints listed before the tracepoint entries.  It's not a big
deal, as this is the way `spdk_trace` generates it, but it's worth
noting, as passing that file through something like `jq -S` might make
it unreadable to the trace script.

[1] https://pypi.org/project/ijson

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I03c0c3fb47091da615a3978b8d63edf4d876b811
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8275
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
2021-07-07 09:43:37 +00:00
Jim Harris
5a1c74bfa4 usdt: add User Space DTrace support to SPDK
For now, we will keep this disabled by default,
enable with --with-usdt option to the configure
script.  Long-term we will want to enable this by
default, and only disable via configure.

Modules can include spdk_internal/usdt.h and add
probes such as:

SPDK_DTRACE_PROBE2(probe_name, ptr, val);

When USDT is enabled, these will translate to
DTRACE_PROBE2(spdk, probe_name, ptr, val).  When
USDT is disabled, these will translate to nothing.

Later patches will add some probe points to the
nvmf target, some bpftrace scripts, and instructions
for how to successfully capture data with these
probe points and scripts.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Id168e2c800fa5522815a175026386319014cfdaa
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7173
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2021-05-21 08:03:27 +00:00
Michal Berger
d97b106f81 pkgdep: Update liburing version to 2.0 release
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I239c6eb2e9919a4605a14d39f4462b3aa5f54883
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7496
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
2021-05-05 13:27:16 +00:00
Michal Berger
3a1d4786f4 pkgdep/common: Add /usr/lib64 to ld's cache
ld on some distros, e.g., ubuntu1604, doesn't include this particular
path in its SEARCH_DIR list. Add it manually to make sure linking
towards liburing is possible in such a case.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I82d7ed870ec9ff257dc72a386f950addb5476d1b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7022
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2021-04-19 14:15:38 +00:00
Michal Berger
360fd2cc5a pkgdep/rhel: Fix typo in alternatives path
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I6d3b7c6855899e3a9d355368c41e4d585db7f382
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6767
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-03-18 14:36:44 +00:00
Michal Berger
ab7ae487a6 pkgdep: Add bash-completion package to DEV_TOOLS
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I7fe3baa15327b9ff509185ff6aa30390e35e9f8c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6239
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
2021-02-26 12:24:56 +00:00
Michal Berger
83adcc14c9 scripts/pkgdep: Remove kernel-modules-extra as FUSE dependency
Rationale behind e663dc82ad which put this package as a dependency
into pkgdep was that it shouldn't be exclusive to vm_setup.sh.

However, this particular package causes unnecessary kernel upgrades
whenever newer version is detected by dnf|yum. This behavior is out of
pkgdep's scope and as such similar upgrades should be performed via
vm_setup.sh only where its sole purpose is to configure system on a
wider spectrum.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I066105f95834f3f8d436b9ff52ec1f2bf97844af
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6437
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-02-22 10:04:38 +00:00
Michal Berger
439688a97e scripts/pkgdep: Update lib cache after installing liburing
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I8097b1db08f951e8240248b585c8aeccc7893d49
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6435
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-02-22 10:04:38 +00:00
Michal Berger
62ab898c8f pkgdep/ubuntu: Include packaged version of meson
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Ib6bff3b5d77d5cc0a08d11e9608c10bf0e4d119f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6412
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2021-02-19 11:30:43 +00:00
Karol Latecki
738cdf143f scripts/pkgdep.sh: install pyelftools for DPDK
Recent builds with mainline DPDK failed due to missing
python dependency "elftools". This patch adds it to
our system provisioning scripts.

Fixes #1770

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I9e3879d9bddd64bb0f7be81120e5fadebefa9eb8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6103
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2021-02-04 08:43:05 +00:00
Michal Berger
2212610cd1 pgkdep/freebsd: Use py37-pycodestyle
py27-pycodestyle was dropped and is no longer available for the
amd64 arch. Use py37-pycodestyle instead (it should be available for
other common architectures too).

Change-Id: Icb56860d358e2fa06f202a47302a5005b32708e3
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5877
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
2021-01-13 10:07:31 +00:00
Michal Berger
3edb26fb0e pkgdep/rhel: Check if given repos can be enabled
In recent Centos8 updates name of the "PowerTools" repo was changed to
"powertools". Since config-manager will fail if we provide a faulty
name of the repo, we need to make sure we tell it to enable what's
actually present on the system.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Iff0c6103d1375fc42f5c967cd713e3e5089f4723
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5478
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2020-12-11 11:07:51 +00:00
Changpeng Liu
0c55825781 pkgdep: install json-c-devel and libcmocka-devel
The coming libvfio-user needs json-c and libcmocka library, so install
the libraries in CI environment.

Change-Id: Ib4871169a4d5b140cda11a2cc449201c963a9f12
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5287
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-12-10 11:16:34 +00:00
Tomasz Zawadzki
e663dc82ad pkgdep: install cuse module as dependency for nvme-cuse
cuse kernel module is not required to build nvme-cuse,
but any usage still requires it.

Packages for that were part of vm_setup script,
but should not be exclusive to it.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ia105454f65ee87ab3f56580c43d33214beac7b2a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5453
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
2020-12-08 13:11:49 +00:00