This patch adds support for --max-lcores configuration
option to ./configure script. This option can be
used to change value of DPDK's RTE_MAX_LCORE
(which is by default set to 128 for x86 architecture).
If specified, DPDK will be configured to use
the value provided by the user instead of
the default one. The option can be useful
in systems where number of physical CPUs is
larger than 128.
When RTE_MAX_LCORE is increased, it is possible
to specify cores with identifiers larger than
128 in the SPDK's CPU mask.
If the option is not specifed, DPDK will use
default value of RTE_MAX_LCORE.
--max-lcores range is [1..1024]
Example usage:
./configure --max-lcores=256
./configure --max-lcores=16
Change-Id: I47d321ba394c9acf27eaa91619aeaad28db6de34
Signed-off-by: Marcin Spiewak <marcin.spiewak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17453
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Community-CI: Mellanox Build Bot
Reduce build time and size by passing -Ddisable_apps.
This option will be used instead of pushing a backport
to DPDK fork that would manually disable app building.
Tested on DPDK 22.07 and DPDK 22.11.1.
Change-Id: I94df5550315567f6dbd72d354f733fe5126c5930
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15929
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Currently those are being sed-ed as a part of the meson setup
command, which elongates that line to over 300 characters.
To reduce size of that command, move sed-ing out to
variables and use only those as part of meson setup
command.
Change-Id: I5f8900d43b8d1f929a68d0e3a3ded962a6f8d1e2
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16533
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
The version of make found in Rocky 8.7 (make-4.2.1-11.el8.x86_64) fails
in the dpdkbuild directory with:
$ make clean
Makefile:64: *** invalid syntax in conditional. Stop.
This removes the # symbol, allowing builds to work again.
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: I0b9ac05d6670615c2a3642c7a833ac64774b9fef
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16440
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
intel-ipsec-mb added NO_COMPAT_IMB_API_053 to allow
for backwards compatiblity for certain enums.
See intel-ipsec-mb patch:
(53a8371)Document and use new compilation flag NO_COMPAT_IMB_API_053
Meanwhile starting with DPDK 22.07, qat crypto updated the required intel-ipsec-mb
and disabled OPENSSL_API_COMPAT. Resulting in redefinitions from aes.h.
Patch breaking it:
(3227bc7)crypto/qat: use intel-ipsec-mb for partial hash and AES
Only _after_ DPDK 22.11 it was fixed:
(2a21107) crypto/qat: fix build
This means that DPDK 22.07 and DPDK 22.11.1 does not work with latest intel-ipsec-mb.
It does not affect earlier versions of DPDK, nor intel-ipsec-mb.
Only for this case, if the DPDK fix is not present,
define NO_COMPAT_IMB_API_053.
Change-Id: Ibe2edc5c79ac09b5260eeb0f7d53b4dcb26bcaa5
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16359
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This is the port of the vbdev compress logic into the accel
framework. It includes just one enhancement, to only fill each
mbuf in either src or dst array with max "window size" param to
avoid QAT errors. Note that DPDK ISAL PMD was not ported as we
have native ISAL compression in accel now.
Note: ISAL w/DPDK is still built w/this patch, that can't be
removed until the vbdev module moves to accel fw as it still
depends on DPDK ISAL PMD.
Follow-on patches will include addition C API for PMD selection,
this patch just gets equivalent functionality going. Upcoming
patches will also convert the vbdev compress module to use the
accel framework instead of talking directly to compressdev.
More patches will also address comments on vbdev common code
that addressed here would make the review challenging.
This patch also fixes a bug in the ported code that needs to
be fixed here to pass CI. Capability discovery was incorrect
causing all devices to appear to not support chained mbufs,
with the mbuf splitting code this is important to get right.
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I7f526404819b145ef26e40877122ba80a02fcf51
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15178
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Always add -larchive to DPDK static link args if
libarchive is available. This is less fragile than
previous mechanism of trying to remove
RTE_HAS_LIBARCHIVE to keep DPDK from trying to use it.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ib26fc204927d8967b98d416373fc91446169d5af
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15951
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Previously we would only set this if we detected that
the DPDK specified using --with-dpdk required it. When
using the DPDK submodule, we would always strip
RTE_USE_LIBBSD to keep DPDK from trying to use it.
This is fragile and unnecessarily complex though - if
libbsd exists, just always add -lbsd to the link line.
Next patch will do the same for libarchive.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I94dd792b392c23a705dead47947b33d899db8f4f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15950
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: John Levon <levon@movementarian.org>
This is in prep for adding a new compressDev accel_fw
module that will contain all of the DPDK compressDev specifics
on it, the vbdev will make calls to the accel_fw instead.
As the accel_fw has SW based compression, we want the configure
option to apply to building the vbdev module but not the accel_sw
software implementation or the upcoming compressdev module.
Renamed to "compress" as reduce is a term specific to the vbdev
implementation of the compression to be provided by the accel_fw
and thus the same reason why we leave the test flag called REDUCE
because it's controlling tests for the reduce library as well as
the vbdev module that is using reduce. The flag does not apply
to the SW implementation of compression.
This does not affect upcoming accel_fw compressdev module, that
will have its own configure option.
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: If8ed3e48e1e3dabcaad1cd161289e78122cd9d58
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15179
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
per Intel policy to include file commit date using git cmd
below. The policy does not apply to non-Intel (C) notices.
git log --follow -C90% --format=%ad --date default <file> | tail -1
and then pull just the 4 digit year from the result.
Intel copyrights were not added to files where Intel either had
no contribution ot the contribution lacked substance (ie license
header updates, formatting changes, etc). Contribution date used
"--follow -C95%" to get the most accurate date.
Note that several files in this patch didn't end the license/(c)
block with a blank comment line so these were added as the vast
majority of files do have this last blank line. Simply there for
consistency.
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Id5b7ce4f658fe87132f14139ead58d6e285c04d4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15192
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
When user specifies SPDK LTO build, pass the
associated option to the DPDK build to enable
LTO there as well.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I5996ae0668ad00c52fcaeb28db055af8dc85a8ca
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14389
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
This is known as a false positive of GCC12.
For present, we can avoid it using Wno-array-bounds
to go on.
fixes issue #2668.
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Change-Id: I99551348c33292fc2352a570d88f706661c8b9b2
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14281
Reviewed-by: GangCao <gang.cao@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Kamil Godzwon <kamilx.godzwon@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This library can't be compiled on aarch, so disable it
Signed-off-by: Aleksey Marchuk <alexeymar@nvidia.com>
Change-Id: I3a8ab2f8d95c122b45f6211a1ce4916389728476
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14288
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
gcc 12 reports reading incorect size from a region.
Seems like false positive, see issue #2460.
This is a temporary workaround until this is resolved
in DPDK or gcc.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ie7b5e7c20ae8523e4d02f58868d1eeaa8f873dc1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13405
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Many open source projects have moved to using SPDX identifiers
to specify license information, reducing the amount of
boilerplate code in every source file. This patch replaces
the bulk of SPDK .c, .cpp and Makefiles with the BSD-3-Clause
identifier.
Almost all of these files share the exact same license text,
and this patch only modifies the files that contain the
most common license text. There can be slight variations
because the third clause contains company names - most say
"Intel Corporation", but there are instances for Nvidia,
Samsung, Eideticom and even "the copyright holder".
Used a bash script to automate replacement of the license text
with SPDX identifier which is checked into scripts/spdx.sh.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iaa88ab5e92ea471691dc298cfe41ebfb5d169780
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12904
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: <qun.wan@intel.com>
Some environments (such as a VM on my M1-based
MacBook Pro) report CPU with an Implementer+Part Number
that DPDK doesn't explicitly support. In my case, it
is Implementer ARM (0x41) but no part number (0x0).
DPDK doesn't just default to a generic build in this
case, it requires the user to specify
-Dplatform=generic.
We could try to plumb this into a configure option,
but for now I'm fine in my dev environment to just
specify this DPDKBUILD_FLAGS in my environment to
work around it.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I0465e3747de9b1ebacaf4ca53c2e6ccf36db1091
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12670
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
DPDK reports that -Dmachine is deprecated, we should
use -Dcpu_instruction_set instead.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I1dbd81fe9f34d875bf00c0fa3a7868a8fc30cac3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12669
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Additionally add dmadev to vhost dependencies,
since it is needed as of commit
53d3f4778c1dbdebbef6c5e94e772b24f0998d35 in DPDK.
Change-Id: Ifd9cac8ef9c50b52da815d43869c8291aa72a326
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12638
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
- Added build system logic for checking mlx5 crypto PMD support. Added
required libs in make files.
- Changes in mlx5 reduce build system logic since both crypto and reduce
use common libs and libmlx5 related checks.
- Both mlx5 crypto and reduce require -libverbs.
Signed-off-by: Yuriy Umanets <yumanets@nvidia.com>
Change-Id: Ice1b88fe74bb04bf715ecaac70c4a8d4f3b5d782
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11620
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
DPDK rte_eal_firmware.c will use libarchive APIs when
libarchive is installed on the system. SPDK doesn't
currently account for this, so if the user has
libarchive installed, DPDK will emit libarchive calls
that don't get resolved during link phase for static
builds.
We've had a similar situation with libbsd for a long
time, where we remove the associated #define from the
DPDK build config header file before we kick off the
DPDK build. So extend that here for the
RTE_HAS_LIBARCHIVE #define.
It would be nicer to use pkg-config somehow to
generate the necessarily libraries to install on the
SPDK link line for static builds. But it's not
exactly straightforward, so this solution ended up
being the simplest by far, extending a mechanism
that's worked well with SPDK's dpdkbuild up until
this point.
Fixes issue #2357.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I79ff4648ee58a3e13f4b873f8acd1b31cca0fc31
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11385
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Most DPDK libraries are not required for SPDK.
Those can be disabled for submodule.
Starting with patch https://review.spdk.io/gerrit/c/spdk/dpdk/+/10540
it is possible to not compile a lot of the DPDK libs.
This reduces the build time for DPDK submodule.
Historically it was done on DPDK submodule side:
https://review.spdk.io/gerrit/c/spdk/dpdk/+/2578
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I4510baf2773fe15835e705bad42cf3ba9f35c418
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10506
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Since DPDK 21.05 -Denable_drivers option is present,
which only enables selected drivers in DPDK.
As name would suggest it is exact oposite of -Ddisable_drivers.
When building DPDK submodule all unused drivers were disabled
anyway.
Simplify this process by using the new option.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ic8ac84b1d6dc88dd7c0b8c4aca391f31bfc93404
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10505
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
DPDK tests were usualy disabled in the DPDK submodule,
as part of skipping dpdk/app directory.
Since DPDK 18.02 -Dtests=false option was added to
skip building tests in dpdk/app/test.
This reduces the overall time taken to build DPDK from
submodule without introducing new changes to DPDK.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I425976fa38e09c140e517cccd8aeedd64c67b06c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10504
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
crypto/aesni_mb is renamed to crypto/ipsec_mb,
starting with DPDK 21.11.
Patch below remedied most of the changes required:
(089cbda8)autobuild: Adjusting crypto build for latest DPDK (21.11)
Yet it missed changing the driver name when using submodule.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Iec23d35e2a64ddab975b394a23e72af1a11c7b6e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10453
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Update submodule DPDK to version 21.08 and modify
CHANGELOG.
Added bus auxiliary dependencies to dpdkbuild/Makefile
and lib/env_dpdk/env.mk. This dependency was introduced
in DPDK 21.08.
Change-Id: I72d9fde456583dc129f4c7fced4f10875bbc38e2
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9211
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
This PMD is availabe for BlueField2 DPU.
It requires libmlx5, so configure file is
updated to check if this library exists
Change-Id: Ic0cfbfdf24af393381667435009fb6afc49d9181
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8780
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
In latest 13.0 release of freebsd sed is not accepting \s anymore,
hence it needs to be replaced. Note that only parts of the repo
which are touched by freebsd during the tests are updated.
Details https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=253893
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I217208511af8f98e7033f8f24a343b5ca9c48825
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7986
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Default to the more common Linux/GNU 'sed' edit in-place argument,
unless the platform is FreeBSD.
Tested by visually checking rte_build_config.h to confirm that the sed
in-place worked correctly.
Signed-off-by: Nick Connolly <nick.connolly@mayadata.io>
Change-Id: I68be69658930fb20318ac3aa2413bbf4a358e9bc
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6531
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
When ./configure --enable-debug is specified, meson is invoked using
the default 'release' buildtype and '-O0 -g' is added to DPDK_CFLAGS.
Instead, specify --buildtype=debug so that meson knows it is a debug
build, will return the correct value from get_option('buildtype') and
can choose the appropriate toolset options to enable symbolic debugging.
Using --buildtype=debug generates unoptimised code which matches
the current intent.
Tested by building with and without --enable-debug and verifying that
meson reports 'debug' for the debug build.
Signed-off-by: Nick Connolly <nick.connolly@mayadata.io>
Change-Id: Iabb79cd2051145e03fea8fd749cfb18b78e625a0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6497
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
If CONFIG_CROSS_PREFIX includes 'mingw', then specify --cross-file
'config/x86/cross-mingw' which is used as part of regular DPDK
testing, so is being maintained. For any other non-null prefix, default
to the current error message that automatic cross builds are not
supported.
Tested by running ./configure --cross-prefix=x86_64-w64-mingw32
and verified that 'make' in dpdkbuild used the cross tools.
Signed-off-by: Nick Connolly <nick.connolly@mayadata.io>
Change-Id: I75c401dfe8422f6c5f1bbe631695e7ae6118f723
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6530
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
DPDK 20.11 moved the kernel modules to separate
dpdk-kmod repository. It has to be built separate
from DPDK.
If needed for testing vm_setup.sh script now contains option
to build this driver.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I98a5eb956eb0cc60ec402d88fcdbd66d4854f19a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6033
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Intel Control-flow Enforcement Technology (CET) is a
processor feature that blocks return/jump-oriented
programming (ROP) attacks.
It is currently only supported on Tiger Lake client
processors, but will be available on other processors
in the future.
CET requires toolchain support. gcc8 does support it.
For now, this will opt-in only at least until the
technology is available on server processors.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I8c7f882eeeaed26484c31dc0d67d5cc42baeaa2d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5921
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
When configuring with --with-shared, you would see all
applications getting relinked, even if nothing has
changed. This was root caused to DPDK re-installing
shared libraries that hadn't been rebuilt.
Switching from 'ninja install' to 'meson install --only-changed'
fixes this behavior. Note that meson install doesn't
support -j, but at this point we are only installing
files, not building them - and now we won't even be
installing the files again if they haven't changed.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: If0491b2ad6e8564ad0fd8dee39a2e357e1329ea9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4465
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
DPDK compilation fails when we configure SPDK using --with-reduce:
../drivers/common/qat/qat_qp.c:18:10: fatal error: qat_sym.h: No such file or directory
#include "qat_sym.h"
^~~~~~~~~~~
But when SPDK is also configured using --with-crypto, DPDK is built successfully
--with-crypto enables build of crypto/qat driver and meson config for
this driver adds dpdk/driver/crypto/qat directory to QAT includes. This
directory contains the missing qat_sym.h and qat_sym_pmd.h files.
Group common drivers required by crypto/reduce
Fixes issue #1529
Change-Id: I7a53411798091e9a3c16442f3951ff73138d7337
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4179
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Makefile support in DPDK was deprecated and will be removed soon,
so switch to the officially supported way of building DPDK -
with meson and ninja. Two new tools. Basically, our Makefiles
will invoke meson+ninja for DPDK, no other SPDK components are
affected.
Apparently DPDK wanted to move away from an octopus-like config
system and the ideology behind meson configuration is simple now:
build everything by default. Some PMDs can be explicitly disabled
with meson command line, but all libraries (both static and shared
versions) and test apps are built unconditionally.
How long does it take to build minimal DPDK with meson? Too much.
On my machine half of the total build time is spent on libraries
we don't need at all. (I have some hacks up my sleeve to disable
building those libraries - see the subsequent patch.) As for the
official way of building a minimal DPDK, there was a patch [1]
on dpdk mailing list to introduce more specific configuration,
but it was rejected:
> We talked about this a few times in the past, and it was actually one
> of the design goals to _avoid_ replicating the octopus-like config
> system of the makefiles. That's because it makes the test matrix
> insanely complicated, not to mention the harm to user friendliness,
> among other things.
>
> If someone doesn't want to use a PMD, they can just avoid installing it
> - it's simple enough.
>
> Sorry, but from me it's a very strong NACK.
Let's not follow that direction, hack the DPDK build system instead.
As for advantages of meson+ninja over Makefiles? I can't find any.
It's another build system that does a lot for you with some functions,
magic options, and a built-in dependency system. It seems nice if you know
the syntax, but it's another component that you need to learn, debug,
and possibly find bugs in (there's a lot of github issues open for meson).
I would compare it to CMake.
As for changes in this patch: rather that explicitly disabling
PMDs we don't need, specify a list of PMDs we do need and disable
everything else found in ./dpdk/drivers/*. This way we won't have
to disable the new PMDs as they're added to DPDK.
Meson configuration also sets RTE_EAL_PMD_PATH #define to a valid directory
with built PMD shared libs. When it's set, DPDK dynamically loads all shared
libraries inside. The drivers there depend on DPDK shared libs and fail to
load in static SPDK builds, so we disable them altogether by unsetting
RTE_EAL_PMD_PATH in the meson-generated config file - just like
DPDK Makefiles did. EAL checks for RTE_EAL_PMD_PATH being empty and skips
loading any external PMDs then. We do it for both static and shared libs.
We specify all PMDs at build time for now, so there's just no need to load
them dynamically.
We have three more hacks in our submodule:
* disable building dpdk apps by commenting-out a line in dpdk/meson.build
* disable building unnecessary libs (build everything that spdk *may*
need)
* build isa-l compress pmd with `-L[...] -lisal`. DPDK expects to find
libisal with pkg-config. We don't want to prepare a pkg-config file,
so comment-out a failing check in another meson.build file and provide
isa-l through CFLAGS and LDFLAGS.
We also need to make some changes to our test/external_code. First of
all, -ldpdk is no more. Meson build generates a pkg-config file with all
libs, but we'll switch to it in a separate patch - for now just specify
all -lrte_ libs one by one. -Wl,--no-as-needed has to be added to some
test cases, otherwise rte_mempool_ring isn't loaded. We don't use any
APIs from this library, it only has a static constructor that provides
a few callbacks used by rte_mempool_create(). Also, since DPDK now builds
both static and shared libraries, we need to add -Wl,-Bstatic to force
using static libswhere required. It's only needed for DPDK libs, but we
use it for SPDK libs as well since there's no harm.
As for performance:
$ ./configure --enable-debug --with-crypto --with-reduce
$ time make -j40 -C dpdkbuild all
with meson:
real 0m8.287s
user 1m7.983s
sys 0m10.548s
before, with the old DPDK makefiles:
real 0m20.232s
user 0m55.921s
sys 0m16.491s
The subsequent builds are much faster too:
$ time make -j40 -C dpdkbuild all
meson:
real 0m0.876s
user 0m0.663s
sys 0m0.217s
makefiles:
real 0m10.150s
user 0m11.740s
sys 0m6.772s
[1] http://inbox.dpdk.org/dev/1a07d1cd59d84dce84e56c10fdabf5e5504560a6.camel@debian.org/
Change-Id: Ic65db563014100bafb12e61ee0530cc2ae64401d
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1440
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Suppres the following warnings which causes compilation errors:
1. gcc 10 complains on operations with zero size arrays in rte_cryptodev.c
Suppress this warning by adding -Wno-stringop-overflow compilation flag
2. gcc 10 disables fcommon by default and complains on multiple definition of
aesni_mb_logtype_driver symbol which is defined in header file and presented in sevral
translation units. Add -fcommon compilation flag.
Fixes issue #1493
Change-Id: I9241bf1fd78e86df6a6eb46b4ff787b2f7027b7d
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3373
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Compiling raid5 has a direct dependency on rte_hash,
which was only built if vhost was built.
The following didn't work:
./configure --with-raid5 --without-vhost
Change-Id: Id36a7d4a21c2e0db00b0641581542e244c4cbbb4
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1013
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This serves as cleanup. Dont build anything by default,
enable stuff if needed. Enabling a library twice doesn't
cause any issues, so drop the piece of logic preventing that.
Change-Id: Ia1877635b100f73b27d9c63ed8baed8184729f1c
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1012
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Apparently make doesn't like it:
make[2]: warning: -jN forced in submake: disabling jobserver mode.
We only did it because of scan-build - it detects a ton
of issues (or false positives) in our dependencies. To
disable it, we don't have to override MAKEFLAGS, but
just CC - that's what we'll do now.
Fixes#896
Change-Id: I5eea984d6bbfbf4caabdd704850fac840fed3524
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/927
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
It turns out that this driver is generally useful, not just within
the context of compress/crypto.
fixes GitHub issue #1189
Change-Id: Ieea8037a240f0302e45ce865632be29821f82a7e
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/792
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
When building SPDK with shared objects, we should also build the DPDK
submodule shared libraries. This will make it much easier for peole to
link their applications against a dpdk shared library if the DPDK shared
libraries installed on their system don't have all the symbols that SPDK
needs.
Also, add a linker argument to DPDK to specify that it should look in
the dpdk subdirectory when linking libraries.
Note: this change will not work until the default config options in the
DPDK submodule are updated.
Change-Id: I9546df7b84952eb20fe1ac98e3b3bbd0fcbda3a7
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/463028
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jan Kryl <jan.kryl@mayadata.io>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
We commented out this build option in our DPDK fork,
but DPDK makefiles check its value specifically against
'n' to set linking flags properly. It's currently unset,
so it fails both == 'n' and == 'y' checks, which fails
the build. Shared lib build has some issues on FreeBSD
right now, so to fix the immediate problem, just define
the shared lib option to 'n' for now.
This matches the original behavior. We've always built
DPDK as a static library.
Change-Id: I5a880b777153cc169c3a135d412aa2014f4de8e0
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/463290
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
There might be a need for user to cross-compile SPDK for other platforms
than native. Since DPDK supports cross-compilation already, only enablement of
cross-compilation for SPDK is missing. This patch aims to provide the user
with additional option to configure script that works similar to what DPDK already provides.
To enable cross-compilation run ./configure.sh with additional --cross-prefix parameter.
The parameter should work in similar manner to DPDK CROSS parameter
and specifies prefix of cross-compiler defined in PATH variable.
Note: To cross-compile, toolchain must have SPDK dependencies such as e.g. Libaio.
Signed-off-by: Amelia Blachuciak <amelia.blachuciak@intel.com>
Change-Id: Ia7cb879d39d624552cad1af98a563070ea7676b2
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/460977
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Configuring SPDK with different options might result
in enabling different CONFIG_RTE_LIB* options, but since
it does not modify any DPDK config files, DPDK doesn't
get rebuilt and keeps using older, potentially outdated
build options.
We'll now clear dpdk/build whenever SPDK config is modified,
which will result in automatically re-configuring DPDK.
Change-Id: I9c7c3434dc317e4f47767d7b3df3df48db2b0d5b
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/460563
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
