Commit Graph

116 Commits

Author SHA1 Message Date
Marcin Spiewak
2903189c40 lib/accel: removed references to ACCEL_FLAG_PERSISTENT
- Removed ACCEL_FLAG_PERSISTENT and related code

Change-Id: Icc8890ad7143fab4bd4b544e82acdf9b84c0a1eb
Signed-off-by: Marcin Spiewak <marcin.spiewak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16951
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2023-03-03 11:20:03 +00:00
Tomasz Zawadzki
3359bf34d6 so_ver: increase all major versions
To allow SO_MINOR updates on LTS for the whole year it is supported,
the major version for all components needs to be increased.
This is to prevent scenario where two versions exists with matching
versions, but conflicting ABI.
Ex. Next SPDK release adds an API call increasing the minor version,
then LTS needs just a subset of those additions.

Increasing major so version after LTS, allows the future releases
to update versions as needed. Yet allowing LTS to increase minor
version separately.

Disabled test for increasing SO version without ABI change, as
that is goal of this patch. This check shall be removed with SPDK 23.05
release.
Looks like this was left over from prior LTS, to avoid that
make sure it is only skipped when running against v23.01.x as latest
release.

This patch:
- increases SO_VER by 1 for all components
- resets SO_MINOR to 0 for all components
- removes suppressions for ABI tests


Short reference to how the versions were changed:
MAX=$(git grep "SO_VER := " | cut -d" " -f 3 | sort -ubnr | head -1)
for((i=$MAX;i>0;i-=1)); do find . -name "Makefile" -exec \
	sed -i -e "s/SO_VER := $i\$/SO_VER := $(($i+1))/g" {} +;  done
find . -name "Makefile" -exec \
	sed -i -e "s/SO_MINOR := .*/SO_MINOR := 0/g" {} +

Change-Id: I3e5681802c0a5ac6d7d652a18896997cd07cc8bf
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16419
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2023-01-24 08:37:21 +00:00
Alexey Marchuk
52c86e21be accel/dpdk_cryptodev: Remove missed noticelog
This was added during debug/development and
was not deleted by accident.

Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: I146748151623a3ea4808c7b5cc34085d30ca81b4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16388
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2023-01-20 15:40:03 +00:00
Alexey Marchuk
a1dfa7ec92 module/accel: Add mlx5 accel module
The mlx5 accel module supports crypto operations.
Data buffer is split into `block_size` chunks and each
chunk is enrypted individually.
mlx5 library contains some utility functions that will
later be used by other libraries, this lib will be
exntended later.

Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: Iacdd8caaade477277d5a95cfd53e9910e280a73b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15420
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2023-01-19 22:00:58 +00:00
Alexey Marchuk
13f97e6737 bdev/crypto: Use accel framework
All DPDK related code is removed, handling of
RESET command was sligthly updated.
Handling of -ENOMEM was updated for cases when
accel API returns -ENOMEM

Crypto tests in blockdev.sh were extended with more
crypto_bdevs to verify NOMEM cases - that failed
with original vbdev_crypto implementation

Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: If1feba2449bee852c6c4daca4b3406414db6fded
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14860
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2023-01-19 22:00:58 +00:00
Konrad Sztyber
dcce5fcf5c accel/dpdk_compressdev: remove handling for non-iovec dst
Handling for scalar dst buffers were removed in 1866faffe, and all
buffers are now described by iovecs.  The dst field was removed from
spdk_accel_task, so we also need to remove any references to that field
from dpdk_compressdev.  This patch fixes the build with compressdev
enabled.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I21fbbb45a15e7250944753e445dede7d586bf35b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16349
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Michal Berger <michal.berger@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2023-01-19 13:39:27 +00:00
paul luse
91f3063b14 lib/accel: add output_size to decompress API
We had it for compress but simply didn't think of a use case for
decompress.  During the develpoment of the compressdev accel_fw
module it was discovered that compressdev does indeed provide the
uncompressed length on completion of decompress and the reducelib
uses it.  So, add it here.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I2f6a8bbbe3ef8ebe0b50d6434845f405afa7d37d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16035
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2023-01-19 11:16:01 +00:00
paul luse
8222c5bae3 module/accel/dpdk_compressdev: change device_qp association
From an earlier comment on the original patch that introduced the
module.  Stop using the thread to associate the device_qp and
simply link it into the chan.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Ib55979247d9bdacdc868bdb9dea97e1941e1e8ab
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15832
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2023-01-19 11:16:01 +00:00
paul luse
976f8b0992 module/accel: Add compressDev accel_module
This is the port of the vbdev compress logic into the accel
framework.  It includes just one enhancement, to only fill each
mbuf in either src or dst array with max "window size" param to
avoid QAT errors. Note that DPDK ISAL PMD was not ported as we
have native ISAL compression in accel now.

Note: ISAL w/DPDK is still built w/this patch, that can't be
removed until the vbdev module moves to accel fw as it still
depends on DPDK ISAL PMD.

Follow-on patches will include addition C API for PMD selection,
this patch just gets equivalent functionality going.  Upcoming
patches will also convert the vbdev compress module to use the
accel framework instead of talking directly to compressdev.

More patches will also address comments on vbdev common code
that addressed here would make the review challenging.

This patch also fixes a bug in the ported code that needs to
be fixed here to pass CI.  Capability discovery was incorrect
causing all devices to appear to not support chained mbufs,
with the mbuf splitting code this is important to get right.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I7f526404819b145ef26e40877122ba80a02fcf51
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15178
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2023-01-19 11:16:01 +00:00
Jim Harris
05e7a8a8bf accel/dpdk_compressdev: start with copy of C file
We will be creating a new accel module for DPDK
compressdev.  We can heavily leverage all of the
code from the existing vbdev_compress.c file.  So
just copy that file exactly to start, and then the
next patch will contain only the deltas for removing
the bdev/reduce plumbing and adding the accel
plumbing.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I4f6ca1c2dad92eb8dd38b351602b65b8cb4ee4e8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15829
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Community-CI: Mellanox Build Bot
2023-01-19 11:16:01 +00:00
Alexey Marchuk
6da9878643 accel/dpdk_cryptodev: Fix io_dev unregister
It was found that if system has no crypto devs,
we exit earlier from init function and don't register
io_device. Later in fini function we may try to unregister
invalid io_device

Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: I23ac9674c02d4fe97f01c66928a5ec5a2b39ec47
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16321
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2023-01-19 08:25:48 +00:00
Alexey Marchuk
10dcf2dbd2 accel/dpdk_cryptodev: Update rte_cryptodev usage for DPDK 22.11
This patch is a combination of commits which update vdev_crypto:

110d8411e bdev/crypto: do not create mempool for session private data
495055b05 bdev/crypto: update rte_cryptodev usage for DPDK 22.11
02caed6b5 bdev/crypto: remove mempool usage matching < DPDK 19.02
5887eb321 bdev/crypto: do not track type of crypto session

Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: I30c4f76e4e7b4865a7daa638d357888bb5e02071
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16039
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2023-01-19 08:25:48 +00:00
Alexey Marchuk
8f36853a84 dpdk_cryptodev: Check queue capacity before submitting a task
When we submit more tasks than supported by qp,
extra tasks are queued on io_channel. Later completion
poller tries to resubmit these tasks one by one. That
is not efficient since every enqueu_burst may cause
doorbell updates in HW.

Instead add a check for qpir capacity and submit
appropriate number of requests. If qpair is full,
tasks are queued in dedicated list. This approach
should remove or minimize the need to resubmit
individual crypto operations.

This also handles a case where there are no entries
in global pools (crypto_ops or rte_mbuf)

Fixes issue #2756

Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: Iab50e623e7a82a4f5bef7a1e4434e593240ab633
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15769
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2023-01-18 18:19:50 +00:00
Alexey Marchuk
bf8e0656e8 dpdk_cryptodev: Remove limit on max IO size
Previously vbdev_crypto used DPDK directly and
the restriction on max IO size was propagated to
generic bdev layer which split big IO requests.

Now, when DPDK code is a standalone accel module,
this restriction on max IO size is not visible to
the user and we should get rid of it.

To remove this limitation, allow to submit crypto
operations for part of logical blocks in big IO,
the rest blocks will be processed when all submitted
crypto ops are completed.

To verify this patch, add a functional test which
submits big IO verify mode

Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: I0ee89e98195a5c744f3fb2bfc752b578965c3bc5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15768
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2023-01-18 18:19:50 +00:00
Konrad Sztyber
957076108f accel: remove nbytes from spdk_accel_task
All operations are using iovecs to describe their buffers and only
encrypt/decrypt additionally used nbytes to store the total size of a
src buffer.  We don't really need this value in the generic accel code,
so we can let modules calculate it, if necessary.  That way, we won't
waste cycles calculating it if a module doesn't use it and it makes the
code a bit easier, as we won't have to deal with the fact that nbytes is
only valid for certain operations.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I29252be34a9af9fd40f4c7fec9d0a0c1139c562d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16306
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2023-01-17 23:34:43 +00:00
Konrad Sztyber
1866faffe2 accel: use iovecs for compress operations
Also, since this was the last operation using dst and nbytes, these
fields were removed from spdk_accel_task.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I0d6b090e101c016d1bdcbe7a3bee7d6f691f1c9e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15943
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2023-01-17 23:34:43 +00:00
Konrad Sztyber
a374f8ba19 accel: use iovecs for copy+crc32c operations
Also, since this was the last operation using src, remove this field
from spdk_accel_task.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I55fd98697ef4f92a13dd0563b4adf9ccb0af171b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15942
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2023-01-17 23:34:43 +00:00
Konrad Sztyber
3d1d5452e0 accel: use iovecs for crc32c operations
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ic9f1f002edf273e9cd2247f353b5d7de9d2dea05
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15941
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2023-01-16 15:35:15 +00:00
Konrad Sztyber
bc6a14636a accel: use iovecs for fill operations
Also, make it possible to remove copy operations following a fill
operation if they're using the same buffers.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I7da195ce80650a02c5db99d9400ee692f797b1f8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15940
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2023-01-16 15:35:15 +00:00
Konrad Sztyber
4d1ba5f294 accel: use iovecs for compare operations
Also, replace src2 with an iovec + iovcnt and rename it to s2 to
keep the naming consistent with the source buffer (s).

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I44787128377addd514818ec5aaec084b1a31f0c3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15939
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2023-01-16 15:35:15 +00:00
Konrad Sztyber
135396b0bc accel: use iovecs for dualcast operations
Also, replace dst2 with an iovec + iovcnt and rename it to d2 to
keep the naming consistent with the destination buffer (d).

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ib394c127eeb5890451535ff485f96f7edd2897a4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15938
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2023-01-16 15:35:15 +00:00
Konrad Sztyber
dee8e1f4c0 accel: use iovecs for copy operations
This patch is first in the series of patches aimed to make all accel
operations describe their buffers with iovecs.  The intention is to make
it easier to handle tasks in a generic way.

It doesn't mean that we change the API - all function signatures are
preserved.  If a function doesn't use iovecs, we use the aux_iovs array.
However, this does mean that each accel module that provides support for
a given operation will need to be adjusted to use iovecs.

Additionally, update the unit test checking copy elision to verify the
buffers of the copy operation that is left.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I9e6d8d1be3b8b9706cb4a6222dad30e8c373d8fb
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15937
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2023-01-16 15:35:15 +00:00
Alexey Marchuk
61fbb000fe module/accel: Add accel_dpdk_cryptodev
The new module replaces functionality in vbdev_crypto.
This module is bdev agnostic, so some inernal parts
were reworked.

io_channel: contains a qp of every configured DPDK PMD
crypto key: for mlx5_pci we register a key on each available
device since keys are bound to Protection Domain.

Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: If1845cb87eadacbb921c593ba82207a97f2209a3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14859
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
2023-01-11 09:16:59 +00:00
Alexey Marchuk
f5d1a924a1 accel/dpdk_cryptodev: start with copy of vbdev_crypto
This patch is just a copy of vbdev_crypto.c and the
corresponding UT file. It makes it easier to review
the next patch which adds accel operations

Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: Ib88b45d573b011b1acb35da9bf4dab922d8fb183
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16182
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2023-01-11 09:16:59 +00:00
paul luse
a6dbe3721e update Intel copyright notices
per Intel policy to include file commit date using git cmd
below.  The policy does not apply to non-Intel (C) notices.

git log --follow -C90% --format=%ad --date default <file> | tail -1

and then pull just the 4 digit year from the result.

Intel copyrights were not added to files where Intel either had
no contribution ot the contribution lacked substance (ie license
header updates, formatting changes, etc).  Contribution date used
"--follow -C95%" to get the most accurate date.

Note that several files in this patch didn't end the license/(c)
block with a blank comment line so these were added as the vast
majority of files do have this last blank line.  Simply there for
consistency.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Id5b7ce4f658fe87132f14139ead58d6e285c04d4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15192
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
2022-11-10 08:28:53 +00:00
paul luse
850cd90082 accel/idxd/iaa: Convert to use iovecs
In prep for upcoming iovec based compression/decompression patches.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I413493f764bead9e56266e488b74f8bca979e225
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14633
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-09-23 00:10:08 +00:00
paul luse
28886ac352 lib/accel: rename iovec elements with src prefix
In prep for adding both src and dst iovec support for compression.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I704b8d2bd459de03deb7f8ee45d76261910a3727
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13746
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-09-23 00:10:08 +00:00
Ben Walker
712e8cb7ef accel: Refer to plugins as 'modules' instead of 'engines'
This is consistent with the use of terms in other parts of SPDK and fits
with the code living under module/

Change-Id: If182f7cf2d160d57443a1b5f24e0065f191b59b2
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13919
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-09-21 08:17:48 +00:00
Ben Walker
dd7140e627 accel: Rename spdk_accel_engine_module_finish to
spdk_accel_module_finish

Also move it into the internal header that defines the interface used by
modules.

Change-Id: I3aeb41e643f27a69556099cb8d166f64c9e5d67f
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13917
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
2022-09-06 07:16:17 +00:00
Ben Walker
225ae35a2f accel: Add per-module task structures
These hold a pointer to the channel which eliminates the need to look it
back up in the completion path.

Change-Id: Ie4fc98d92d6434262e64b9483ef8b3b0591d764a
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13914
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2022-08-16 10:22:55 +00:00
Ben Walker
df892eed67 accel: Return correct values for .get_ctx_size()
This expects the full size of the task for each module. This only worked
because the software module returned the right size.

Change-Id: I481cfad8b4bb9c3748301bdacd90e7f44fd2d878
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13913
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2022-08-16 10:22:55 +00:00
Ben Walker
aa156d53be accel: Combine spdk_accel_engine and spdk_accel_module_if
These are 1:1 - they do not need to be separate objects.

Change-Id: I74ab52863f911d9be59ce98e1525302b5bd40846
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13910
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2022-08-16 10:22:55 +00:00
Ben Walker
32ee475a5e accel: SPDK_ACCEL_MODULE_REGISTER is now passed the module
Instead of passing each parameter to create a module, just have the user
make one and pass it in. This makes it easier to change the module
definition later.

Change-Id: I3a29f59432a6f0773129d7b210fbc011175b2252
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13909
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-08-10 11:00:17 +00:00
Jim Harris
488570ebd4 Replace most BSD 3-clause license text with SPDX identifier.
Many open source projects have moved to using SPDX identifiers
to specify license information, reducing the amount of
boilerplate code in every source file.  This patch replaces
the bulk of SPDK .c, .cpp and Makefiles with the BSD-3-Clause
identifier.

Almost all of these files share the exact same license text,
and this patch only modifies the files that contain the
most common license text.  There can be slight variations
because the third clause contains company names - most say
"Intel Corporation", but there are instances for Nvidia,
Samsung, Eideticom and even "the copyright holder".

Used a bash script to automate replacement of the license text
with SPDX identifier which is checked into scripts/spdx.sh.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iaa88ab5e92ea471691dc298cfe41ebfb5d169780
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12904
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: <qun.wan@intel.com>
2022-06-09 07:35:12 +00:00
wanghailiangx
465f99e9ff accel module: remove support for deprecated RPC names
These were deprecated in 2019, it's time to remove
support for them now.

Change-Id: I5a715e9b9e991c6febec5e505384728281eee8b7
Signed-off-by: wanghailiangx <hailiangx.e.wang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12773
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-05-23 16:14:41 +00:00
paul luse
b483811ff1 modules/accel/iaa: add IAA accel_fw module
And associated RPC to enable.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I06785bcd8b8957293ad41d13bab556fe62f29fd5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12765
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-05-23 16:04:57 +00:00
paul luse
ecaa8e1000 lib/idxd: prepare some plumbing for adding IAA
Misc internal IDXD changes needed to support the upcoming addition
of IAA.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Idb180088af545b174ed33a4f8ee113e58640477f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12764
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-05-23 07:02:21 +00:00
paul luse
ffef30ae0d modules/accel_dsa: update IDXD references to DSA where it makes sense
IDXD has always been used everywhere but technically it stands for
the driver, not the HW (Intel Data Streaming Accelerator Driver)
where the X comes from "Streaming Accelerator" somehow.  Anyway, the
underlying hardware is just DSA.  It doesn't matter much now but
upcoming patches will add support for a new HW accelerator called
the Intel In-Memory Analytics Accelerator which we'll call IAA and
it will use the same (mostly) device driver (IDXD) as DSA.  So, calling
the HW what it is will lessen confusion when adding IAA support.

This patch just does renaming for the accel_fw module and associated
files (RPC, etc).

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Ib3b1f982cc60359ecfea5dbcbeeb33e4d69aee6a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11984
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-05-23 07:02:21 +00:00
paul luse
d58a2f6cc5 lib/accel: support multiple accel modules (aka engines) at once
We enable multiple engines by:

* getting rid of the globals that point to the one available HW
and one available SW engine

* adding a submit_tasks() entry point for the SW engine so that
it is treated like any other engine allowing us to just call
submit_tasks() to the assigned engine for the opcode instead of
checking what is supported

* changing the definition of engine capabilities from
"HW accelerated" to simply "supported"

* during init, use a global (g_engines_opc) that contains engines
and is indexed by opcode so we know what the best engine is for each
op code

* future patches will add RPC's to override engine priorities or
specifically assign an opcode(s) to an engine.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I9b9f3d5a2e499124aa7ccf71f0da83c8ee3dd9f9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11870
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2022-05-05 07:11:32 +00:00
paul luse
bf5aca3274 module/accel/idxd: change json write of kernel mode from U32 to bool
g_kernel_mode is a bool everywhere it is used but was written as a U32
in write_config_json routine.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I16b68b88a259df0d8240b64464729bd4a0ef84ec
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12275
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: <wayne.gao@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-04-21 08:12:37 +00:00
paul luse
37b68d7287 accel: cleanup by getting rid of capabilties enum
In support of upcoming patches and to greatly simplify things,
the capabilites enum which held bit positions for each opcode
has been removed.  Only the opcodes enum remains and thus only
opcodes are used throughout.  For the capabiltiies bitmap a helper
function is added to convert from opcode to bit position.  Right
now it is used in the IO path but in upcoming patches that goes away
and the conversion is only done at init time.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Ic4ad15b9f24ad3675a7bba4831f4e81de9b7bc70
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11949
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-04-14 08:32:50 +00:00
paul luse
bc72388eb6 modules/accel/ioat: remove duplicate call to free device
It's already done in ioat_destroy_cb(). Proper fix required moving
the rest of the cleanup code in accel_engine_ioat_exit() to a new
device unregister callback.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Iaaa595cf5b51f7a4842315fc06270b0857ebf0c5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11930
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2022-03-25 08:18:16 +00:00
paul luse
dacb66d7f4 module/accel/ioat: fix bug with 'fill' handling
Fill is sent in as a uint8, we need to populate the full uint64
input with the uint8 pattern or we'll get a miscompare. This is
how idxd was doing it, instead of adding the same code to ioat just
move it up a layer.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Ia4aab1c6230f35ab88bb8a0e3b8e16dbd93007c7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11947
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2022-03-25 08:18:16 +00:00
Ben Walker
f0bf4e75f5 idxd: Eliminate configs
SPDK has settled on what the optimal DSA configuration is, so let's
always use it.

Change-Id: I24b9b717709d553789285198b1aa391f4d7f0445
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11532
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2022-03-21 11:05:28 +00:00
paul luse
8951c15759 accel/idxd: add and respect flag to support writes to PMEM
Plumbing for flags was added in prior pathces.  This patch
introduces and respects the relevant flags for use with PMEM
aka durable memory through the accel_fw, IDXD, IOAT and SW
modules.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I792f31459e061d220965feced60e0c236d819a68
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9455
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-03-04 21:56:54 +00:00
paul luse
7d63e716eb idxd: Add flags parameter to all low level API for prep/submission
Previously required flags were hardcoded in the low level library.
By having the user pass them in there is more flexbility and control.
This was driven by the need to add a new flag for pmem durability,
coming in a future patch in this series.

There is no change in functionality with this patch, just movement
of where flags are set and by whom and the plumbing of 'flags'..

Also note that some flags in scenarios that we know are required are
still set by the library.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I194278f9e3cec0886628585cf84bcc2eae635e0a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9449
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: GangCao <gang.cao@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-03-04 21:56:54 +00:00
Tomasz Zawadzki
047c067c05 so_ver: increase all major versions
To allow SO_MINOR updates on LTS for the whole year it is supported,
the major version for all components needs to be increased.
This is to prevent scenario where two versions exists with matching
versions, but conflicting ABI.
Ex. Next SPDK release adds an API call increasing the minor version,
then LTS needs just a subset of those additions.

Increasing major so version after LTS, allows the future releases
to update versions as needed. Yet allowing LTS to increase minor
version separately.

Disabled test for increasing SO version without ABI change, as
that is goal of this patch. This check shall be removed with SPDK 22.05
release.

This patch:
- increases SO_VER by 1 for all components
- resets SO_MINOR to 0 for all components
- removes suppressions for ABI tests

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Id1a5358882dc496faa5b0b5c9a63b326c378c551
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11361
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-01-31 15:29:56 +00:00
Ben Walker
5914f02b2a accel: Don't query the channel queue depth. Rely on -EBUSY
We can just queue things up until we get -EBUSY and not track the queue
depth.

Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: I49d3bcae0e6705a322de54fa91c9e1c6dfaea0c2
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11028
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2022-01-20 07:54:55 +00:00
Ben Walker
61c9017c64 idxd: Eliminate spdk_idxd_configure_chan
We can do all of the configuration in spdk_idxd_get_channel, and the
configuration step was always done immediately after getting the channel
anyway.

Change-Id: I9fef342e393261f0db6308cd5be4f49720420aa0
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10349
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
2022-01-19 08:49:25 +00:00
Ben Walker
8d2b6e6873 idxd: Add support for vectored crc32 + copy
Change-Id: Ib017280d6d0b2e115f5609b6b1a50793953ffa29
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10290
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2022-01-12 08:20:39 +00:00