This file should be external to enable out-of-tree accel modules.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I2e973d0e88d7145d0fc9714f56db48486b00f3b7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17419
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
SW PMD might process a crypto operation but failed
to submit it to a completions ring.
Such operation can't be retried if crypto operation
is inplace.
Handle such crypto op as a completed.
Verified by integrating rte openssl driver and
adding additional logs to check that SUCCESS
status received and completed as expected.
Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: Ida161cec045167af752ebd5b57f41b2bbfe8b97c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16995
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
If we were not able to submit all configured
crypto ops, then we can just release crypto_ops
and mbuf object of these crypto ops and save
the actual number of submitted operation in
the accel task. Once all submitted operations
complete, poller will call
accel_dpdk_cryptodev_process_task func to submit
cyrpto operations for reamining data blocks.
If no crypto ops were submitted then the task
will be palced in the channel's queued_tasks
and poller will try to resubmit the task.
That in theory should increase performance
since we attempted to resubmit queued ops
with burst size==1 which is not efficient
Fixes issue #2907
Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: I4d17e8ed1ad5383848e4d09c46009c6cb2834360
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16784
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
When accel task is processed is processed in
several iterations (submit part of cryops, wait
for completion and submit next part of cryops),
sgl is initialized with offset to exclude previously
processed blocks. However there is a bug since
spdk_iov_sgl_init doesn't advance iovs, as result
when we do sgl->iov->iov_len - sgl->iov_offset,
we may get unsigned int underflow.
Fix is init sgl with 0 offset and then advance it
with offset.
Modified unit test and added an assert in code to
verify this fix.
Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: Ib53ff30f0c90d521f2cf6b3ec847b0d06869c2b5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17456
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This was added during debug/development and
was not deleted by accident.
Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: I146748151623a3ea4808c7b5cc34085d30ca81b4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16388
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
All DPDK related code is removed, handling of
RESET command was sligthly updated.
Handling of -ENOMEM was updated for cases when
accel API returns -ENOMEM
Crypto tests in blockdev.sh were extended with more
crypto_bdevs to verify NOMEM cases - that failed
with original vbdev_crypto implementation
Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: If1feba2449bee852c6c4daca4b3406414db6fded
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14860
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
It was found that if system has no crypto devs,
we exit earlier from init function and don't register
io_device. Later in fini function we may try to unregister
invalid io_device
Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: I23ac9674c02d4fe97f01c66928a5ec5a2b39ec47
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16321
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This patch is a combination of commits which update vdev_crypto:
110d8411e bdev/crypto: do not create mempool for session private data
495055b05 bdev/crypto: update rte_cryptodev usage for DPDK 22.11
02caed6b5 bdev/crypto: remove mempool usage matching < DPDK 19.02
5887eb321 bdev/crypto: do not track type of crypto session
Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: I30c4f76e4e7b4865a7daa638d357888bb5e02071
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16039
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
When we submit more tasks than supported by qp,
extra tasks are queued on io_channel. Later completion
poller tries to resubmit these tasks one by one. That
is not efficient since every enqueu_burst may cause
doorbell updates in HW.
Instead add a check for qpir capacity and submit
appropriate number of requests. If qpair is full,
tasks are queued in dedicated list. This approach
should remove or minimize the need to resubmit
individual crypto operations.
This also handles a case where there are no entries
in global pools (crypto_ops or rte_mbuf)
Fixes issue #2756
Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: Iab50e623e7a82a4f5bef7a1e4434e593240ab633
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15769
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Previously vbdev_crypto used DPDK directly and
the restriction on max IO size was propagated to
generic bdev layer which split big IO requests.
Now, when DPDK code is a standalone accel module,
this restriction on max IO size is not visible to
the user and we should get rid of it.
To remove this limitation, allow to submit crypto
operations for part of logical blocks in big IO,
the rest blocks will be processed when all submitted
crypto ops are completed.
To verify this patch, add a functional test which
submits big IO verify mode
Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: I0ee89e98195a5c744f3fb2bfc752b578965c3bc5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15768
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
All operations are using iovecs to describe their buffers and only
encrypt/decrypt additionally used nbytes to store the total size of a
src buffer. We don't really need this value in the generic accel code,
so we can let modules calculate it, if necessary. That way, we won't
waste cycles calculating it if a module doesn't use it and it makes the
code a bit easier, as we won't have to deal with the fact that nbytes is
only valid for certain operations.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I29252be34a9af9fd40f4c7fec9d0a0c1139c562d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16306
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
The new module replaces functionality in vbdev_crypto.
This module is bdev agnostic, so some inernal parts
were reworked.
io_channel: contains a qp of every configured DPDK PMD
crypto key: for mlx5_pci we register a key on each available
device since keys are bound to Protection Domain.
Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: If1845cb87eadacbb921c593ba82207a97f2209a3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14859
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
This patch is just a copy of vbdev_crypto.c and the
corresponding UT file. It makes it easier to review
the next patch which adds accel operations
Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: Ib88b45d573b011b1acb35da9bf4dab922d8fb183
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16182
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
