Commit Graph

146 Commits

Author SHA1 Message Date
paul luse
c746637df8 lib/idxd: add some flag overrides when doing PMEM writes
Per upcoming specification changes.

Fixes: 2486

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Ic2534148a87b3dec7512f7b01384f484fee4c30f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13572
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: <wayne.gao@intel.com>
2022-08-18 18:47:02 +00:00
paul luse
61631dadb3 lib/idxd: Save device version during kernel and user initialization
We'll likely need this eventually to address silicon version
specific workarounds.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Ie6957674113cf0c7b7d695b468c694668ebbf2bd
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13571
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-08-18 18:47:02 +00:00
Chen, You
43ebecdf60 lib/idxd: break spdk_idxd_process_events loop after processing DESC_PER_BATCH ops
To prevent the processing of outstanding commands from starving the rest of the system

Fixes: #2586

Signed-off-by: Chen, You <you.chen@intel.com>
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Change-Id: I392db2359408cdef32cc1f46b76ecd94f0c3332c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13685
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-08-18 10:09:23 +00:00
paul luse
3d5fd5a59f lib/idxd: fix bugs with IAA decompression descriptor construction
Masked by how accel_perf was doing decomp verificiation which is
changed in the next few patches and verifies these fixes.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Icb03fc169bf8d2f05396addaf1db56d6de1827d1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13038
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-08-10 07:25:29 +00:00
Jim Harris
79c9b1e5df idxd: rename internal header file to idxd_internal.h
This avoids confusion with the public idxd.h
header file which causes problems with abidiff.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I7910c93d9d95b99c82f4dfdba845e6804e1b6568
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13729
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-07-21 07:03:26 +00:00
Jim Harris
488570ebd4 Replace most BSD 3-clause license text with SPDX identifier.
Many open source projects have moved to using SPDX identifiers
to specify license information, reducing the amount of
boilerplate code in every source file.  This patch replaces
the bulk of SPDK .c, .cpp and Makefiles with the BSD-3-Clause
identifier.

Almost all of these files share the exact same license text,
and this patch only modifies the files that contain the
most common license text.  There can be slight variations
because the third clause contains company names - most say
"Intel Corporation", but there are instances for Nvidia,
Samsung, Eideticom and even "the copyright holder".

Used a bash script to automate replacement of the license text
with SPDX identifier which is checked into scripts/spdx.sh.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iaa88ab5e92ea471691dc298cfe41ebfb5d169780
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12904
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: <qun.wan@intel.com>
2022-06-09 07:35:12 +00:00
paul luse
fe044f6988 lib/idxd: add raw request for low level testing
Provide an interface to allow the caller to provide a proprely
formatted descriptor.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I5c397761f556361040ec962d61169459150b6494
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12703
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-05-23 19:09:50 +00:00
paul luse
0ff560ea3b lib/idxd: Add compress/decompress support to low level lib
Accel module coming in next patch...

Add support for compress and decompress. The low level IDXD
library supports both DSA and IAA hardware.  There are separate
modules for DSA and IAA.

accel_perf patch follows.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I55014122f6555f80985c11d49a54eddc5d51c337
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12292
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-05-23 16:04:57 +00:00
paul luse
4d9a00d791 lib/idxd: factor out batch allocation in spdk_idxd_get_channel()
In prep for upcoming IAA additions.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Id89124a3c3d5b1bcfd4d805ff4ee84a2f64f8a4a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12767
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-05-23 07:02:21 +00:00
paul luse
ecaa8e1000 lib/idxd: prepare some plumbing for adding IAA
Misc internal IDXD changes needed to support the upcoming addition
of IAA.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Idb180088af545b174ed33a4f8ee113e58640477f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12764
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-05-23 07:02:21 +00:00
paul luse
76fae14976 lib/idxd: update names from IDXD->DSA where it makes sense
In prep for upcoming addition of IAA.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I47c5880aac37da9a38d6af6e52a51cefbfec91b9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12762
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-05-23 07:02:21 +00:00
Ben Walker
7ac08606e9 idxd: Support running without an IOMMU
This requires handling vtophys entries that cross page boundaries.

Fixes #2316

Change-Id: I9e9aafc1612bc89375c783bcf91bd04ab523ab9e
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12217
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
2022-05-20 19:40:47 +00:00
Ben Walker
855390a585 idxd: Release batches based on refcnt
Instead of releasing the batch memory when the batch generates a
completion, instead do it via refcnt. This will allow us to later hold
onto batch memory longer if vectored transactions end up spanning a
batch.

Change-Id: I942d6aa5052029eb0951e51a046dd98943108b94
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12259
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-05-18 07:28:57 +00:00
Ben Walker
26f503b00f idxd: Set the impl in idxd for kernel mode
Without this, it does not think there is a driver available.

Change-Id: I0b8b42374e0ed82abb22bf27e0b8907bb03c61f6
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12641
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: GangCao <gang.cao@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-05-16 10:15:08 +00:00
Ben Walker
f2efa8f293 idxd: Keep ops on a singly linked list
This does not actually need a doubly linked list. Single is enough. That
frees up 4 more bytes in the op for other uses.

Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: I8c2a30de175b42815afd0a3ba3c694aef2f35882
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12258
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2022-04-28 07:29:51 +00:00
Ben Walker
863200c7d4 idxd: Rearrange logic in spdk_idxd_process_events
If we don't find a completion, break out of the loop at the top. This
removes a level of indentation but most importantly makes it very clear
that we only remove elements from ops_outstanding from the front of the
list.

Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: I5e8784a5af5449c14ff7015bc8e6062e6aee6b4e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12257
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2022-04-28 07:29:51 +00:00
Ben Walker
7f75e1081a idxd: Do not allow calls to spdk_idxd_set_config after devices have been
probed

This can cause a mismatch of kernel vs user driver and isn't allowed.

Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: I9c572ea1fa1da89d7b41e31ab4719eec719fb50a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10588
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-04-21 08:11:33 +00:00
Ben Walker
85580d47e1 idxd: Remove _idxd_batch_is_valid
The only place a batch can be created is by assigning it to the channel
now, so this isn't a mistake that can be made and the checks can all be
removed.

Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: I915edb4f212c0751396554655ffe95ae3bb20cd6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11538
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-04-21 08:11:33 +00:00
Ben Walker
b2bdbbac56 idxd: Always store the current batch in chan->batch
This effectively means there is only ever a single batch being build at
a time, which simplifies a lot of the APIs.

Change-Id: Ifd66cd1ce6f6f0abe2011528dd862c5324213658
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11223
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2022-04-21 08:11:33 +00:00
Ben Walker
de732691a3 idxd: Simplify the kernel mode to only create 1 WQ per device
Change-Id: I32e4fe2592c63752f08c326fb9845aa44ef7775b
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11537
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2022-04-21 08:11:33 +00:00
Ben Walker
e22c933edb idxd: Make many internal idxd_user functions take an idxd_user object
This reduces a lot of casting.

Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: Ibc04f422858642d0e20c9b020bb6c5d1b70256fe
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11534
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2022-04-20 08:20:45 +00:00
paul luse
75209b1d53 lib/idxd: fail init if IOMMU is not enabled
Currently the idxd driver requires VFIO so avoid unexpected errors
if someone tries without it (with UIO).

Temp workaround for issue #2316

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I430cd2193bc8dbd6939af7d0ca799832e7a73213
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11816
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-04-01 08:30:46 +00:00
paul luse
b9d44da07d lib/idxd: Further simplify WQ configuration code
As we now only support a single WQ, there's no need for a teble of
them and no need to assert that the stride from WQ to WQ is the
same as the WQ struct size.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I205f36aae22070f532653726dd75249bbafbe3ef
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12081
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
2022-03-31 17:59:21 +00:00
Ben Walker
7dfe90df60 idxd: Remove idxd_group altogether
The driver always creates a single group containing all of the engines
and a single work queue.

Change-Id: I83f170f966abbd141304c49bd75ffe4608f5ad03
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11533
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2022-03-25 12:49:22 +00:00
Ben Walker
9de35e7fc8 idxd: Remove idxd_wq
It is not used for anything.

Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: I1d967b2d0e404756f7ceda98ddc4ee9017ec83f7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11489
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2022-03-25 12:49:22 +00:00
Ben Walker
225cf4b6ed idxd: Remove idxd_wqcfg from idxd_wq
It turns out that this can stay on the stack.

Change-Id: I961366307dae5ec7413a86271cd1dfb370b8f9f3
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11488
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2022-03-25 12:49:22 +00:00
Ben Walker
3b9c7ade6c idxd: Introduce grptbl and wqtbl structs to further simplify
initialization

We can make the structs do all of the offset math for us.

Change-Id: Ibe6d86c2abc58655c1354f1eb31091c95cfb283c
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11487
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2022-03-25 12:49:22 +00:00
Ben Walker
7a9b023008 idxd: Don't cache any register values
These aren't ever accessed in the main I/O path, so we can read them in
whenever we need them and make the code a lot simpler.

Change-Id: Icfdbfe9f2d9db13f4d0d28b2b4103cd0c443bcf4
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11485
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2022-03-25 12:49:22 +00:00
Ben Walker
d24bbd592f idxd: Calculate number of descriptors per channel based on total wq size
Assume any given WQ we are allocated can submit the full device allowed
queue depth.

Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: I044e1f70031ea83ae722ed285b84c06b3e5efb27
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11486
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2022-03-25 12:49:22 +00:00
Ben Walker
b3d3f2028b idxd: Eliminate config struct from idxd_user
This is no longer needed.

Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: I08c788ca0451e739804b568d613c1e52e071c61f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11794
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
2022-03-25 08:20:08 +00:00
Ben Walker
f0bf4e75f5 idxd: Eliminate configs
SPDK has settled on what the optimal DSA configuration is, so let's
always use it.

Change-Id: I24b9b717709d553789285198b1aa391f4d7f0445
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11532
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2022-03-21 11:05:28 +00:00
Ben Walker
049bb9e41f idxd: Update register names for idxd_group_flags
The names on these were changed.

Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: Ib75a60342c08f72dad39635a9244421c1cca5485
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11793
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2022-03-21 11:05:28 +00:00
paul luse
0d5ce8c2cf idxd: introduce new IDXD flag for cache control
A new public flag, SPDK_IDXD_FLAG_NONTEMPORAL, is introduced
that hints to DSA that it should bypass CPU cache. An example
use case of this would be where the target is PMEM. This flag
is not set by the low level library (so default is that CPU
cache is the target).

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I99fc62d6bfd6ec97985a86fc1ae5454a21679684
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11482
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: GangCao <gang.cao@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-03-04 21:56:54 +00:00
paul luse
37ae5a2fb9 idxd: Set cache control flag in 3 places where it was missing
Was simply missed earlier.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I74aed97d4858d9b364bd7043a221efeaf260f730
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11481
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: GangCao <gang.cao@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-03-04 21:56:54 +00:00
paul luse
7d63e716eb idxd: Add flags parameter to all low level API for prep/submission
Previously required flags were hardcoded in the low level library.
By having the user pass them in there is more flexbility and control.
This was driven by the need to add a new flag for pmem durability,
coming in a future patch in this series.

There is no change in functionality with this patch, just movement
of where flags are set and by whom and the plumbing of 'flags'..

Also note that some flags in scenarios that we know are required are
still set by the library.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I194278f9e3cec0886628585cf84bcc2eae635e0a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9449
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: GangCao <gang.cao@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-03-04 21:56:54 +00:00
paul luse
492d576795 Revert "idxd: No longer set token configuration"
This reverts commit 3bacd6653d.

Change-Id: I8dbaffc9f50cf9627720667644496cdaf4e81c3f
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11723
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-03-02 17:22:08 +00:00
paul luse
0ae7f4d816 idxd: zero out batch element descriptors before each use
Although there are no use cases right now where a batch can have
mixed op types, there may be in the future and rather than have
one blow up because ops have different reserved fields and its
not valid to submit an op with a non-zero reserved field, go
ahead and zero these out like we do with descriptors in the non
batch case.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I6d1bb416dc84aa1f76407c76aedf0768dd003218
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11325
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-02-10 22:50:47 +00:00
paul luse
7aa1650d28 idxd: minor one LOC cleanup
We always set num_batches to the same value we set num_descriptors
so just make it explicit. Makes it easier to experiment with
different values when performance testing.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I648262001772c791a032d6cab38dc3b03c1d55c1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11354
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-02-10 22:50:47 +00:00
Ben Walker
3bacd6653d idxd: No longer set token configuration
This has changed to control the number of read buffers allocated to the
group, but it is only valid to set this register if the device has
indicated it supports it. Further, the default value is what we want
anyway, so we can skip setting it altogether.

Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: Ic54672ea6cb16acc7613860e36d9f7033048bd98
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11484
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
2022-02-10 22:41:12 +00:00
Ben Walker
8c400e494a idxd: Remove unused nop_check callback
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: Ibda86694edc10a3edec8cb85638b4950647b13eb
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11435
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-02-10 22:41:12 +00:00
Ben Walker
dbdd27ff47 idxd: Rename idxd_cmdsts_reg to idxd_cmdsts_register
All of the other structs and unions spell out register, so match the
style.

Change-Id: Ie502e80206305037d1518a1db590d89b7479abb4
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11433
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2022-02-10 22:41:12 +00:00
Ben Walker
4ac05e3abc idxd: Rename idxd_genstatus_register to idxd_gensts_register
This is called GENSTS in the spec, so match that name.

Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: I0e8f917e13908f3920ab297e14cb3adee856eaa5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11432
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-02-10 22:41:12 +00:00
Tomasz Zawadzki
047c067c05 so_ver: increase all major versions
To allow SO_MINOR updates on LTS for the whole year it is supported,
the major version for all components needs to be increased.
This is to prevent scenario where two versions exists with matching
versions, but conflicting ABI.
Ex. Next SPDK release adds an API call increasing the minor version,
then LTS needs just a subset of those additions.

Increasing major so version after LTS, allows the future releases
to update versions as needed. Yet allowing LTS to increase minor
version separately.

Disabled test for increasing SO version without ABI change, as
that is goal of this patch. This check shall be removed with SPDK 22.05
release.

This patch:
- increases SO_VER by 1 for all components
- resets SO_MINOR to 0 for all components
- removes suppressions for ABI tests

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Id1a5358882dc496faa5b0b5c9a63b326c378c551
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11361
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-01-31 15:29:56 +00:00
paul luse
21c84e363d idxd: fix bug with cancelling a batch
The new batching code needs to call the cb_fn for each of the
elements of the batch when a batch that hasn't been submitted
yet needs to be cancelled (due to an error in building it).

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I6f94b27dd7c64f756193ec3532de98b644b41d7e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11212
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
2022-01-28 08:08:55 +00:00
paul luse
d5e63730ae idxd: fix busy handling
In several functions. Busy handling also maans paying attention
to the rc when submitting a batch and not clearing chan->batch
unless the call was a success.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Ic45b10ade2ebdcd845dc33e54dd9c93068ceb98c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11221
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
2022-01-28 08:08:55 +00:00
paul luse
057fdedbff idxd: zero out descriptor before use
To avoid re-use of descriptors that may have fields set that are
reserved by the one being used now.  For example:

If a batch desc is being built and was previously used by a copy
we need to clear out the dst_addr field or things will explode
as this is a reserved field for a batch.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I6ba50b76589e38a276683291f5ec2970c80e8aa8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11308
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-01-27 19:00:22 +00:00
Ben Walker
097ccf06a9 idxd: Correctly propagate the callback when converting a 1 sized batch
to a single command

Change-Id: Ic0ca65b7399f3cbc4153327d83de7db69de48709
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11209
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-01-26 08:12:44 +00:00
paul luse
8a6c8ba9ae idxd: updates to WQ config routine to match updated spec
Using the latest DSA we aren't supposed to (a) touch WQ space that
we aren't configuring and (b) touch WQ config fields that we are
configuring even if we are configuring that WQ.  So, this patch
will read in initial values of only the number of desired WQs
and update them accordingly before updating the HW.

Also updates a few vars to use shorter local variables consistently.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I7641cdfc5ccc839e37a1d46d760248799a8fce1f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10981
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-01-20 21:42:48 +00:00
paul luse
af6d280247 idxd: disable the idxd device on shutdown
Required by spec.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Ib13ca2d7ab7045a87ddc483dc1f70e649b3b9294
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10982
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
2022-01-20 21:42:39 +00:00
paul luse
c501d2b37c idxd: fix issue w/multiple WQ config
Found via inspection during spec review of latest HW. We were using the
wrong stride for the WQCFG regsiter when configuring but it just so
happened to be the right value for the current DSA version.  We were
mixing up the size of the WQCFG register with the stride value used to
configure the next WQCFG regsiter as they are not contiguous in HW, we
need to read another capabilities bit to determine the address of the
next wqcfg to configure..

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I14d1ff95e0131fd30121aa955bfbc8c8fb3fc512
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10968
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-01-20 21:39:56 +00:00