This patch fixes issue # 2809, by changing the max
completions processed per poll. A new parameter called
IDXD_MAX_COMPLETIONS is used to set maximum completions
processed per poll to 128 because we observed performance
degradation on a system with 16 NVMe SSDs at a queue depth
of 64 per SSD. When using DSA to compute the data digest,
the target application can issue upto 1024(16x64) request
to compute data digest concurrently to DSA. Limiting the
maximum completions processed per poll to 32 using
DESC_PER_BATCH cause up to 43% IOPS degradation.
Use IDXD_MAX_COMPLETIONS to control the number of
completions proccessed per poll in spdk_idxd_process_event
based on your workload. For example, if your application
is issuing 1000s of concurrent request to DSA you might
want to set IDXD_MAX_COMPLETIONS to a value higher than
128.
Change-Id: I2a1db993283a83a20266f40dac851728d63e6127
Signed-off-by: John Kariuki <John.K.Kariuki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15801
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Per specification.
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Ic93349c7d3ed50fa6e502e39db0347141804d4c4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15673
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
per Intel policy to include file commit date using git cmd
below. The policy does not apply to non-Intel (C) notices.
git log --follow -C90% --format=%ad --date default <file> | tail -1
and then pull just the 4 digit year from the result.
Intel copyrights were not added to files where Intel either had
no contribution ot the contribution lacked substance (ie license
header updates, formatting changes, etc). Contribution date used
"--follow -C95%" to get the most accurate date.
Note that several files in this patch didn't end the license/(c)
block with a blank comment line so these were added as the vast
majority of files do have this last blank line. Simply there for
consistency.
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Id5b7ce4f658fe87132f14139ead58d6e285c04d4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15192
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
When crc32c is invoked with a multiple entry input iov,
only the last op has crc_dst set in order to write the final
crc value into the user supplied location.
spdk_idxd_process_events() for every successfully completed
CRC op writes the value into *op->crc_dst
UNLESS it is NULL.
The problem is that _idxd_prep_batch_cmd() that allocates
new ops left op->crc_dst uninitialized.
This results in a memory corruption (use after free)
in the following scenario:
1) op A is allocated an crc_dst is set to point to user memory X.
2) Op A is compeleted
3) User memory X is freed.
4) Ops B and C are allocated (chained), C has crc_dst set.
=> B reused op A memory and crc_dst still points to the
now stale user location (1)
5) B is complered, spdk_idxd_process_events() writes into X
as B->crc_dst = X.
Fix: _idxd_prep_batch_cmd() should initialize crc_dst to NULL.
Signed-off-by: Anton Eidelman <anton@lightbitslabs.com>
Change-Id: I9e7d57ec43a8fbcb3750906015a5cb7291278c35
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15115
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
When spdk_idxd_submit_crc32c() handles input
with multiple iovs (or multiple ops are generated
due to physically discontinuous buffers),
the first op has the original seed, while the
subsequent ops instruct the hardware to
to fetch the seed from the output of the previous op
(op->hw.crc32c_val):
void *prev_crc;
...
desc->flags |= IDXD_FLAG_FENCE | IDXD_FLAG_CRC_READ_CRC_SEED;
desc->crc32c.addr = (uint64_t)prev_crc; <<< virtual addr
The problem is the prev_crc is a virtual address,
so the hardware (at least with no IOMMU configured)
reports: DSA_COMP_HW_ERR1
spdk_idxd_process_events: Completion status 0x20
Solution:
Set crc32c.addr to the physical address of
the crc32c_val field in the previous desc.
Since desc->completion_addr already holds the physical address
of the dsa_hw_comp_record, we use this with the crc32c_val offset.
Signed-off-by: Anton Eidelman <anton@lightbitslabs.com>
Change-Id: I330e98c2f3fd6da5cb4fc03d0745df09a9ff0e0c
Signed-off-by: Anton Eidelman <anton@lightbitslabs.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14954
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
If the kernel is booted with the IOMMU enabled and Shared Memory mode
enabled (which are the expected boot parameters for production servers),
then the kernel idxd driver will automatically register a dedicated work
queue with the PASID for the process that opens it. This means that the
descriptors written into the portal for that work queue should be
*virtual* addresses.
If the IOMMU is enabled but Shared Memory mode is disabled, then the
kernel has registered the device with the IOMMU and assigned it I/O
virtual addresses. We have no way to get those addresses from user
space, so we cannot use the kernel driver in this mode. Add a check to
catch that.
If the IOMMU is disabled, then physical addresses are used everywherre.
Change-Id: I0bf079835ad4df1128ef9db54f5564050327e9f7
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14019
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
The DSA specification calls out that software must use a memory barrier
such as sfence prior to writing a descriptor or incorrect data may be
transferred during the operation.
Change-Id: I12f20e5a748e41616c7a542ccdb158c6b548eea4
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14018
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
In prep for upcoming iovec based compression/decompression patches.
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I413493f764bead9e56266e488b74f8bca979e225
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14633
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
To prevent the processing of outstanding commands from starving the rest of the system
Fixes: #2586
Signed-off-by: Chen, You <you.chen@intel.com>
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Change-Id: I392db2359408cdef32cc1f46b76ecd94f0c3332c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13685
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Masked by how accel_perf was doing decomp verificiation which is
changed in the next few patches and verifies these fixes.
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Icb03fc169bf8d2f05396addaf1db56d6de1827d1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13038
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This avoids confusion with the public idxd.h
header file which causes problems with abidiff.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I7910c93d9d95b99c82f4dfdba845e6804e1b6568
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13729
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Many open source projects have moved to using SPDX identifiers
to specify license information, reducing the amount of
boilerplate code in every source file. This patch replaces
the bulk of SPDK .c, .cpp and Makefiles with the BSD-3-Clause
identifier.
Almost all of these files share the exact same license text,
and this patch only modifies the files that contain the
most common license text. There can be slight variations
because the third clause contains company names - most say
"Intel Corporation", but there are instances for Nvidia,
Samsung, Eideticom and even "the copyright holder".
Used a bash script to automate replacement of the license text
with SPDX identifier which is checked into scripts/spdx.sh.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iaa88ab5e92ea471691dc298cfe41ebfb5d169780
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12904
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: <qun.wan@intel.com>
Provide an interface to allow the caller to provide a proprely
formatted descriptor.
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I5c397761f556361040ec962d61169459150b6494
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12703
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Accel module coming in next patch...
Add support for compress and decompress. The low level IDXD
library supports both DSA and IAA hardware. There are separate
modules for DSA and IAA.
accel_perf patch follows.
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I55014122f6555f80985c11d49a54eddc5d51c337
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12292
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
In prep for upcoming IAA additions.
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Id89124a3c3d5b1bcfd4d805ff4ee84a2f64f8a4a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12767
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Misc internal IDXD changes needed to support the upcoming addition
of IAA.
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Idb180088af545b174ed33a4f8ee113e58640477f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12764
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
In prep for upcoming addition of IAA.
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I47c5880aac37da9a38d6af6e52a51cefbfec91b9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12762
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This requires handling vtophys entries that cross page boundaries.
Fixes#2316
Change-Id: I9e9aafc1612bc89375c783bcf91bd04ab523ab9e
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12217
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Instead of releasing the batch memory when the batch generates a
completion, instead do it via refcnt. This will allow us to later hold
onto batch memory longer if vectored transactions end up spanning a
batch.
Change-Id: I942d6aa5052029eb0951e51a046dd98943108b94
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12259
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This does not actually need a doubly linked list. Single is enough. That
frees up 4 more bytes in the op for other uses.
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: I8c2a30de175b42815afd0a3ba3c694aef2f35882
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12258
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
If we don't find a completion, break out of the loop at the top. This
removes a level of indentation but most importantly makes it very clear
that we only remove elements from ops_outstanding from the front of the
list.
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: I5e8784a5af5449c14ff7015bc8e6062e6aee6b4e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12257
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
probed
This can cause a mismatch of kernel vs user driver and isn't allowed.
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: I9c572ea1fa1da89d7b41e31ab4719eec719fb50a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10588
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
The only place a batch can be created is by assigning it to the channel
now, so this isn't a mistake that can be made and the checks can all be
removed.
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: I915edb4f212c0751396554655ffe95ae3bb20cd6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11538
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This effectively means there is only ever a single batch being build at
a time, which simplifies a lot of the APIs.
Change-Id: Ifd66cd1ce6f6f0abe2011528dd862c5324213658
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11223
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Currently the idxd driver requires VFIO so avoid unexpected errors
if someone tries without it (with UIO).
Temp workaround for issue #2316
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I430cd2193bc8dbd6939af7d0ca799832e7a73213
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11816
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Assume any given WQ we are allocated can submit the full device allowed
queue depth.
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: I044e1f70031ea83ae722ed285b84c06b3e5efb27
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11486
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
SPDK has settled on what the optimal DSA configuration is, so let's
always use it.
Change-Id: I24b9b717709d553789285198b1aa391f4d7f0445
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11532
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
A new public flag, SPDK_IDXD_FLAG_NONTEMPORAL, is introduced
that hints to DSA that it should bypass CPU cache. An example
use case of this would be where the target is PMEM. This flag
is not set by the low level library (so default is that CPU
cache is the target).
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I99fc62d6bfd6ec97985a86fc1ae5454a21679684
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11482
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: GangCao <gang.cao@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Previously required flags were hardcoded in the low level library.
By having the user pass them in there is more flexbility and control.
This was driven by the need to add a new flag for pmem durability,
coming in a future patch in this series.
There is no change in functionality with this patch, just movement
of where flags are set and by whom and the plumbing of 'flags'..
Also note that some flags in scenarios that we know are required are
still set by the library.
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I194278f9e3cec0886628585cf84bcc2eae635e0a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9449
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: GangCao <gang.cao@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Although there are no use cases right now where a batch can have
mixed op types, there may be in the future and rather than have
one blow up because ops have different reserved fields and its
not valid to submit an op with a non-zero reserved field, go
ahead and zero these out like we do with descriptors in the non
batch case.
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I6d1bb416dc84aa1f76407c76aedf0768dd003218
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11325
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
We always set num_batches to the same value we set num_descriptors
so just make it explicit. Makes it easier to experiment with
different values when performance testing.
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I648262001772c791a032d6cab38dc3b03c1d55c1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11354
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
The new batching code needs to call the cb_fn for each of the
elements of the batch when a batch that hasn't been submitted
yet needs to be cancelled (due to an error in building it).
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I6f94b27dd7c64f756193ec3532de98b644b41d7e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11212
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
In several functions. Busy handling also maans paying attention
to the rc when submitting a batch and not clearing chan->batch
unless the call was a success.
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Ic45b10ade2ebdcd845dc33e54dd9c93068ceb98c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11221
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
To avoid re-use of descriptors that may have fields set that are
reserved by the one being used now. For example:
If a batch desc is being built and was previously used by a copy
we need to clear out the dst_addr field or things will explode
as this is a reserved field for a batch.
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I6ba50b76589e38a276683291f5ec2970c80e8aa8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11308
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
to a single command
Change-Id: Ic0ca65b7399f3cbc4153327d83de7db69de48709
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11209
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Convert the batch to the single command inside of it.
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: Ia117175ef3f4a8290d313e0bdc794f6a3276e042
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11166
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Transparently group independent operations into idxd batch operations
between polls. This increases the effective queue depth.
Change-Id: Ic09b21ed29aaefe2eccef9a6ae0e1b05990ef631
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10998
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
This is no longer needed.
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: I11b7e9acbcf1239a0ad2f49169d7e3d5844a1b93
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11029
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Nightly build failing on Centos 7 machine
C compiler for the host machine: cc (gcc 4.8.5 "cc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-44)")
C linker for the host machine: cc ld.bfd 2.27-44
Host machine cpu family: x86_64
Host machine cpu: x86_64
Errors like:
idxd.c: In function ‘spdk_idxd_submit_crc32c’:
idxd.c:902:24: error: ‘prev_crc’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
desc->crc32c.addr = (uint64_t)prev_crc;
Signed-off-by: Pawel Piatek <pawelx.piatek@intel.com>
Change-Id: Ib40160b1974ecd3f1579566b6eb5d88e03b5bb2b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11082
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
We can do all of the configuration in spdk_idxd_get_channel, and the
configuration step was always done immediately after getting the channel
anyway.
Change-Id: I9fef342e393261f0db6308cd5be4f49720420aa0
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10349
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
The driver still may use batches to implement some operations or for
efficiency reasons.
Batching my be resurrected in the future, but for now we need to do some
fairly extensive performance changes on the driver and eliminating all
of this unused/inactive code makes that much easier.
Change-Id: I92fcec9e4c7424771f053123d821cc57dba9793c
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10348
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
These will be used internally by some of the other code paths.
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: Idc3c024cf1bf3d468f87176373ef97bf064ced8f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11033
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
The batching API will be removed from idxd shortly.
Change-Id: I04cf61112f7831a9fb0fefc269706495761d0889
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11032
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
dynamically
This can be done once on allocation rather than every time the batch is
submitted.
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: I8092f65f1b864cc3cc78db9fdee085d8bb0471df
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10293
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
This only needs to be updated on the last step of the CRC calculation.
Change-Id: I0b41f33bfbbc195a857d1c39d9f8f7164d2bba8d
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10292
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This uses a batch with the fence flag for now. There are several other
implementation options that will be explored in the future.
Change-Id: I4f344d671400508de05f80b026d42f775c5b9588
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10289
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
