Commit Graph

155 Commits

Author SHA1 Message Date
Seth Howell
f6dcd815f4 lib/idxd: add a map file.
While we are here, give the library an so suffix
which was missed when the library was initially
created.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I68929127bf672c0f2f7153f9716882d03151480e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2209
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-05-11 12:29:07 +00:00
paul luse
d7d9dc126b lib/idxd: factor out common submission code into a prep routine
Prior to adding more operations, make this a bit more efficient.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I02b864f998800c25076233183840bba8bff92196
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2069
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-05-07 10:41:44 +00:00
paul luse
9d94a8d53a idxd: clean up some enum style issues for consistency
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I523176cc49951e6d6513b86b1e05ca8e2b5e62f5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2022
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-04-30 17:51:19 +00:00
paul luse
5b03dd938c module/idxd: accel framework plug-in for idxd
Docs, RPC, unit tests, etc., will follow.  Notes:

* The current implementation will only work with VFIO.

* The current implementation supports only the existing accel
framework API. The API will be expanded for DSA exclusive features
in a subsequent patch.

* SW is required to manage flow control, to not over-run the work queues.
This is provided in the accel plug-in module. The upper layers use public
API to manage this.

* As we need to support any number of channels (we can't limit ourselves
to the number of work queues) we need to dynamically size/resize our
per channel descriptor rings based on the number of current channels. This
is done from upper layers via public API into the lib.

* As channels are created, the total number of work queue slots is divided
across the channels evenly.  Same thing when they are destroyed, remaining
channels will see the ring sizes increase. This is done from upper layers
via public API into the lib.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Ifaa39935107206a2d990cec992854675e5502057
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1722
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-04-23 15:48:32 +00:00
paul luse
e58e9fbda8 lib/idxd: add low level idxd library
Module, etc., will follow. Notes:

* IDXD is an Intel silicon feature available in future Intel CPUs.
Initial development is being done on a simulator. Once HW is
available and the code fully tested the experimental label will be
lifted. Spec can be found here: https://software.intel.com/en-us/download/intel-data-streaming-accelerator-preliminary-architecture-specification

* The current implementation will only work with VFIO.

* DSA has a number of engines that can be grouped based on application
need such as type of memory being served or QoS. Engines are processing
units and are assigned to groups. Work queues are on device structures
that act as front-end groups for queueing descriptors. Full details on
what is configurable & how will come in later doc patches.

* There is a finite number of work queue slots that are divided amongst
the number of desired work queues in some fashion (ie evenly).

* SW (outside of the idxd lib) is required to manage flow control, to not
over-run the work queues.This is provided in the accel plug-in module.
The upper layers use public API to manage this.

* Work queue submissions are done with a 64 byte atomic instruction

* The design here creates a set of descriptor rings per channel that match
the size of the work queues. Then, an spdk_bit_array is used to make sure
we don't overrun a queue.  If there are not slots available, the operation
is put on a linked list to be retried later from the poller.

* As we need to support any number of channels (we can't limit ourselves
to the number of work queues) we need to dynamically size/resize our
per channel descriptor rings based on the number of current channels. This
is done from upper layers via public API into the lib.

* As channels are created, the total number of work queue slots is divided
across the channels evenly. Same thing when they are destroyed, remaining
channels with see the ring sizes increase. This is done from upper layers
via public API into the lib.

* The sim has 64 total work queue entries (WQE) that get dolled out to the
work queues (WQ) evenly.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I899bbeda3cef3db05bea4197b8757e89dddb579d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1809
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-04-23 15:48:32 +00:00