Commit Graph

2380 Commits

Author SHA1 Message Date
Jim Harris
4300c62167 nvme: add spdk_nvme_ctrlr_disable_read_changed_ns_list_log_page()
Commit a119799b ("test/nvme/aer: remove duplicated changed NS list log")
changed the nvme driver to read the CHANGED_NS_LIST log page before
calling the application's AER callback (previously it would read it
after).

Commit b801af090 ("nvme: add disable_read_changed_ns_list_log_page")
added a new ctrlr_opts member to allow the application to tell the
driver to not read this log page, and will read the log page itself
instead to clear the AEN.  But we cannot add this option to the 22.01
LTS branch since it breaks the ABI.  So adding this API here, which
can then be backported manually to the 22.01 branch for LTS users
that require it.

Restoring the old behavior is not correct for applications that
want to consume the CHANGED_NS_LIST log page contents itself to
know which namespaces have changed.  Even if the driver reads the
log page after the application, that read could happen during a
small window between when a namespace change event has occurred and
the AEN has been sent to the host.  The only safe way for the
application to consume ChANGED_NS_LIST log page contents itself
is to make sure the driver never issues such a log page request
itself.

Fixes issue #2647.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iaeffe23dc7817c0c94441a36ed4d6f64a1f15a4e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14134
Reviewed-by: Michael Haeuptle <michaelhaeuptle@gmail.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-08-25 07:31:44 +00:00
Kozlowski Mateusz
71f20c9a74 FTL: Add compaction logic
During compaction FTL moves valid user data from the nv cache drive to
the bottom device.

Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Change-Id: Ia200af39cec80014fac3a10f20d2859b10a81088
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13337
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-08-19 17:37:14 +00:00
Artur Paszkiewicz
0291b2845a FTL: Add read path
Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Change-Id: Ib5bac109b59d5a21a7dad1f8e79b5da7633ffa9d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13334
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-08-19 17:37:14 +00:00
Jim Harris
e36f0d363e nvme/pcie, nvme/tcp: add cb_arg context tracepoint argument
This allows mapping an nvme_request back to the
nvme_bdev_io.

This requires bumping up the max number of arguments per
tracepoint.  5 was previously chosen as max since it
exactly fit in 64 bytes (1 cacheline) when all
arguments were stored as uint64_t, but now that we
support uint32_t arguments we can afford extra
arguments when some of them are uint32_t.  I've
bumped it to 8 so we can avoid having to touch
this value multiple times if we find some cases
where we need 7 or 8 args.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ie2ef5e59d10549860b47542e68c1c34efa63047f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13995
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2022-08-19 11:06:31 +00:00
Jim Harris
54f1603954 bdev/nvme: add tracepoint support
This will allow us to map spdk_bdev_io events
to nvme_request events coming in a future patch.
Since we pass the nvme_bdev_io to the nvme driver
(not the spdk_bdev_io), we need to add tracepoints
for the nvme_bdev_io so that spdk_trace can
do the spdk_bdev_io->nvme_bdev_io->nvme_request
mapping.

An alternative would have been to pass the spdk_bdev_io
as the cb_arg to the nvme driver, but that change
seemed to invasive, and I think we will find other
uses for the nvme_bdev_io events anyways.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Id7519e689b01875093359f41a1ca2af912061a8b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13994
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-08-19 11:06:31 +00:00
Artur Paszkiewicz
0f99700db9 ftl: user write limits
Calculates general priorities and trigger points for writers
(gc and compaction) dependent on number of free bands. GC will
be started at SPDK_FTL_LIMIT_START level, while at SPDK_FTL_LIMIT_CRIT
compaction needs to be stopped and only GC is allowed to work.
This is done to make sure FTL doesn't run out of free bands and deadlock
itself.

Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Change-Id: I1aab98503c2e79e97f8e4e9fb1257530fa9770e2
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13326
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-08-18 19:09:50 +00:00
Kozlowski Mateusz
1bbefed63b FTL: Remove leftover ZNS code
Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Change-Id: Ica358805a69582d78e0d6c4f17b5a97ff38e44ca
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14112
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-08-18 19:09:50 +00:00
paul luse
c746637df8 lib/idxd: add some flag overrides when doing PMEM writes
Per upcoming specification changes.

Fixes: 2486

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Ic2534148a87b3dec7512f7b01384f484fee4c30f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13572
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: <wayne.gao@intel.com>
2022-08-18 18:47:02 +00:00
Jim Harris
2508d170c8 sock: make opts structures packed
This ensures that when fields are added, that the
size of the structure will change, ensuring different
versions of the structure can be detected using
sizeof.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ia30d57b25b559a89997963a3f68797ff3b9d6409
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14093
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2022-08-18 10:08:40 +00:00
Jim Harris
1ef258104d nvmf: make opts structures packed
This ensures that when fields are added, that the
size of the structure will change, ensuring different
versions of the structure can be detected using
sizeof.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ifdbd46d3f3f2ddb9c422eeca5e3f487fd056438f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14092
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-08-18 10:08:40 +00:00
Jim Harris
92335c01cf event: make opts structures packed
This ensures that when fields are added, that the
size of the structure will change, ensuring different
versions of the structure can be detected using
sizeof.

Adding -Wno-address-of-packed-member to Makefiles
here, although we should consider disabling this
warning globally in SPDK just like DPDK.

Suppress abidiff errors around spdk_app_opts -
structure size and offsets of all existing members
were unchanged, so there is no ABI breakage here.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I2249eddb604d7b44180cadb92ba30edcd946b9bd
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14091
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2022-08-18 10:08:40 +00:00
Jim Harris
a4ec8e6a60 blob: make opts structures packed
This ensures that when fields are added, that the
size of the structure will change, ensuring different
versions of the structure can be detected using
sizeof.

Suppress abidiff errors around spdk_bs_opts - structure
size and offsets of all existing members were unchanged,
so there is no ABI breakage here.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I2b109e0787446a598eee413d1595a68b4c87f830
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14090
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2022-08-18 10:08:40 +00:00
Jim Harris
98e3a5d3ae bdev: make opts structures packed
This ensures that when fields are added, that the
size of the structure will change, ensuring different
versions of the structure can be detected using
sizeof.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: If346cb6628b3a288a3ac720104b501589eba3b83
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14089
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2022-08-18 10:08:40 +00:00
Jim Harris
b801af090a nvme: add disable_read_changed_ns_list_log_page
Similar to the disable_read_ana_log_page ctrlr_opt,
this enables the application to tell the NVMe
driver to *not* read the CHANGED_NS_LIST log
page in response to a NS_ATTR_CHANGED AEN, and
will do the read itself.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ie447734187d4a4cb95ceef6e0131b640b8ba5984
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14088
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2022-08-18 10:08:40 +00:00
Jim Harris
57e90e4378 nvme: make opts structures packed
This ensures that when fields are added, that the
size of the structure will change, ensuring different
versions of the structure can be detected using
sizeof.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I7d8e71531cfd8823f1594149217b841ecf490e3e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14087
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Community-CI: Mellanox Build Bot
2022-08-18 10:08:40 +00:00
Jim Harris
c50cb569de include: add STATIC_ASSERTS for opts structures with size member
Various opts structures in SPDK have a size member, to enable
ABI compatibility should fields be added in the future.

But this requires the strucures to be packed, otherwise for
example a structure may be padded at the end, and a new
field added may just consume some of that padding.

So add STATIC_ASSERTS for the current sizes in this
patch.  Upcoming patches will make the structures packed
and add in reserved fields to fill in holes.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I9107d01d7b533f8542385a3538894bcd9f8c465d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14086
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Community-CI: Mellanox Build Bot
2022-08-18 10:08:40 +00:00
Kozlowski Mateusz
e8c5ccf039 FTL: Add write path
Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Change-Id: I41985617b5879bd3f4bf6d49d2a03eaffdd5ccb5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13322
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-08-18 08:34:47 +00:00
Ziv Hirsch
eda407a6f0 nvme: add support for verify command
Signed-off-by: Ziv Hirsch <zivhirsch13@gmail.com>
Change-Id: Ic9859d5078d9568bb28eefcf8fb70a7fc222ee15
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13928
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
2022-08-16 10:25:01 +00:00
Jim Harris
a6b7e1839d nvme/tcp: add trace points for cmd submit/complete
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iad56e7a96cf0210bcf54825c8bcc39af9366b72c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13992
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
2022-08-16 10:23:10 +00:00
Jim Harris
b0396da090 nvme/pcie: rename trace object to NVME_PCIE_REQ
We were using "TR" for "tracker" previously, but
we are tracing the nvme_requests, not nvme_trackers,
so use the right names for the trace object to avoid
confusion.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ia3886d74b162138c2cdbe0017224d9494f74966c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13990
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-08-16 10:23:10 +00:00
Ben Walker
081f080a49 accel: Rename public header to accel.h
The public interface of lib/accel is now include/spdk/accel.h

Change-Id: Id94f623a494eb1b524b060f4413f633073ea7466
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13916
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2022-08-16 10:22:55 +00:00
Ben Walker
10ab81b83e accel: Hide the definition of accel_io_channel from modules
They no longer need to see the definition of this structure.

Change-Id: I3e3bb5942a50da22e0bf34aa8c10b9d812f42d2f
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13915
Community-CI: Mellanox Build Bot
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-08-16 10:22:55 +00:00
Ben Walker
678025c914 accel: Move the software module to its own file
This will help keep the mixing of this code with the framework code to a
minimum.

Change-Id: I5937ebd84f32068456cdf2b9e03d3e194c760a87
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13912
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-08-16 10:22:55 +00:00
Ben Walker
6074b3a3f9 accel: Move definitions not needed by modules to accel_internal.h
spdk_internal/accel_engine.h will become the API for accel modules. Move
anything in there that a module doesn't need to see into
lib/accel/accel_internal.h

Some of the software fallback definitions didn't even need to be in a
header and were moved to accel_engine.c

Change-Id: Idb8b12b1c0c1de3d462b906e3df3ba9ee8f830b8
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13911
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2022-08-16 10:22:55 +00:00
Ben Walker
aa156d53be accel: Combine spdk_accel_engine and spdk_accel_module_if
These are 1:1 - they do not need to be separate objects.

Change-Id: I74ab52863f911d9be59ce98e1525302b5bd40846
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13910
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2022-08-16 10:22:55 +00:00
Boris Glimcher
6212597bda sock/ssl: Add psk_key and psk_identity options to spdk_sock_impl_opts
Note, this change only sets defaults for the ID/KEY,
more specific use cases like NVMe/TCP may set the ID and KEY on a per connection basis.

Also simplify PSK identity string, that isn't NVMe focused.
NVMe libraries using this will need to construct more complicated
identity strings and pass them to the sock layer.

Example:
  rpc.py sock_impl_set_options -i ssl --psk-key 4321DEADBEEF1234
  rpc.py sock_impl_set_options -i ssl --psk-identity psk.spdk.io

  ./build/examples/perf --psk-key 4321DEADBEEF1234 --psk-identity psk.spdk.io

  ./build/examples/hello_sock --psk-key 4321DEADBEEF1234 --psk-identity psk.spdk.io

Change-Id: I1cb5b0b706bdeafbccbc71f8320bc8e2961cbb55
Signed-off-by: Boris Glimcher <Boris.Glimcher@emc.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13759
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2022-08-15 16:52:28 +00:00
Jim Harris
21d9b32fa3 trace: remove number of args from comment
This avoids having to update this comment should
we increase the number of arguments supported.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I67591ce96433f38a2463ec462151b3ce0ae44e53
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13987
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2022-08-12 14:18:05 +00:00
Shuhei Matsumoto
f1dabc228f nvme: Add SPDK_NVME_TRANSPORT_CUSTOM_FABRICS for custom fabric transport
There was SPDK_NVME_TRANSPORT_CUSTOM but it was not usable for fabric
custom transport because spdk_nvme_trtype_is_fabrics() returned false.

Use 4097 for SPDK_NVME_TRANSPORT_CUSTOM_FABRICS and update
spdk_nvme_trtype_is_fabrics() to include SPDK_NVME_TRANSPORT_CUSTOM_FABRICS
because it is not defined in the specification.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I457845e74d4e8150c9376f9bc253a8ee1ad8c1d3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13947
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2022-08-12 08:59:52 +00:00
Shuhei Matsumoto
b5f360c425 rdma: Maintain per device PD which is persistent across reconnect
SPDK NVMe RDMA initiator used the default PD per RDMA device. Default PD
may be changed when all QPs for the RDMA device are destroyed and created
again.

For multipath, the RDMA zero copy feature require the PD per RDMA device
to be persistent when all QPs for the RDMA device are destroyed and
created again.

Maintain such persistent PDs in this patch.

Add two APIs, spdk_rdma_get_pd() and spdk_rdma_put_pd().

In each call of two APIs, synchronize RDMA device list with
rdma_get_devices().

Context may be deleted anytime by rdma-core. To avoid such deletion,
hold the returned array by rdma_get_devices().

RDMA device has PD, context, ref. count, and removed flag. If context
is missing in rdma_get_devices(), set the removed flag to true. Then,
if the ref count becomes zero, free the PD and the RDMA device.

The ref. count of a RDMA device is incremented when spdk_rdma_get_pd()
is called and decremented when spdk_rdma_put_pd() is called.

To simplify synchronization, sort the returned array by
rdma_get_devices().

To avoid resource leakage, add destructor function and free all PDs
and related data at termination.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I093cb4ec2c7d8432642edfbffa270797ccf3e715
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13769
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2022-08-12 08:59:43 +00:00
Ben Walker
32ee475a5e accel: SPDK_ACCEL_MODULE_REGISTER is now passed the module
Instead of passing each parameter to create a module, just have the user
make one and pass it in. This makes it easier to change the module
definition later.

Change-Id: I3a29f59432a6f0773129d7b210fbc011175b2252
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13909
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-08-10 11:00:17 +00:00
paul luse
3d5fd5a59f lib/idxd: fix bugs with IAA decompression descriptor construction
Masked by how accel_perf was doing decomp verificiation which is
changed in the next few patches and verifies these fixes.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Icb03fc169bf8d2f05396addaf1db56d6de1827d1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13038
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-08-10 07:25:29 +00:00
paul luse
efa33b8590 lib/accel: add RPC to enable override of opcode to engine
Docs explaining how to use the RPC are in the next patch in the
series.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I7dab8fdbeb90cdfde8b3e916ed6d19930ad36e66
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12848
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-08-10 07:25:29 +00:00
Evgeniy Kochetov
3dd0bc9e09 nvme: Add transport controller ready step
This step allows custom transports to perform extra actions or checks
at controller initialization and fail initialization if required.

Signed-off-by: Evgeniy Kochetov <evgeniik@nvidia.com>
Change-Id: Ic7cadae5398a35903917ceace3828f4371be63a3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12631
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-08-04 07:29:03 +00:00
paul luse
44cbea402e lib/accel: Add new RPC to get valid engine info.
The RPC provides a list of initialized engine names along with
that engine's supported operations.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I59f9e5cb7aa51a6193f0bd2ec31e543a56c12f17
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13745
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2022-08-03 07:53:21 +00:00
paul luse
c6ecddcc1c lib/accel: add RPC to get list of OP codes per module
In prep for upcoming patch that will provide an RPC to override
and automatic assignment of an op code to an engine.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I17d4b962fb376a77f97ce051a513679d0fba698e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12829
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-08-03 07:53:21 +00:00
Artur Paszkiewicz
884980d0aa ftl: vss null buffer workaround
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Change-Id: I94ea399ed30fae29f92b4216eaa9209c02b3478b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13310
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-08-02 19:00:42 +00:00
Artur Paszkiewicz
06790f25f1 FTL: Add ftl_io helper structure
Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Change-Id: I608b500c6fb14efe289932955f508484f2ecf1b6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13305
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-08-02 19:00:42 +00:00
Jacek Kalwas
b17919d8bc lib/nvmf: add lock around few transport ops
this is a prework for further changes - with lock on generic layer
lock on specific transport (e.g. tcp, rdma) layer becomes optional

possibly it won't be required if some contract introduced on public
interfaces (to be considered)
- spdk_nvmf_poll_group_[create|destroy]
- spdk_nvmf_tgt_listen_ext, spdk_nvmf_tgt_stop_listen
- spdk_nvmf_get_optimal_poll_group

Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: Ib132babf9e7022342129fe795991cdad834e7f53
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13665
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-07-29 16:34:41 +00:00
MengjinWu
7fc2c0856e lib/nvmf: use DSA to offload recv data digest crc32 in nvmf-TCP
allow DSA device to async offload crc32 calculation in nvmf-TCP

This patch can use DSA to accelerate crc32 computation, making
the io performance of TCP paths using crc32 approach the io
performance of TCP paths that do not use crc32.

Using SLIST to minimize the performance drop. SLIST has less
operation compared to TAILQ.

Thinking about memory thrashing, we should use the same memory as
possible to receive new PDUs. So, insert newly freed PDU in to head
is better.

The performance drop is within 1% compared to the TCP path without
crc32.

Signed-off-by: MengjinWu <mengjin.wu@intel.com>
Change-Id: I480eb8db25f0e730cb198ca5ec19dbe3b4d38440
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11708
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: GangCao <gang.cao@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-07-27 08:39:35 +00:00
Artur Paszkiewicz
c682c78992 FTL: Add FTL bdev module
Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Change-Id: I8c40b96f0726d83d6a307e8b9a04b7c210b80255
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13299
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-07-25 07:19:29 +00:00
Boris Glimcher
806744b7c8 sock: Add ktls and tls_version to spdk_sock_impl_opts
Since `sock_impl_opts` was added to `sock_opts`
Can remove `ktls` and `tls_version` from spdk_sock_opts

Example:
  rpc.py sock_impl_set_options -i ssl --enable-ktls
  rpc.py sock_impl_set_options -i ssl --disable-ktls
  rpc.py sock_impl_set_options -i ssl --tls-version=12

  ./build/examples/perf --enable-ktls
  ./build/examples/perf --disable-ktls
  ./build/examples/perf --tls-version=12

Check kTLS statistics here: /proc/net/tls_stat

Change-Id: Icf7ee822bad92fda149710be77feb77fc8d4f163
Signed-off-by: Boris Glimcher <Boris.Glimcher@emc.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13510
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-07-22 06:41:39 +00:00
Artur Paszkiewicz
d974bad6fc ftl: retrieve device’s attributes and configuration
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Change-Id: Ide6bb24d2c1ec2b0da3f20ce4013a4cd6e339114
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13297
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2022-07-21 10:53:01 +00:00
Artur Paszkiewicz
e49ccfc820 ftl: device startup and shutdown
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Change-Id: Ia4a3439a2ac79e24bc6dc11a5c131d44ecb2ad80
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13291
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2022-07-21 10:53:01 +00:00
Changpeng Liu
c650b7eae3 bdev: increase metadata buffer size to 64
Many SSD drives can support up to 64 bytes of metadata
size, so here we define a macro in bdev.h and set the
default size to 64.

Change-Id: Ie30831d61cd66b8f5904daa38101d1a77141214e
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12355
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-07-19 12:31:59 +00:00
Konrad Sztyber
7f83361553 sock: add sock_impl_opts to sock_opts
Some of the options in sock_impl_opts could be different for different
sockets (even if they're using the same impl).  However, outside of a
few selected options (recv_buf_size, send_buf_size), there was no
interface to change them.

This change will allow users to change impl_opts on a per-socket basis
when creating a socket.  Sockets created through accept() inherit
impl_opts from the listening socket.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I7628ae19def25cef6ffa62aa54bd34e446632579
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13661
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
2022-07-19 09:35:03 +00:00
Konrad Sztyber
cfe2d76db2 sock: remove zerocopy_threshold from spdk_sock
Now that spdk_sock has impl_opts, we no longer need to store a copy of
impl_opts.zerocopy_threshold in spdk_sock.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I96377e330351b1afb57811578acfadf05d53f49c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13660
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
2022-07-19 09:35:03 +00:00
Konrad Sztyber
50afaf1ee8 sock: store impl_opts in socket structure
It'll make it possible to change some of the impl_opts options on a
per-socket basis, as well as make it easier to use fields common to all
implementations in the generic layer.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Id3b5e0a0b302fdecc2387d07fb87b75b487dc5c5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13659
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-07-19 09:35:03 +00:00
Konrad Sztyber
466f9e8b20 sock: extract prepping iovs for a single req to a function
It'll make it possible to reuse this code for asynchronous read
requests.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I2c2c44feee0821c972aa2a43d8a8ec81f3ce4ef4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12591
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
2022-07-14 09:45:54 +00:00
Konrad Sztyber
f34ce6fb07 sock: add spdk_sock_request_complete
Extracted the code that completes requests to a separate function.  This
will make it possible to use it with read requests, which aren't using
request queues.

Change-Id: I101f36bab6def753d548d766acd17020b504873e
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12173
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-07-14 09:45:54 +00:00
Konrad Sztyber
3e47d7fa22 sock: asynchronous readv interface
This patch defines a new function, spdk_sock_readv_async(), which allows
the user to send a readv request and receive a callback once the
supplied buffer is filled with data from the socket.  It works simiarly
to asynchronous writes, but there can only be a single outstanding read
request at a time.

For now, the interface isn't implemented and any calls will return
-ENOTSUP.  Subsequent patches will add support for it in the uring
module and as well as emulation in the posix module.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I924e2cdade49ffa18be6390109dc7e65c2728087
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12170
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-07-14 09:45:54 +00:00