L2P cache allows for partial storing of L2P in memory, paging in and out
as necessary, lowering the total memory consumption.
Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Change-Id: I727fec9d2f0ade4ca73e872d62a2ec10cfdb0a88
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13353
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Adds P2L validity map tracking - a bitmap marking all physical LBAs
as containing valid (current) user data or not. A clear bit denotes
the location has no valid data and may be skipped during relocation
or compaction. A set bit means it may have valid data (it's still
necessary to do the necessary comparision against L2P).
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Change-Id: I6a831a97b3080eb7c880d9c4feab41b523467885
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13350
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Adding API for the bringup part of fast shutdown/startup. Adds
shared memory utilization for necessary functions during initialization.
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Change-Id: Iab2da102fd0ccaa56fbdb9b3c765be5eeefff145
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13349
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Adds API for fast shutdown - the ability for FTL to skip most
of the metadata persists made during clean shutdown, and relying
on their representation in shared memory instead. This allows for
faster update of SPDK (or just FTL, assuming no metadata changes),
with downtime reduction from 2-5 seconds to 500-1000 ms (for
14TiB+800GiB base and cache drives).
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Change-Id: I5999d31698a81512db8d5893eabee7b505c80d06
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13348
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Helper functions which determine which md regions will be stored
in shm.
Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Change-Id: I94cbfca66dfb56457a350874dbd1de63a2e07661
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14159
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Allows for using shared memory in memory pools. Adds API for
accessing such pools after dirty shutdown (claiming them, ie.
marking an entry as actively used; calling the
ftl_mempool_initialize_ext will reclaim all unused entries back
to the pool). Also introduces API for accessing objects, since
using direct pointers is not possible (as addresses may change
inbetween application startups).
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Change-Id: I5325b39d68aef7e231945cee9d92c925cab2fb2a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13346
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Main use case is to allow for keeping it in shared memory, to
speed up the recovery time after application crash.
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Change-Id: I36b6b8331cd6483c5bd202e5f9103c351d705da8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13345
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
During compaction FTL moves valid user data from the nv cache drive to
the bottom device.
Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Change-Id: Ia200af39cec80014fac3a10f20d2859b10a81088
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13337
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Calculates general priorities and trigger points for writers
(gc and compaction) dependent on number of free bands. GC will
be started at SPDK_FTL_LIMIT_START level, while at SPDK_FTL_LIMIT_CRIT
compaction needs to be stopped and only GC is allowed to work.
This is done to make sure FTL doesn't run out of free bands and deadlock
itself.
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Change-Id: I1aab98503c2e79e97f8e4e9fb1257530fa9770e2
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13326
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
