The cache needs to be scrubbed during the initial device creation as
well as after power loss recovery. This patch extracts the scrubbing
code into a separate function.
Change-Id: I2cb32e6993a3531470f29f466d990f0d96e45def
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459621
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
The header is being written from multiple places, so having a discrete
function serializing and writing it at the appropriate place in the
cache makes sense.
Change-Id: I7a1e6ebd05e8a4974d141f04202803f507b978e4
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459620
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
This patch adds a function that marks all active open bands to be
flushed and once all of them are closed it notifies the caller. This
needs to be done before the data from non-volatile cache can be scrubbed
to make sure its stored on bands the device can be restored from.
Change-Id: I9658554ffce90c45dabe31f294879dc17ec670b9
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459619
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This patch makes sure we're on the thread that requested creation /
deletion of the device when calling the notification callback.
Change-Id: Ia11a8054692874f6b57d4ebe3e3cb290c58e83b6
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459618
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Added ftl_dev_has_nv_cache to check if the FTL is configured to use
non-volatile cache or not. It makes these checks a bit more readable.
Change-Id: I0140df184d89a675e40bd5056718cd64301c553e
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459617
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Use the data placed on the non-volatile cache to perform recovery in
case the device wasn't shut down cleanly. The write phase ranges are
read and their data is copied onto the OC device.
The code added in this patch will correctly copy the data from
overlapping ranges, however it won't do anything about these overlapping
areas, so subsequent power loss happening quickly after recovery might
result in data loss.
Change-Id: Ib4c66092cee858496ec66f789fcfb1e7e32f5c20
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458105
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Scan the cache to find ranges of blocks written with the same phase.
This prepares the structures needed to perform data recovery from the
non-volatile cache.
Change-Id: I0c901d010d6ca76feabca13116d831c1d9931833
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458103
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
This patch adds tracking of the phase of the writes to the non-volatile
cache. The phase is changed each time the whole buffer is filled. Along
with every block's LBA, current phase is stored in its metadata. This
allows for replaying the sequence of writes when recovering the data
from the cache after (unclean) shutdown.
Since there are only three possible phases to be stored on the device at
a time, phase is defined as a 2-bit counter cycling through 1 -> 2 -> 3
-> 1, with 0 marking blocks that were never written.
Change-Id: Id47880367934027fd102c32f183110acc9d4c62a
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458098
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
After filling whole non-volatile cache, block all further writes until
the header with metadata is written. This means that metadata stored on
the device will always be up-to-date with the most recent write
sequence.
Change-Id: I15b724b52814289622374ce77e5c3b23173a75c6
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458097
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
When creating FTL device using non-volatile cache, zero out the
non-volatile cache and store metadata (device's UUID, size of the cache)
in the first block.
Change-Id: Id8f212aef756e86e8a215582ab7c32a635e18938
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458094
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
In case ANM event occurs on open band there
can be situation that reloc will try to read
block on which there is ongoing write.
This is happening because lba valid map is
updated before write submission to allow sent
consistent metadata to disk before all user
writes are completed.
Added write offset to the each chunk and
add check to reloc if particular ppa is written
on that chunk.
Change-Id: Ic95a06e69381d2152a86984b65a0975afaff955d
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458056
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Initialize the memory pool for storing metadata (LBAs) when writing data
to the non-volatile cache. The mempool's object count and size can be
configured via nv_cache.max_request_cnt / nv_cache.max_request_size
respectively.
Change-Id: I376df9a75be13d4b29ba475f350edf402c868d48
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458092
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Address translation wasn't correct for >32 bit length packed address.
This commit fixes the issue and adds a corresponding unit test.
This patch fixes issue #774:
https://github.com/spdk/spdk/issues/774
Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: Idce67c47f2a9888f9e2ae2eadaf71ccc34e5c260
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457114
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Keep track of read lba map segments to avoid
unnecessary traffic in case ANM events.
Lba map is divided on 4KB segments which can
store 512 lba entries.
In case multipe read request on same segment
keep pending request list and process it in
read_lba_map() completion callback.
Change-Id: I2661bdb716ab7c975140e0b37aebcb17aa23901d
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453371
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
The patch also changes, so the command calculates the needed
chunk offset based on PPA, instead of relying on the user to
provide one.
Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: Ic4eec1b86ded4eb71de860015403294ed0c8c266
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455973
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Direct wptr allows for writing to band with externally
provided PPA, rather than one assigned by the wptr. This
can be useful when padding open bands during recovery or
bdev creation, as ftl can find partially written end meta
with no way to close the band with valid data, that would
conform to an already written CRC value.
This does mean that I/O sender is responsible for calculating
and obeying per-chunk write pointer arithmetics.
Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: Ifcbc4e8ffb69c829469a571bed8e27b148d7a651
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455511
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
There's no point in synchronously returning an error from ftl_io_write
/ ftl_io_read as they're also reported in the IO's completion callback.
All errors are now reported through io->status.
Change-Id: I4adf4e13221b63715625276042e1172cd63b8f9b
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455521
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Added means to configure libftl to use (optionally) another bdev as
persistent write buffer cache.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I97028a681be168d9386eac8a226631ff772f803b
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/448629
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Added internal retry IO queue. In case ENOMEM from
nvme layer request is put to retry queue. When some
inflight operations are completed try to make progress
on IOs in retry queue in first place.
Change-Id: Ie9d3d20bd34431ee57f9454f242b0cdca349c804
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/448461
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
FTL should pass IO errors to upper layer only in
completion callback to be consistent on sync and
async error paths.
Change-Id: I29c81af477b476c1e10b7933126737a222553ffa
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/447176
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This patch adds supplementary tracing for OCSSD library to allow
for more efficient debugging and profiling of the user I/O path
as well as the background tasks (maintaining write pointer,
defragmentation, ANM event handling, etc.).
Change-Id: I741f1304f4ee0eba019e31bea7814af475c3296e
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/431327
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
The patch adds support for retrieving asynchronous media management
events sent by the drives. SSDs needs to inform the host of various
incidents concerning the data placed on the media. The events include
wear-leveling, read disturb, and background data refresh.
In order to poll for this asynchronous events, a separate thread is
added that polls for completion of admin commands for each of Open
Channel disk and dispatches the ANM events to the appropriate block
devices.
Change-Id: Ic5d4943614c65a9d8849c500826841ae21d4c9c5
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/431324
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This patch implements module responsible for data relocation due to
defragmentation, wear-leveling, read disturb and/or background data
refresh.
The submodule supports moving whole bands, chunks, as well as
single logical blocks.
Change-Id: Iaef37f50e0bb93986a21e495174f6d5ef8a1f30d
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Signed-off-by: Jakub Radtke <jakub.radtke@intel.com>
Reviewed-on: https://review.gerrithub.io/c/431323
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This patch adds core FTL (flash translation layer) submodules.
On regular SSDs the FTL is part of firmware, whereas Open Channel
enables moving it to the host and allows for having full controll
of data placement on the device.
Main functionalities added:
* logical to physical address map
* read / write buffer cache
* wear-leveling
* bad block management
Change-Id: I5c28aa277b212734bd4b1f71ae386b3d6f8c3715
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Signed-off-by: Jakub Radtke <jakub.radtke@intel.com>
Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/431322
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This patch introduces core structures required for implementing FTL on
top of Open Channel drives. The Open Channel specification describes raw
access to the media on the SSD. The FTL consumes that API and exposes a
block device interface.
The implementation is based on the revision 2.0 of the Open Channel SSD
specification.
Change-Id: Ie306cdfb7920df3b02233fcb60896745f3184cdc
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/431321
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
