Commit Graph

53 Commits

Author SHA1 Message Date
paul luse
a6dbe3721e update Intel copyright notices
per Intel policy to include file commit date using git cmd
below.  The policy does not apply to non-Intel (C) notices.

git log --follow -C90% --format=%ad --date default <file> | tail -1

and then pull just the 4 digit year from the result.

Intel copyrights were not added to files where Intel either had
no contribution ot the contribution lacked substance (ie license
header updates, formatting changes, etc).  Contribution date used
"--follow -C95%" to get the most accurate date.

Note that several files in this patch didn't end the license/(c)
block with a blank comment line so these were added as the vast
majority of files do have this last blank line.  Simply there for
consistency.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Id5b7ce4f658fe87132f14139ead58d6e285c04d4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15192
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
2022-11-10 08:28:53 +00:00
Artur Paszkiewicz
8a76d5500d ftl: I/O tracing
Adds tracepoints in FTL.

Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Change-Id: I661703e42b8b531822a2ba74a09cdc716daa1c46
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13391
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-09-20 19:24:26 +00:00
Kozlowski Mateusz
66fe5f75bb ftl: Unmap functionality
Adds ability to send trim commands to FTL - only 4MiB aligned requests (both
for offset and length of request) will be processed. During a trim
operation an L2P page (containing 1024 4B entries, 1 per user LBA; which
is where the 4MiB alignment comes from) will be marked as unmapped.
After this point any L2P access to that page will actually set the
entries themselves as FTL_ADDR_INVALID. This is done to make the trim as
fast as possible, since for large requests it's probable that most of
the L2P pages aren't actually in DRAM.

Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Change-Id: I4a04ee9498a2a6939af31b06f2e45d2b7cccbf19
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13378
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-09-09 19:44:29 +00:00
Kozlowski Mateusz
8c519d31bd FTL: Add internal band state changes
Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Change-Id: Icaecc4e77996919a23f70c1ffad15b783741fd5e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13327
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-08-18 19:09:50 +00:00
Kozlowski Mateusz
a68a12a478 FTL: Initial nv cache structure
Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Change-Id: Ie40cc25ed9bf28976a5ae6d6a67491f438152fca
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13317
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-08-12 09:01:40 +00:00
Artur Paszkiewicz
b16bdc6d49 FTL: Add L2P API and flat L2P implementation
Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Change-Id: Ifadc8c6986164584235ee6a67799025fa7703b5d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13315
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-08-12 09:01:40 +00:00
yidong0635
5daedcc22e ftl: Fix compile warning.
Issue reports:

spdk/lib/ftl/ftl_io.c:121:9: warning: variable ‘result’
set but not used [-Wunused-but-set-variable]
  121 |  size_t result;
      |         ^~~~~~

Signed-off-by: yidong0635 <dongx.yi@intel.com>
Change-Id: I9ed7daea97f311ca33c4116299be32f275e33fbb
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13838
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: GangCao <gang.cao@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2022-08-03 07:42:38 +00:00
Artur Paszkiewicz
d9a631ad4c FTL: Add io channel logic
Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Change-Id: Ibf6bfbabc03c43e7938531c4fe08fde01ce02a3f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13307
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-08-02 19:00:42 +00:00
Artur Paszkiewicz
06790f25f1 FTL: Add ftl_io helper structure
Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Change-Id: I608b500c6fb14efe289932955f508484f2ecf1b6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13305
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-08-02 19:00:42 +00:00
Wojciech Malikowski
81dca28884 ftl: remove deprecated ftl library
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Change-Id: I3ebb05be3f1b9864b238cb74f469b4fdf573cd0d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11120
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
2022-07-11 07:23:58 +00:00
Ben Walker
8dd1cd2104 check_format: For C files only, fix return type breaks
In SPDK, declarations have the return type on the same line. Definitions
have the return type on a separate line. Astyle has an option for
enforcing this. Unfortunately, it seems to have two bugs:

1) It doesn't work correctly at all on C++ files.
2) It often fails on functions that return enums, or long type names

Deal with 1) by adjusting the check_format.sh script to only tell astyle
to fix return type line breaks for C files and not C++. Deal with 2) by
adding a few typedefs to work around the problem.

Change-Id: Idf28281466cab8411ce252d5f02ab384166790c6
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13437
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
2022-06-27 09:33:48 +00:00
Jim Harris
488570ebd4 Replace most BSD 3-clause license text with SPDX identifier.
Many open source projects have moved to using SPDX identifiers
to specify license information, reducing the amount of
boilerplate code in every source file.  This patch replaces
the bulk of SPDK .c, .cpp and Makefiles with the BSD-3-Clause
identifier.

Almost all of these files share the exact same license text,
and this patch only modifies the files that contain the
most common license text.  There can be slight variations
because the third clause contains company names - most say
"Intel Corporation", but there are instances for Nvidia,
Samsung, Eideticom and even "the copyright holder".

Used a bash script to automate replacement of the license text
with SPDX identifier which is checked into scripts/spdx.sh.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iaa88ab5e92ea471691dc298cfe41ebfb5d169780
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12904
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: <qun.wan@intel.com>
2022-06-09 07:35:12 +00:00
Konrad Sztyber
336505abbf lib/ftl: use per-io_channel write buffers
Replaced single global write buffer with the per-io_channel write
buffers.  This means that the "rwb" module and all of its references
were removed and replaced with the recently added interfaces.

Change-Id: Idc899d3a4d63a8a2bede1ac26549ed06e9a2e784
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/909
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
2020-03-02 10:27:06 +00:00
Konrad Sztyber
085bfa0d13 lib/ftl: allocate separate memory for IO channels
This is the first patch of a series that replaces single global write
buffer with per-io_channel buffers.  This change is intended to improve
performance for multithreaded workloads.

This patch changes the way the ftl_io_channels are allocated by only
keeping an ftl_io_channel pointer inside spdk_io_channel's context.  It
allows for delaying IO channel destruction, which in turn allows the FTL
to iterate over all exisiting IO channels without locking.

Change-Id: I5e0cab8043a2b5f747e971dd3d65ed2546c8cf26
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/900
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-03-02 10:27:06 +00:00
Konrad Sztyber
f4e42560b4 lib/ftl: use parent's iovecs during child initialization
Fill in the child's iovec during its initialization by calculating it
based on the parent's iovecs and the request's block offset.

Change-Id: I61a1d5cf8a1ceae71e996d50db31077c43c4e3d9
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/898
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Maciej Szczepaniak <maciej.szczepaniak@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-02-20 09:53:03 +00:00
Konrad Sztyber
4a3bf3a942 lib/ftl: use user's iovec instead of copying it
There's no need to copy user's iovec, as it's bound to be valid until
request's completion.  Storing a pointer to user's iovec is both faster
and simplifies the IO allocation path.

Change-Id: Ia5b37fcbc230edac6f52989c8ef09655af8ec178
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/897
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Maciej Szczepaniak <maciej.szczepaniak@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-02-20 09:53:03 +00:00
Konrad Sztyber
7f45a12636 lib/ftl: use iovecs instead of single data pointer
Allow ftl_io initialization with a iovec array instead of a single data
pointer.  It'll make it possible to use the vector version of the IO
commands.

Change-Id: I34c1e398cea681f59321605e081366785aa29128
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/894
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Maciej Szczepaniak <maciej.szczepaniak@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-02-20 09:53:03 +00:00
Wojciech Malikowski
39965ab048 lib/ftl: Change "lbk" name to "block"
"lbk" name is more OCSSD specific so in
generic FTL "block" name is more suitable.

Change-Id: I792780297b792bf5e02f13cc20346da56b032918
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/472284
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-17 08:59:08 +00:00
Wojciech Malikowski
b48113b296 lib/ftl: Update lba map during write completion
Lba map could be updated during write
completion. This change is needed to have
common io path when append support will
be added.

Change-Id: I942bfd8b54dc6b40136ca53434f0ef8e3c415c5b
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/471637
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2020-01-17 08:59:08 +00:00
Wojciech Malikowski
d6234332fb lib/ftl: Represent ftl_addr just with the offset value
This patch removes parallel unit and zone id fields
from ftl_addr struct.

Change-Id: Ica42208eafdbbecab60430708197f5f3ba6167d6
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/468701
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-17 08:59:08 +00:00
Wojciech Malikowski
843f296e2e lib/ftl: Replace Open Channel dependencies with zone bdev API
This patch replaces NVMe Open Channel API usage
inside FTL library with corresponding zone bdev
API calls. This include following calls:
 - spdk_nvme_ctrlr_cmd_get_log_page -> spdk_bdev_get_zone_info
 - spdk_nvme_ocssd_ns_cmd_vector_reset -> spdk_bdev_zone_management
 - spdk_nvme_ns_cmd_read -> spdk_bdev_read_blocks
 - spdk_nvme_ns_cmd_write_with_md -> spdk_bdev_write_blocks

Change-Id: I1b5a6863d9ce72f4af1cfbb0e449fc1a5b638144
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/479702
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2020-01-15 12:24:44 +00:00
Wojciech Malikowski
e47e16d3a4 lib/ftl: Replace ftl_ppa struct with ftl_addr
FTL working on top of zoned bdev doesn't need
physical page address (PPA) anymore.
ftl_ppa was replaced with ftl_addr which represents
zoned device addressing schema.

Change-Id: Ied5750a7ab2f4ce42067ff3e69c1f26f85f5022a
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/467633
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
2020-01-10 08:42:51 +00:00
Wojciech Malikowski
748785c2a9 lib/ftl: Change ftl_ppa fields to be zone specific
Remove some Open Channel dependencies from ftl_ppa struct.

Change-Id: Ic088b84c56a928906c6c01c6ef74e69be6d0107f
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/479549
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-10 08:42:51 +00:00
Mateusz Kozlowski
0cb8a3319c lib/ftl: Update initialization of io channel
Changed initialization of io channels to be made in the core/read
thread themselves. This fixes potential hangs when using the ftl lib
directly, without bdev and initializing an ftl device from a different
thread than will be used for IO.

Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: I7089cc46cc6de5e9008e7c22cc01d2b4103ebb14
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/464891
Reviewed-by: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-08-12 21:37:13 +00:00
Tomasz Zawadzki
06c7ef7f6a lib/ftl: check for io not being NULL before dereferencing
ftl_io_free() handles NULL io being passed, but shouldn't
dereference it before returning.

Change-Id: Ia705d4ab9f8b3569e9704a1f97e70f7579084efb
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462487
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-07-23 19:55:30 +00:00
Mateusz Kozlowski
3500d9a98d lib/ftl: Add error log after IO failure
Adding some extra debug information for any failing IOs.

Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: If38de30bcb80d291778ad7325a33b7786d02390d
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/461588
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-18 08:49:02 +00:00
Konrad Sztyber
9a42d7fc30 lib/ftl: initialize LBA when allocating internal IOs
Initialize children IOs with the appropriate LBA of its parent when
allocating internal IOs.

Change-Id: I191ad741b9d88d7f18cae05982e0a06a8f371f78
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458099
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
2019-07-12 12:39:38 +00:00
Richael Zhuang
6708b70938 lib/ftl: fix the unclear PAGE_SIZE
In lib/ftl PAGE_SIZE is used in many places. But I think there's no relation with
the host memory PAGE_SIZE.
In most changes, PAGE_SIZE is replaced by FTL_BLOCK_SIZE for ftl block size.
In ftl/ftl-anm.c, PAGE_SIZE is replaced by 4096 for alignment.

Change-Id: I72e7c65c83b9fe1a4e50944dc5d90b9459b4e593
Signed-off-by: Richael Zhuang <richael.zhuang@arm.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455347
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-06-11 18:12:27 +00:00
Wojciech Malikowski
a1f1581968 lib/ftl: Remove FTL_IO_KEEP_ALIVE flag from ftl_io_flags
FTL_IO_KEEP_ALIVE flag is deprecated.

Change-Id: Id074f45a3e9ecb6d05c6a88a834476e2c19288a3
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/454326
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-06-11 06:21:46 +00:00
Wojciech Malikowski
4769f058e0 lib/ftl: Stop using ftl_io for internal relocation logic
Added ftl_reloc_move structure to internally track
relocation requests. This patch remove unnecessary
dependency between relocation module and ftl_io.

Change-Id: I9dd1bca6039b7c8961cb22a9f85ea1c5737a0a6f
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/454006
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-06-11 06:21:46 +00:00
Mateusz Kozlowski
e49ae312b1 lib/ftl: Add ftl_io argument to internal I/O callbacks
Almost all I/O callbacks in ftl utilize data in ftl_io, which is
initialized as callback argument in ftl_io_init_internal. This patch
changes the behavior to always returning the ftl_io struct in addition
to an extra opaque buffer.

Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: I611ab1b33575f599798a2bb65c231a724c852c7f
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455831
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-06-07 18:26:57 +00:00
Wojciech Malikowski
95ace66f8d lib/ftl: Remove ftl_md struct from band
Keep lba map related fields in separate struct directly
in the band. Cleanup interfaces depended on ftl_md.

Lba map structure will be extended in next commit.

Change-Id: I1cfc2f2ff0c0e90bb63f39808780845673002e70
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453370
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-06-07 04:39:14 +00:00
Konrad Sztyber
819545ae7e lib/ftl: split IOs based on the number of iovecs
Split the IO in case io->iov_cnt exceeds the FTL_IO_MAX_IOVEC. It allows
the user to pass any number of iovecs. Each IO request is now split if
necessary and submitted via ftl_io_call_foreach_child submitting each
child individually. Any resubmissions need to be handled on a child
level, as parent request doesn't track its children submission status.

Change-Id: If54249d54225b34191216a366c227f79215abc90
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455527
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-06-06 18:07:41 +00:00
Konrad Sztyber
8f9aed50c2 lib/ftl: added ftl_io_call_foreach_child
As the name suggests, the function calls a function passed as the
argument for all children IOs.

Change-Id: Idd7b6013e68fdd188db6e682bb9ba9ec2e6f52f1
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455526
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-06-06 18:07:41 +00:00
Konrad Sztyber
e1dbe1c84c lib/ftl: fixed metadata pointer arithmetic
The current location of the metadata pointer should be calculated based
on the metadata size, not block size.

Change-Id: I90177557cf4e194cd77adfbfe1c5a6e0e6df7967
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455921
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-06-04 00:07:02 +00:00
Konrad Sztyber
1d97387549 lib/ftl: use pre-allocated iov buffer
Keep an array of iovecs tied to each IO. Internal IOs can have
iov_cnt > 1 without having to allocate additional memory. User's iovec
can now be modified too (e.g. when splitting the request).

Change-Id: Iec73c6dad59acdc331a1461fd7feac085138a8de
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455525
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-06-03 04:39:37 +00:00
Konrad Sztyber
8e6a5601e4 lib/ftl: allocate ftl_io inside ftl_io_user_init
On the user IO path, the flow always follows the same procedure:
allocate ftl_io and initialize it using ftl_io_user_init. There's no
point in separating these two actions.

Change-Id: I578347ccc7c85e5945368dd6bf76c58985af3939
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455523
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-06-03 04:39:37 +00:00
Konrad Sztyber
1d29386066 lib/ftl: stop splitting internal IO data buffers
Some of the internal IOs split the contiguous data buffer into multiple
iovecs to separate the requests. It's unnecessary and can be replaced
by checking the position within the data buffer.

Change-Id: I9255ea0072fee6c4e6a7dca21e4008780bc45610
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455519
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-06-03 04:39:37 +00:00
Konrad Sztyber
b52a5a45d8 lib/ftl: automatically advance parent's IO position
Advance parent's IO position when children positions are advanced.
There's no need to call ftl_io_advance twice for the same request
anymore.

Change-Id: I1f7f04a3a83575b11e7bf0b13c0c8f3bf98b5522
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455518
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-06-03 04:39:37 +00:00
Konrad Sztyber
36ff0eeb88 lib/ftl: mirror writes to non-volatile cache
Apart from writing the data to OCSSD, mirror the latest two bands of
data on the persistent write buffer cache. Currently the data is only
sent there, further patches will add metadata support, shutdown recovery
and L2P updates.

Change-Id: Ief05d0c23fa0e25bd6085e0ce3e1528d6736d174
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/450266
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-05-22 16:17:23 +00:00
Konrad Sztyber
aba66c0644 lib/ftl: added ftl_io_shrink_iovec
This function can be used to reduce the number of iovecs on exisiting IO
structure. It's useful when only the maximum number of iovecs is known
at the ftl_io allocation time.

Change-Id: Ief06634446c65889401f5e0b07b2d1ce7912af34
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/450630
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
2019-05-22 16:17:23 +00:00
Konrad Sztyber
196bb8a557 lib/ftl: set ftl_io's PPA to invalid during init
Set the PPA to FTL_PPA_INVALID during IO initialization to explicitly
mark it uninitialized.

Change-Id: I75ca1644b59501056f0d4ea8a4f446fd1b770045
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453684
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-05-16 07:56:20 +00:00
Konrad Sztyber
3363ab70dd lib/ftl: retrieve LBA / iovec via getter functions
Replaced all direct accesses to io->lba / io->iov with getter functions,
as they both should behave differently based on the FTL_IO_VECTOR_LBA
flag and io->iov_cnt respectively.

Change-Id: I0d387a2a908cc37afa9e1b97ca5b408cef0d6b08
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/450265
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-05-16 07:56:20 +00:00
Konrad Sztyber
8c6f63a3f1 lib/ftl: name ftl_io's LBA / iovec unions
Replaced the unnamed unions with named ones. It makes the assignments
easier (it allows for direct assignment instead of having to choose the
correct union member).

Change-Id: I130233a096070092275a040e73cbaf36106ae1ab
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453682
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-05-10 22:20:57 +00:00
Wojciech Malikowski
049624823b lib/ftl: Rely on internal IO position during ftl_io_done()
Change-Id: I7a7667e1c4e6321150e5bc8eb4287fb61847ccb5
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/450174
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Young Tack Jin <youngtack.jin@circuitblvd.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-04-24 23:24:21 +00:00
Konrad Sztyber
ef5ede4b51 lib/ftl: allow parent/child relations between IOs
Extended the IO descriptors with the ability to group them together in a
parent / child(-ren) relationship. It allows to delay parent completion
until all children are done and enables the children to have additional
context in their completion routines (as opposed to having a single IO
describe all requests).

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I6ad111dc231813e397697109a333e2a5d7f5941d
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449068
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Young Tack Jin <youngtack.jin@circuitblvd.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-04-24 21:57:06 +00:00
Claire J. In
3be9febdf9 lib/ftl: check for null
Fixes #726.
This prevents a segmentation fault by checking the pointer
before accessing it.

Change-Id: I18a3776fbcd6242c6efd1b88044debaa88c7342f
Signed-off-by: Claire J. In <claire.in@circuitblvd.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449591
Reviewed-by: wuzhouhui <wuzhouhui@kingsoft.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-04-02 08:22:48 +00:00
Wojciech Malikowski
f8399d4539 lib/ftl: Update internal IO position in one place
Change-Id: Ie925659e8ffca1fb8aaa1252118bc670b3b274fd
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449015
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-03-28 14:17:11 +00:00
Wojciech Malikowski
e27f64fb98 lib/ftl: Use ftl_io_done() to determine if IO is completed
Change-Id: I76b58764153793f26b167071041ee17673fec63f
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449160
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-03-28 14:17:11 +00:00
Wojciech Malikowski
5d82fa6af7 lib/ftl: Pass errors from nvme layer only in completion callback
FTL should pass IO errors to upper layer only in
completion callback to be consistent on sync and
async error paths.

Change-Id: I29c81af477b476c1e10b7933126737a222553ffa
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/447176
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-03-15 05:03:39 +00:00