This patch allows multiple iovecs as argument in spdk_dif_generate_stream.
Subsequent patches will support DIF strip and insert in SPDK NVMe-TCP
target based on the patch series.
Change-Id: I1f3d6c5b9f924bb52525e1611db403846d087563
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453756
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
As a subsequent effort, this patch changes the interface of
spdk_dif_generate_stream to accept SGL data buffer.
This patch allows only a single iove. The next patch will allow
multiple iovecs.
Change-Id: I56f901d73ca3b9da4b56c213ebafcd7706b1fef8
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453755
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
This patch allows multiple iovecs as argument of
spdk_dif_set_md_interleave_iovs.
Subsequent patches will support SGL data buffer in
spdk_dif_generate_stream too. UT code tests only
spdk_dif_set_md_interleave until then.
Change-Id: I7ac03a3c8f7bcd922af4f29b404ebf3acc4b89e5
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453736
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Currently iSCSI target have used a single contiguous data buffer
both for read and write I/O. Hence spdk_dif_set_md_interleave_iovs
accepts a single contiguous data buffer and its size as arguments.
On the other hand, NVMe-TCP target recently has changed to use
SGL data buffer instead.
DIF strip and insert will be supported in NVMe-TCP target next,
and updating spdk_dif_set_md_interleave_iovs to accept SGL data
buffer will be helpful.
This patch changes the interface of spdk_dif_set_md_interleave_iovs,
but allows only a single iovec. The next patch will allow multiple
iovecs.
Change-Id: I31b09814f8ec920e463a5b1be8fb88cad7d277fb
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453735
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This change unifies the DIF library to use dif_sgl throughout.
Change-Id: I60addcf05e75b823e4dfb216122bf46edbc42e78
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/454528
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
The struct _iov_iter has been used to iterate the passed iovec array.
Subsequent patches will share the struct among iteration and
creation. To clarify this update, rename the struct _iov_iter to
_dif_sgl. The name sgl has been already adopted widely in SPDK.
Change-Id: Iab81bcb0658fbd0e39305e045c3d0aadba8fc55f
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/454525
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Add an new API spdk_dif_generate_stream() to insert DIF into
the metadata space of newly read data block by stream fashion.
This API is tested by replacing the previous DIF generation such
that DIF is generated once all data was read.
Change-Id: I757b4e6f10fe2d9d643f56ebb678c731cdad63bb
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/446594
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: wuzhouhui <wuzhouhui@kingsoft.com>
This patch adds an API spdk_dif_set_md_interleave_iovs().
This function is used to leave a space for metadata for each block
when the network socket reads data, or to make the network socket
ignore a space for metadata for each block when the network socket
writes data. This function removes the necessity of data copy in
the SPDK application during DIF insertion and strip.
Change-Id: I018efd77bdadcaec4679eea4dc1e404f155b0879
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/446216
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: wuzhouhui <wuzhouhui@kingsoft.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Previously, test data buffer was filled by one value, 0xAB, but
this may not be enough to detect future potential issues.
ut_data_pattern_verify() already did per-byte check, but size of
test data buffer is small and completion time of tests is not long.
So, even if we change ut_data_pattern_generate() to set per-byte
data instead of memset(), extra overhead of test completion time
will be negligible.
Change-Id: I35677b238f96a73c0c408f0818f080a92492dac6
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/c/445430
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: wuzhouhui <wuzhouhui@kingsoft.com>
Allow user to add seed value for guard compuation to DIF context.
This will avoid the guard being zero in case of all zero data.
NVMe controller doesn't support seed value for guard computation
explicitly, and hence if we want to use such a seed value in
NVMe controller, we have to format metadata more than 8 byte,
and add seed value into the reserved metadata field.
But some popular iSCSI/FC HBAs and SAS controllers have supported
seed value for guard computation, and so supporting seed value
in the SPDK DIF library is very helpful for some use cases.
Hence this patch makes the DIF library possible to specify seed
value for those use cases.
Change-Id: I7e9e87cb441bf263e64605c7820409fdc22dd977
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/c/444334
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: wuzhouhui <wuzhouhui@kingsoft.com>
Next patch will introduce enum spdk_dif_check_type for user to
know easily if checking DIF field is enabled or not.
This patch renames bitmask macros from SPDK_DIF_*_CHECK to
SPDK_DIF_FLAGS_*_CHECK to avoid mis-interpretation .
Using FLAGS was derived from SPDK_NVME_IO_FLAGS_PRCHK_* in
include/spdk/nvme_spec.h.
Change-Id: I89e155d047352f54091c14b9251464cd3a72a162
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/c/443338
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
SPDK NVMe driver and NVMe CLI define the parameter about DIF location as
follows:
If set to 1 and DIF is enabled, then DIF is transferred as the first 8
bytes of metadata. If set to 0 and DIF is enabled, then DIF is transferred
as the last 8 bytes of metadata. Defaults to 0.
This patch fixes the apparent inconsistency.
Change-Id: I481332bb9a6caa488664dbd1fe6039270b9f9663
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/c/440672
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: wuzhouhui <wuzhouhui@kingsoft.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Because we have called _are_iovs_valid() to ensure that all blocks are
covered by iovec, and we process iovec compactly and orderly, so we
don't need to do _iov_iter_cont() check.
Since all callee are removed, so reomve definition of _iov_iter_cont()
as well.
While here, add an assertion in _iov_iter_advance() to detect
unexpected error.
Change-Id: I6b6077bafbf7ec18f14ee2835d3b99199b5e559b
Signed-off-by: wuzhouhui <wuzhouhui@kingsoft.com>
Reviewed-on: https://review.gerrithub.io/c/439869
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
For separate metadata payload, Introduce a struct to collect DIF error
information and pass the reference to the struct to bit-flip error injection
and verification.
This change will make logging of DIF error and comparison between
injection and verification possible for separate metadata payload.
Merging this patch to the previous is possible but add this as a separate patch
to reduce the patch size.
Change-Id: Ifce2130e902bc090cbe205fa8df12559739ced57
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/c/435097
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
This patch adds APIs to generate and verify DIF for SGL
payload with separate metadata as byte alignment and granularity.
Change-Id: I4e553c9b6f8c96e576a69115002963d32379d439
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/c/434150
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
For copied extended LBA payload, Introduce a struct to collect DIF error
information and pass the reference to the struct to bit-flip error injection
and verification.
This change will make logging of DIF error and comparison between
injection and verification possible for copied extended LBA payload.
Merging this patch to the previous is possible but add this as a separate patch
to reduce the patch size.
Change-Id: I9f7ac1332fe63cb206c692d0876c8de41ef9286f
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/c/435227
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: wuzhouhui <wuzhouhui@kingsoft.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This patch adds APIs to copy data and generate and verify DIF
for internally created extended LBA payload.
This patch is for read strip and write insert operation to make
DIF transparent to the upper layer.
Change-Id: I025f35571490aa197aac5e7572549e31c2479b41
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/c/432987
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: wuzhouhui <wuzhouhui@kingsoft.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Introduce a struct to collect DIF error information and pass
the reference to the struct to bit-flip error injection and
verification.
This change will make logging of DIF error and comparison between
injection and verification possible.
Change-Id: I43941f1a7f4b0ad1e8ff324f2ac8fad9e195a100
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/c/438023
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: wuzhouhui <wuzhouhui@kingsoft.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
This patch reduces the number of parameters of APIs by setting
necessary information to context struct and pass it to APIs.
This will simplify the code and will clarify the core logic.
Besides, by introducing context data, update of reference tag
is further consolidated.
Change-Id: I1b3eec596847eb4ef1cf43441035df21f746192a
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/c/438021
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: wuzhouhui <wuzhouhui@kingsoft.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
The case that metadata is supported but DIF is not enabled was
handled as error. But the case is described as an normal case
in NVMe specification.
This patch changes the case to normal.
Besides a break was missing and added in this patch.
Change-Id: I784c36fed3b92da9ae5500d945f1dc20a2ca406b
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/c/438362
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: GangCao <gang.cao@intel.com>
Reviewed-by: wuzhouhui <wuzhouhui@kingsoft.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This patch adds APIs to inject bit flip error into any field
of the extended LBA payload.
Change-Id: I3ca601999e55ea6228bb525ac8c0744c7df32398
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/434292
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Introducing helper functions will simplify a little complex
implementation.
Change-Id: I94911497b8aecc0ffbd148519ce22501ef69c79d
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/435387
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
DIF and DIX will be supported in SPDK throughout, e.g., NVMe
driver, NVMe-oF initiator and target, NVMe block device, malloc
block device, SCSI, iSCSI target, FIO plugin, and Perf.
Generic and common APIs to generate and verify DIF and inject
bit flip error to any field will be helpful for them.
This patch is the first in the patch series.
This patch adds APIs to generate and verify DIF for SGL extended
LBA payload as byte alignement and granularity.
Change-Id: Ie6588d960113761f10efbf2d2a3cae004af37ce8
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/432261
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
