Commit Graph

9344 Commits

Author SHA1 Message Date
Mateusz Kozlowski
e49ae312b1 lib/ftl: Add ftl_io argument to internal I/O callbacks
Almost all I/O callbacks in ftl utilize data in ftl_io, which is
initialized as callback argument in ftl_io_init_internal. This patch
changes the behavior to always returning the ftl_io struct in addition
to an extra opaque buffer.

Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: I611ab1b33575f599798a2bb65c231a724c852c7f
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455831
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-06-07 18:26:57 +00:00
Vitaliy Mysak
5875bb0e3b test/ocf: disable norandommap flag
disable FIO norandommap flag because it is incompatible with
  verify which in context of async IO could cause verification errors.
More on norandommap:
  https://fio.readthedocs.io/en/latest/fio_doc.html#cmdoption-arg-norandommap

Change-Id: I94392495a1f9f06360e3791fca31b88908d19dcb
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457001
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-06-07 18:18:42 +00:00
Vitaliy Mysak
7b7590f838 ocf: Improve handling of partial IOs
Improve handling of partial IOs for case when iovcnt == 1.
This case is pretty frqeuent because all data allocated by OCF
  have iovcnt == 1 (see ctx->data->alloc() or vbdev_ocf_ctx_data_alloc())

Change-Id: I8d38ca9e9d2bd3e6ce298bf788ce6ed782b56594
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456202
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-06-07 18:18:42 +00:00
Vitaliy Mysak
2bfa860dfe ocf: fix offset check
This patch fixes submission of partial IOs in bottom adapter

Existing check (if offset > 0) was not sufficient to detect partial IOs
  because there could be an IO with offset = 0 but length < total size of iovs.
This patch changes the check, but also free operation on completion
  because now the old free does not cover all situations when we allocate iovs.

`Partial IOs` are the IOs handled by bottom adapter
  which specify only part of the internal iovs vector.
So their length is less that the length of internal iovs vector.
They exist because sometimes parts of single IO from top adapter
  need to be sent to different locations.
Also, in general, IOs initiated by OCF (such as cleaner IOs) are represented
  as single big iov that is submitted by parts in form of 'Partial IOs'.

Change-Id: I8ae47659fb34904c593a696d74d683a418ac9962
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455821
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-06-07 18:18:42 +00:00
Vitaliy Mysak
e0edd0a205 test/ocf: add persistent metadata tests
We check if get_ocf_bdevs after shutdown is the same as before,
  which shows that state for all vbdevs wes correctly restored from metadata.

Change-Id: I9089642aeb840fc940b07908aa39c8193887432f
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455424
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-06-07 18:18:42 +00:00
Vitaliy Mysak
1e0e0ea1fc ocf: RPC add WriteBack to allowed cache modes
Make construct_ocf_bdev call accept 'wb'
  as allowed cache mode.
Previusly user could only create OCF bdevs
  in WriteThrough or PassThrough modes.

Change-Id: Ic2e1c8d2905cb51fc13c080aa7b7a4dfd7d9387f
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455628
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2019-06-07 18:18:42 +00:00
Vitaliy Mysak
9686948334 ocf: implement metadata probe
Implement metadata probe functionality to load cache state
  from disk.
During metadata probe, we inspect UUIDs of core devices
  and create vbdev configurations based on them.
  Then, to start vbdev, we use load path (loadq = true).

After this change persistent metadata is officially supported,
  we can save and restore cache state from persistant storage.

WriteBack mode is now safe to use in respect to unexpected shutdowns,
  because all information about dirty data is also restored during cache load.

Change-Id: I6cf86aabd68177b88638a68ea6a5b78a1068a4d0
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455417
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-06-07 18:18:42 +00:00
Vitaliy Mysak
73224f880e test/ocf: enable serialize_overlap flag for FIO
Set serialize_overlap=1 to prevent data races.
This solves the issue of fio --verify fails in OCF tests
  when persistent metadata is enabled.
  Example:
     https://dqtibwqq6s6ux.cloudfront.net/public_build/autotest-per-patch_33325.html

The serialize_overlap flag prevents from
  creating undefined regions by doing simultaneus writes
  (occurs on certain storages).
  More on that:
     https://fio.readthedocs.io/en/latest/fio_doc.html#cmdoption-arg-serialize-overlap

We should probably use this flag for all FIO --verify tests.

Change-Id: Ia4f73f9900835faf4f94018f1d148fb64f06dc8a
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456991
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-06-07 18:18:42 +00:00
Vitaliy Mysak
5983f92cfc ocf: Update ocf submodule to version OCF v19.3.2
This patch updates submodule and appropriate functions.
Cleaner poller is now registered only when needed.

Change-Id: Ic4ca7ce6f77b71ac12c19462f62ae7cd96c59006
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455408
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2019-06-07 18:18:42 +00:00
Shuhei Matsumoto
1c21756a1e dif: Fix wrong block size info to test dif_generate_stream.
Data block size was specified by mistake, and this patch changes
to use block size instead.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Icfb1df40438ef82211279b0b7c79036283744361
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456292
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-06-07 05:11:00 +00:00
Shuhei Matsumoto
3894a7bc68 dif: Rename parameters of dif_generate_stream to match related APIs
using data_offset and data_len is consistent with related APIs,
and is done in this patch.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ic9d216e25880f7b5ab33b764f45d332c090fa88c
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456291
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-06-07 05:11:00 +00:00
Shuhei Matsumoto
6d156d5b7d iscsi: Dump PDU contents to know what PDU was sent in illegal state
In some test cases, unknown iSCSI PDU other than the login request
has been sent to the iSCSI target when the iSCSI target is not in
runnable state, and it has caused failure in iSCSI target.

To know what PDU was received by the iSCSI target, this patch changes
the iSCSI taget to collect dump of the PDU.

SPDK has already the SPDK_LOGDUMP macro but the SPDK_ERRLOGDUMP macro
will be appropriate in this use case and added. Then the SPDK_ERRLOGDUMP
is used in iSCSI library.

We can decode PDU and output any format easy to read by human, but
creating good format is not easy task and error prone. So this patch
uses simple dump.

Dump outputs like:

PDU
00000000  40 80 00 00 00 00 00 00  00 00 00 00 00 00 00 00  @...............
00000010  0b 06 00 10 ff ff ff ff  00 00 00 5a 00 00 00 03  ...........Z....
00000020  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00   ................

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I3ed2fca7fec24ccff17b89ba749a58c397b72c13
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456952
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-06-07 05:11:00 +00:00
Changpeng Liu
cf680fe47e vhost_blk: set the maximum supported data segment based on bdev
AIO backend requires aligned data buffers, and the maximum
IOVs supported in bdev module is defined to 32, there are
cases for Windows Guest which will send data segments more
than 32, SPDK can't process such cases, so here we can set
the 'seg_max' parameter based on bdev module capability.
Also set the maximum segment size for those requests.

Fix issue #625.

Change-Id: I0ff61e55872af17115c0b6b28425e70cb8769790
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452378
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-06-07 05:05:22 +00:00
Changpeng Liu
69db40f677 bdev/raid: add additional NULL check for data iovec
iSCSI application may set the data iovec to NULL for READ requests.

Change-Id: Ic6b73f62f75dc2d927c18219fa1c2cb242f0528e
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457079
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-06-07 05:05:22 +00:00
Wojciech Malikowski
95ace66f8d lib/ftl: Remove ftl_md struct from band
Keep lba map related fields in separate struct directly
in the band. Cleanup interfaces depended on ftl_md.

Lba map structure will be extended in next commit.

Change-Id: I1cfc2f2ff0c0e90bb63f39808780845673002e70
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453370
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-06-07 04:39:14 +00:00
Konrad Sztyber
d87493378d env: added spdk_mempool_obj_iter
As the name suggests, this function iterates through all elements of the
mempool invoking a callback function on each one. It's particularly
useful when deinitializing mempool that requires freeing resources tied
to each element (e.g. allocated through spdk_mempool_create_ctor).

Change-Id: I3da1fee527a36bf99f0b0e2dd3d6f9297422ff25
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455971
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-06-07 04:39:14 +00:00
Konrad Sztyber
819545ae7e lib/ftl: split IOs based on the number of iovecs
Split the IO in case io->iov_cnt exceeds the FTL_IO_MAX_IOVEC. It allows
the user to pass any number of iovecs. Each IO request is now split if
necessary and submitted via ftl_io_call_foreach_child submitting each
child individually. Any resubmissions need to be handled on a child
level, as parent request doesn't track its children submission status.

Change-Id: If54249d54225b34191216a366c227f79215abc90
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455527
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-06-06 18:07:41 +00:00
Konrad Sztyber
8f9aed50c2 lib/ftl: added ftl_io_call_foreach_child
As the name suggests, the function calls a function passed as the
argument for all children IOs.

Change-Id: Idd7b6013e68fdd188db6e682bb9ba9ec2e6f52f1
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455526
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-06-06 18:07:41 +00:00
Wojciech Malikowski
2646670781 lib/ftl: Added assert checks to ftl_band_set_state()
Change-Id: I142ea5be2e5d03b29dc17896fe590a1af798b11f
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/451873
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2019-06-06 11:54:24 +00:00
Wojciech Malikowski
fcd2364c6a lib/ftl: Metadata read/write interfaces refactor
Extended ftl_band_read_lba_map() interface to provide
range of lba map to read.

Added ftl_xfer_offset_from_ppa() function for
calculating offset of read portion of lba map
based on ppa address from IO. This offset is
used for coping read lba map portion  to internal
ftl buffer.

Change-Id: I1c72a18c79eda8c33cd0b20ea36a5d9521a09d06
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449435
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-06-06 07:26:35 +00:00
Darek Stojaczyk
0dd5f32ebe doc/compression: fix diagrams
They were not interpreted as "code" blocks and looked
really broken on spdk.io. Fix it by explicitly wrapping
them with code tags (```).

Change-Id: Ia880f67124dda6f8c0cb24d7e2e138d9f297958f
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456299
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-06-06 05:35:25 +00:00
Tomasz Zawadzki
9fda9814b7 test/asan: move LSAN_OPTIONS to autotest_common.sh
Further patches in series will modify the suppression file,
so this patch now creates such file in /var/tmp/.
Meanwhile adding the known false positive to the list.

Any known false positives or leaks in external libs/execs
should be added by adding further entries to the suppression
file.

Change-Id: I7b78d900a86c6eca0b41240fb34dc4f7ad597079
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456622
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-06-06 05:17:16 +00:00
Tomasz Zawadzki
37b5912545 fio_plugin: fix io completion queue leak in nvme plugin
fio_thread->iocq allocated on line 403 was leaked when
fio finished its run.

Change-Id: I740dcaa1e0037283d099ddf4bc125cec57cfdbcc
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456623
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-06-06 05:17:16 +00:00
Tomasz Zawadzki
3b9db6c403 test/fio: export single variable for running fio_plugins
This patch adds functions in autotest_common.sh,
fio_bdev() and fio_nvme() for their respective plugins.
It simplifies its usage throughout the scripts.

Additionally will help with expanding the fio cmd
line with nessecary changes only in single place.

This will be used in next patch in series to
LD_PRELOAD ASAN before fio_plugin.

Note: Did not implement changes for perf scripts,
since they are usualy run separately and didn't want
to affect those.
Similarly didn't change vhost initiator tests,
because the exported directories do not work for
inside the VM. Will need to think of a way to change it
there as well.

Change-Id: Idf750ee7aa9d5e30efc0ce430d15fa45ceccb203
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456314
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-06-06 05:17:16 +00:00
Shuhei Matsumoto
2761079168 iscsi: Remove SPDK_SUCCESS and use 0 instead
SPDK_SUCCESS is a remnant and all other SPDK libraries have used
just 0. So this patch removes SPDK_SUCCESS from iSCSI library and
uses 0 instead.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ie33fd26238411e994ea9ea0c898da1abb502cad6
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456928
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-06-06 02:08:15 +00:00
Shuhei Matsumoto
872d8d8ea7 iscsi: correct data digest computation for extended LBA payload
This patch uses the newly added API spdk_dif_update_crc32c() when
DIF insert/strip is enabled.

Change-Id: Icf32a0ddef6cf92b0887b38495457e7fcac30987
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456172
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-06-06 02:08:15 +00:00
Shuhei Matsumoto
a7d832f28c iscsi: Correct iovec increment when data digest is enabled
iscsi_sgl_append_with_md had not increment _iscsi_sgl::iov
by appended count of iovecs. Hence if data digest is enabled,
data segment will be overwritten by data digest.

This patch fixes the bug.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ibcdacb883b2b97ad86cfc39a035c76264090401d
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456451
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-06-06 02:08:15 +00:00
Shuhei Matsumoto
4fa7987047 dif: Compute CRC-32C for extended LBA payload
Data digest computation should take extended LBA payload but
could not do yet.

This patch adds an new API spdk_dif_update_crc32c() to compute
CRC-32C value for extended LBA payload.

In the next patch, spdk_dif_update_crc32c will be used in iSCSI
target first.

Change-Id: I327f384bb7dfd8b68279b0acec0ee78a40264a26
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456171
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-06-06 02:08:15 +00:00
Shuhei Matsumoto
85e6f154b8 iscsi: Get DIF context once when allocating data buffer
DIF context had been got for every read of data segment but
DIF context is not changed and so getting DIF context once
when allocating data buffer is enough and is done in this
patch.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I0f386ab594a0d9076fa0206d5fc240f5c790181d
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456772
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
2019-06-06 02:08:15 +00:00
Shuhei Matsumoto
1204ff34e0 iscsi: Update _pdu only when completing read in spdk_iscsi_read_pdu
The only caller iscsi_conn_handle_incoming_pdus accesses the returned
pdu only when the return code is 1. So we can remove update of
_pdu for other cases in spdk_iscsi_read_pdu.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I54b9f050c3b45c87f4797a90d7606638d6c821ad
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456771
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-06-06 02:08:15 +00:00
GangCao
2882b9895a iscsi: return the proper error to append the connection
Instead of just return "-1", the proper error detail will be
returned and reported out.

Change-Id: Ief900494081ddc9ae6329ee7a56723d9cb5efe13
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456937
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-06-06 01:44:33 +00:00
GangCao
9aed5fa5b6 iscsi: update the version error message
Change-Id: I3a2488cdd81812f1fce136b97e49eee18b49fbc1
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456915
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-06-06 00:59:57 +00:00
Ziye Yang
679257db88 nvme/tcp: Properly deal with supporting single r2t
According to the TP 8000 spec in Page 26:
Maximum Number of Outstanding R2T (MAXR2T): Specifies the maximum
number of outstanding R2T PDUs for a command at any point in time
on the connection.

This patch makes the current host driver implementation support one r2t.
We cleanup the code to do the right advertising to the target in the
icreq and avoid attempts to deal with multiple rt2s.

Reported-by: Or Gerlitz <ogerlitz@mellanox.com>

Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: If06ad2e8bde31c2fd7e1c3739f651fb64040e3a9
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455750
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Or Gerlitz <gerlitz.or@gmail.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-06-06 00:58:58 +00:00
Jim Harris
d6e3c3779c .travis.yml: remove IRC notifications
Since we're using Slack now, there's no need to send
notifications to the #spdk IRC channel.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ie484bbeac2d8a959da8371d54e0d74f2cb4ae1d3

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456893
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-06-05 20:31:48 +00:00
paul luse
83afe68690 dpdk: build and link QAT for both crypto and compression
In prep for full QAT compression support later in this patch
series. dpdkbuild/Makefile slightly refactored for readability,
x86 crypto check removed as it pre-dated checks we now have in
configure.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Iaaaf51b9eb5e18840f47d2d4f431c5a6e8c420ee
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456408
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-06-05 19:07:32 +00:00
Ben Walker
c66685defc test/vhost: Add a vhost_rpc function to common.sh
This sends an RPC to the vhost target with the given number.

Change-Id: Ifd74443087475242d3a65a0e8621dcd20ab2d1b0
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456703
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-06-05 18:30:34 +00:00
Ben Walker
bfdb94035e test/vhost: Rename spdk_vhost_kill to vhost_kill
No need to prefix name with spdk.

Change-Id: I9b07a52f6c88d77e1dda2c63ec951c1847780cca
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456702
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-06-05 18:30:34 +00:00
Ben Walker
a87ba5a7cf test/vhost: Rename spdk_vhost_list_all to vhost_list_all
No need to prefix name with spdk.

Change-Id: I413f78fac427a0955db495bb926bf8d6bc3dfb2a
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456701
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-06-05 18:30:34 +00:00
Ben Walker
9132128ff3 test/vhost: Rename spdk_vhost_run to vhost_run
We typically don't prefix bash functions with spdk.

Change-Id: Icda57e42acb83f4aa51731ac62f8db7ee26e658a
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456700
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-06-05 18:30:34 +00:00
Ben Walker
86f38e70e9 test/vhost: Remove config file option to spdk_vhost_run
Nothing used this anymore. Everything is json.

Change-Id: Iaa1239422e18436b3b8819ce734955683ec63735
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456699
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-06-05 18:30:34 +00:00
Li Feng
02363ccb3b lib/iscsi: fix iscsi connection schedule
If the qualified core wasn't found, then we should compare
spdk_env_get_core_count() with i, instead of lcore.

Signed-off-by: Li Feng <fengli@smartx.com>
Change-Id: Ie92f56712b7f0e51636008fe12fff5584b6be8ab
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456415
Reviewed-by: wuzhouhui <wuzhouhui14@mails.ucas.ac.cn>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-06-05 17:39:12 +00:00
Tomasz Kulasek
b7080d47c5 test/json_config: wait for clean config
Fixes issue #789

Change-Id: I795247e3b19c24dbf1564e2c0f6c5c44e850ab8b
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455668
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-06-05 17:08:51 +00:00
GangCao
3a0886c408 iscsi: correct the error detail
According to https://tools.ietf.org/html/rfc3720#page-196,

Error detail of "Authorization failure" is for the case that
the initiator is not allowed to the given target.

Change-Id: Ie628c4c857e965aa6694399a9832ce0501e50745
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456826
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-06-05 16:48:48 +00:00
paul luse
d5d7c57043 bvdev/compress: move comp transform structs to static globals
Code cleanup.

Change-Id: I9f96a889ed39afbd14a42b50e2537b1c4844f514
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456517
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-06-05 16:48:36 +00:00
paul luse
5ef2b14e3c bdev/compress: rename function to avoid collision with bdev layer
Makes debug easier...

Change-Id: I3b975e0a494c3920387b2d1921ebf8089b935bf5
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456512
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-06-05 16:48:36 +00:00
Ziye Yang
0bb626672b nvmf/tcp: Support single r2t usage
According to the TP 8000 spec in Page 26:
Maximum Number of Outstanding R2T (MAXR2T): Specifies the maximum
number of outstanding R2T PDUs for a command at any point in time
on the connection.

Note that by the spec, the target may only support single r2t
(which is the minimum possible), it doesn't have to use multiple r2ts
even if the initiator supports that. So remove the maxr2t and
pending_r2t variable in the tcp qpair structure.

In the original design, we think that maxr2t is the maximal active
r2t numbers for each connection. So if the initiator sends out maxr2t=16,
it means that all the commands of a qpair can use such number of R2T pdus.
So we need to wait for the available R2Ts for the request when the maxr2t
reaches the maximal value. But it is the wrong understanding of the spec.

In fact, each command has its own number of maximal r2t numbers, then we
do not need to use the wait method for R2T method anymore. So we remove
the state TCP_REQUEST_STATE_DATA_PENDING_FOR_R2T. Futhermore, we adjust
the related SPDK_TPOINT_ID definition.

In current patch, the target will support one active R2T for each
write NVMe command. Thus, we remove the function spdk_nvmf_tcp_handle_queued_r2t_req.

Reported-by: Or Gerlitz <ogerlitz@mellanox.com>

Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I7547b8facbc39139b4584637ccc51ba8b33ca285
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455763
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Or Gerlitz <gerlitz.or@gmail.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-06-05 16:46:55 +00:00
Ziye Yang
fe2dddbbbc nvme/tcp: Correct nvme_tcp_qpair_disconnect behavior
The current nvme_tcp_qpair_disconnect behaviour
is not exactly correct, we do not re-initialize
the state of some data structures of the tqpair.
And this caused the coredump.

Purpose: Fixes #808.

Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I4d2cad8fc0712dbebfc2f3e52373cbe3b9908bf7
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456755
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-06-05 16:13:55 +00:00
Wojciech Malikowski
f97809e3cb lib/vmd: Moved MAX_VMD_TARGET definition to public header
MAX_VMD_TARGET defines size of array that need to be passed
to spdk_vmd_pci_device_list()

Change-Id: Ib2a33fe50072036e6a8f1709ac4e3ee82c1bb3f1
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456480
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-06-05 15:19:51 +00:00
Wojciech Malikowski
d3a0531bc1 lib/vmd: Removed typedefs from structures definitions
Removed typedefs and unused structures.

Change-Id: I711094097520333cbeda2f26a1a7d403750070b1
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456479
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-06-05 15:19:51 +00:00
Wojciech Malikowski
f7e68be171 lib/ftl: Minor cleanup in error path from nvme layer
Change-Id: If64d721fac28683b9b385238ae1c63aad77c0914
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/451869
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-06-05 06:50:03 +00:00