Commit Graph

312 Commits

Author SHA1 Message Date
Shuhei Matsumoto
20bbd038aa ut/bdev/raid: try to break lines after 100 characters
Some lines in this file are long and should be broken
as bdev/raid/raid_rpc.c was done recently.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I98a7f965b7230f32ab6a2d33a0f204591fa92ac1
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/464484
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-08-13 00:27:17 +00:00
Shuhei Matsumoto
953975fe87 ut/bdev/raid: Reorder stub definitions by libraries or modules
Ordering stub definitions by libraries or modules will be a little
easier to read.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I7568c94691229188c348a9c5e3fba8f2bc1ecf04
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/464483
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-08-13 00:27:17 +00:00
Shuhei Matsumoto
47155be6f3 ut/bdev/raid: Use cached pointer to expected output in IO verification
Use cached pointer to g_io_output in IO verification to improve
readability.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I71ced5d1e438d5046b9a5913a1a2860f8435d79b
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/464485
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-08-13 00:27:17 +00:00
Shuhei Matsumoto
2b015b7a26 ut/bdev/raid: Factor out operation to setup expected IO output
Create a helper function set_io_output() to setup expected IO output.
Besides change the name of cached pointer to expeced IO output
from p to output because p is ambiguous.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I7574bf8362f7c5d7841e08b4fbd9a9b66836012a
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/464482
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-08-13 00:27:17 +00:00
Shuhei Matsumoto
c0329efef3 ut/bdev/raid: Move create_construct_req() to clarify it is for RPC
create_construct_req() had been located just above raid_bdev_init().
However this location was not intuitive. Hence move create_construct_req()
to just above spdk_rpc_construct_raid_bdev().

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ia5998146ce7b4b9a3c3bdeb38b6dbc8dd21fa276
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/464481
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-08-13 00:27:17 +00:00
Shuhei Matsumoto
cdde42d52e ut/bdev/raid: Factor out operation to create request for get_raid_bdevs RPC
Create a helper function create_get_raids_req() and move duplication
of category name and initialization of related global variables into
the function. Some operations are re-ordered accordingly.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I03e12e8d77872634a65a94b4cd4092554e376b13
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/464480
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-08-12 04:20:15 +00:00
Shuhei Matsumoto
898c67d722 ut/bdev/raid: Factor out operation to create request for destroy_raid_bdev RPC
Create a helper function create_destroy_req() and move duplication
of bdev name and initialization of related global variables into
the function. Some operations are re-ordered accordingly.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I60f01103faec3ec7ec6c317470b043f84187a4c1
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/464479
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-08-12 04:20:15 +00:00
Shuhei Matsumoto
309be937f0 ut/bdev/raid: Factor out operation to create config for .INI config loading
Create a helper function create_construct_config() and move
create_test_req() and initialization of related global variables
into the function.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ia675f67e8332247cce4585a2d287b8062d4e1359
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/464478
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-08-12 04:20:15 +00:00
Shuhei Matsumoto
0f9704a685 ut/bdev/raid: Factor out operation to create request for construct_raid_bdev RPC
Create a helper function create_construct_req() and move create_test_req()
and initialization of related global variables into the function.

The next patch will create an another helper function
create_construct_config() and move create_test_req() and initialization
of related global variables into the function.

Hence move the common part of create_construct_req() and
create_construct_config() into create_test_req().

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I6852fdc7966359a1029c809565ceb7230b749dc6
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/464477
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-08-12 04:20:15 +00:00
Shuhei Matsumoto
5135e6e76d ut/bdev/raid: Remove unnecessary NULL byte appends after snprintf()
snprintf() do not write more than specified bytes including the
terminating null byte. If the output was truncated due to this limit
then the return byte is the number of chars excluding the terminating
null byte.

So remove NULL byte appends after snprintf() in this patch.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ic5b7719791721c6ae7b1dd8367be454b34df0fe5
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/464476
Reviewed-by: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-08-12 04:20:15 +00:00
Shuhei Matsumoto
6e90531549 ut/bdev/raid: Use string constants for verification
Some places had used dynamically allocated strings but they were
just duplicated from string constants. Hence we change them to
use string constants to make refactoring easier.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I0a5c4c1a72ee4aede12703c2dd50057c47098e28
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/464475
Reviewed-by: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-08-12 04:20:15 +00:00
Shuhei Matsumoto
cd6630d009 ut/bdev/raid: Add g_ prefix to pointer and size of RPC request
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Id701260521f7d91a8c9c1eedd54941261e092f52
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/464359
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
2019-08-12 04:20:15 +00:00
Shuhei Matsumoto
d98458f6d2 ut/bdev/raid: Move free(bdev_io) into bdev_io_cleanup()
This simplifies the UT code a little bit.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I8aae0a8f0b0aa7e931ed4303b9504feabf76e2b0
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/464358
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
2019-08-12 04:20:15 +00:00
Shuhei Matsumoto
e5f28d5d39 bdev/raid: Unify data type of number of base bdevs to uint8_t
Data type of number of base bdevs are corrupted. uint8_t is enough
and unify almost all related variables into uint8_t.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I323aaa229a76c3d5b5361d7643c3a9591ccad340
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/464357
Reviewed-by: yidong0635 <dongx.yi@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-08-12 04:20:15 +00:00
paul luse
ca1a3bf18e bdev/compress: use base bdev alias as base for comp_bdev name
If available, if not use the bdev unique name. This results in
a much friendlier comp_bdev naem for the user.  For example,
now it would look something like this: COMP_lvs0/lvs as opposed
to like this COMP_0b149b31-b66b-4cf7-ab39-a55b50788cd1

Change-Id: I319a141221ed8880edcec930ca5be9f256b105e0
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/463246
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-07-26 19:47:33 +00:00
Darek Stojaczyk
b701087f87 ut/bdev: silence scan-build warnings about garbage value comparison
bdev_ut.c:1300:17: warning: The left operand of '!=' is a garbage value
        CU_ASSERT(desc != NULL);
                  ~~~~ ^

Change-Id: Ie87cd4d7218a380e61616a83f3a9bceccc0798cb
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/463258
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-07-26 17:47:34 +00:00
Darek Stojaczyk
26f6c2a03d ut/bdev: silence scan-build warnings about histogram memory leaks
bdev_ut.c:1863:2: warning: Potential leak of memory pointed to by
'histogram'
        poll_threads();
        ^~~~~~~~~~~~~~

We used to free g_histogram instead of histogram. Those
should be the same thing, but scan-build gets confused.

Change the code to free histogram - this should also make
it slightly easier to read.

Change-Id: Ifaad1c2d9c7f9cc1a106f6edf3d22b54e42fe867
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/463256
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-07-26 17:47:34 +00:00
yidong0635
e3b90fe2d5 bdev: Fix warning about potenial memory leak about histogram.
Pointer passthrough g_histogram and histogram implicitly ,but the checker
still complains about this, change to spdk_histogram_data_free(histogram)
directly, no need any assert any more.

bdev_ut.c:1486:2: warning: Potential leak of memory pointed to by 'histogram'
        poll_threads();
        ^~~~~~~~~~~~~~
1 warning generated.

Change-Id: I609c1c92643ad03b8e3c62cd8434882e7942efa4
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462789
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-23 03:50:05 +00:00
yidong0635
2ab927030d bdev/vbdev_lvol: Fix scanbuild error about Null poniter parameter.
Issue reorts:
vbdev_lvol_ut.c:97:7: warning: Null pointer passed as an argument to a 'nonnull' parameter
                if (strncmp(alias, tmp->alias, SPDK_LVOL_NAME_MAX) == 0) {
                    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Change-Id: I0cc95c735ea7486263842f91a5866710f9ff61a5
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462598
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-07-23 00:17:25 +00:00
yidong0635
a1de2fe007 compress: Clear some simple scanbuild errors.
On Fedora30, GCC9+ reports errors:
Issues report:
vbdev_compress.c:482:17: warning: Assigned value is garbage or undefined
        comp_op->m_src = src_mbufs[0];
                       ^ ~~~~~~~~~~~~
vbdev_compress.c:502:17: warning: Assigned value is garbage or undefined
        comp_op->m_dst = dst_mbufs[0];
                       ^ ~~~~~~~~~~~~
vbdev_compress.c:491:3: warning: 1st function call argument is an uninitialized value
                rte_pktmbuf_attach_extbuf(dst_mbufs[iov_index],

compress_ut.c:826:2: warning: Attempt to free released memory
        free(g_mbuf_mp);
        ^~~~~~~~~~~~~~~
Should remove free(g_mbuf_mp), for pointer exchange.

Change-Id: I1a48b7c309a4e2aa0aa513f9be44ae3504d9e385
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462574
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-22 14:10:32 +00:00
Darek Stojaczyk
fa3e0a68c1 ut: move all spdk_env_* mocks to test_env.c
Cleanup.

Change-Id: If6c24ef6fcea5531007e6d4f10ee896939ffaa12
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/460906
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-07-18 04:21:08 +00:00
paul luse
06f6c90626 bdev/crypto: add IO queueing for out of mem condition via bdev layer
Also made on the prints a DEBUG message instead and noticed the really
name that was being registered by this component so updated it to
make it look like the rest of SPDK.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I747a846cb365e7db49be50db941e83fb1b265ea0
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/460244
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-07-08 09:24:29 +00:00
paul luse
f499ef7191 ut/compress: misc unit test fixes
Required due to recent changes to functional code, unclear how
the UT passed without these.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I3ea4401ee1842ef577793338f1567d20d73ad8bc
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459868
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-07-05 11:56:03 +00:00
Pawel Kaminski
c72e3bfe5b ut/raid: remove negative tests for spdk_jsonrpc_begin_result()
spdk_jsonrpc_begin_result() will be refactored to always
return non-null in subsequent patch and those raid tests
will no longer make sense, so we remove them now.

Change-Id: Ifd4570af4daab9c96cd8701440366e292f879204
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459659
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-04 09:05:19 +00:00
yidong0635
90513d3c5f bdev_ut.c: Fix scanbuild issue about potential memory leak.
Issue reports:
bdev_ut.c:1486:2: warning: Potential leak of memory pointed to by 'histogram'
        poll_threads();
        ^~~~~~~~~~~~~~
1 warning generated.

Add Tear down the channels like other tests in this file.

Change-Id: Iaeff93ec6f17716dfb7638cf4dd5f6cfe2c2de06
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459939
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Hailiang Wang <hailiangx.e.wang@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-07-04 00:41:01 +00:00
Tomasz Zawadzki
dae8e89253 test/ut: initialize spdk_bdev_desc to NULL
This fixes error on scan-build on Fedora 30, which assumes spdk_bdev_open()
is called from non-SPDK thread - failing before allocating spdk_bdev_desc:

bdev_ut.c:1766:17: warning: The left operand of '!=' is a garbage value
        CU_ASSERT(desc != NULL);

This is related to issue #822.

Change-Id: I8ee2112bb4d71aafc93d63bddca083c009ec11f3
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459755
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-07-04 00:26:17 +00:00
Hailiang Wang
9c3774681b test/bdev_raid_ut: fix a warning of a null pointer dereference
Compilation Warning on fedora30.
bdev_raid_ut.c:659:13: warning: Array access
(via field 'base_channel') results in a null pointer dereference
CU_ASSERT(ch_ctx->base_channel[index] == g_io_output[index].ch);
      ^~~~~~~~~~~~~~~~~~~~~~~~~~~
bdev_raid_ut.c:2181:14: warning: Array access
(via field 'base_channel') results in a null pointer dereference
CU_ASSERT(ch_ctx->base_channel[j] == (void *)0x1);
      ^~~~~~~~~~~~~~~~~~~~~~~
This is related to issue #822.

Change-Id: I9fd066b16e9f4c1fedc95403d2ac6e609f862c65
Signed-off-by: Hailiang Wang <hailiangx.e.wang@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/460094
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-07-04 00:21:43 +00:00
Changpeng Liu
efd7b514d4 bdev: rewind child offset to last block size aligned iov
Here is the an example to describe existing issue:

There is a Write request with 64KiB data length, and this IO is cross the IO
boundary.  We assume that the parent IO will have 2 children requests, one is
33KiB length, the other one is 31KiB.  Here is the view of parent iovs, the
first 33KiB length data has 33 iovs:

iov.[0].iov_length = 1024;
.
.
iov.[31].iov_length = 256;
iov.[32].iov_length = 768;
.
.
iov.[64].iov_length = 1024;

In function _spdk_bdev_io_split(), then you can see that for the 33KiB length
child request, exiting code will run out of child child_iov space and return
error due to last one data buffer is not block size aligned.

Here we can rewind the existing offset to last block size aligned buffer to
avoid the error case, for backend which need aligned data buffer such as
AIO backend, the request will go through spdk_bdev_io_get_buf() again to
do the data copy, otherwise for those backend devices such as NVMe with
hardware SGL support, 256 data segment is fine for them.

Change-Id: I96ebdf29829d86f9b38fab28a7406eedc9fa44ef
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453604
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-07-01 04:20:21 +00:00
Changpeng Liu
45f480535b UT/bdev: add IO boundary splitting unit test
Change-Id: If7c74dec31468b2fcd5fb761e5bc4f35a8d52f3a
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453195
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-06-24 04:51:46 +00:00
Konrad Sztyber
e2918289a1 lib/bdev: clear metadata when emulating write_zeroes
If bdev is configured to use separate buffers for metadata transfer,
but it doesn't support the write zeroes command, pass empty zeroed
buffer to make sure the metadata is cleared out as well.

Change-Id: If6f024266067e5764a28a276296f651d31da4792
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457628
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-06-19 03:53:02 +00:00
Maciej Szwed
622127d7e1 blobstore: Make possible to remove snapshot if there is only one clone
Starting with this patch it is possible to remove a
snapshot if there is only a one clone created from it.

In such case snapshot can be removed without any data
copying. This is achieved with following steps (case
with only one clone):
1. Open snapshot (Snapshot1) that shall be removed
2. Check if the Snapshot1 has no more than 1 clone (Clone1)
3. Remove Clone1 entry from Snapshot1
4. If the Snapshot1 has a parent snapshot (Snapshot2):
 4a. Add Clone1 entry to the Snapshot2 clones list
 4b. Remove Snapshot1 entry from Snapshot2 clones list
5. Open Clone1 blob
6. Freeze I/O operations on Clone1
7. Temporarily override md_ro flag for Snapshot1 and Clone1
   for MD modification
8. Merge Snapshot1 and Clone1 clusters maps into Clone1
   clusters map
9a. If Snapshot2 is present switch parent ID and backing
    bs_dev on Clone1
9b. If Snapshot2 is not present set parent ID to
    SPDK_BLOBID_INVALID and backing bs_dev to zeroes_dev
10. Sync MD on Clone1
11. Sync MD on Snapshot1
12. Restore MD flags for Clone1 and Snapshot1
13. Unfreeze I/O on Clone1
14. Close Clone1 blob
15. Remove Snapshot1

Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I800724b981af894e01e1912d0077c5b34a2ae634
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/445576
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-06-18 13:19:32 +00:00
paul luse
d5d7c57043 bvdev/compress: move comp transform structs to static globals
Code cleanup.

Change-Id: I9f96a889ed39afbd14a42b50e2537b1c4844f514
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456517
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-06-05 16:48:36 +00:00
paul luse
5dc25edde0 bdev/raid: fix order of operations bug in UT
Identified in earlier unrelated patch.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I890ce3d3411fd4e4b3d5d8a15616b5b39dc13a0e
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455733
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-06-04 02:57:18 +00:00
paul luse
cac906f322 bdev/raid: remove 1 bogus LOC from UT
Change-Id: I96925dbc9bc8d987801addfbbc62c2f0805ebc72
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/454654
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-06-04 02:57:18 +00:00
Changpeng Liu
9da1c7384d bdev: don't call spdk_bdev_free_io() for the error case
Existing code in spdk_bdev_write_zeroes_blocks() will call spdk_bdev_free_io()
for the error case, which will cause assertion because the bdev_io isn't
submitted to the backend yet, so we will check the condtion first to
avoid the error case.

Change-Id: If27d78217f709a3315e74c00869d345abd6b9a69
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453491
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-05-31 18:53:48 +00:00
paul luse
9fb414e2af bdev/compress: add missing .gitignore file for UT binary
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Id0e7707f61a920f5de08c860f5a533b73afc611a
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455274
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-05-30 17:56:54 +00:00
paul luse
4397e38782 bdev/compress: add UT for the poller
Adding these tests identified a small fix in the code under test
that is also included here.  The compressdev 'produced' field
is unsigned and reduce is expecting a negative errno in the
callback.

Change-Id: I28ab11ee3ef54768a9d6ccd26282cf7dd022be43
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/454806
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-05-22 15:36:48 +00:00
Changpeng Liu
c55c85f807 bdev: complete parent IO only when all the children IO are finished
When parent IO was splitted into several children requests, SPDK
may return parent completion callback with error status before
all the children requests are finished.

Change-Id: I63221a0ae1a5925a7fcd9744b4f5d8079c641252
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453611
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-05-22 08:18:40 +00:00
paul luse
45e17cb242 bdev/raid: drop max raids to 2
For UT there's no additional value in having more than 2 raid
devices, the value in having > 1 is in confirming the array
structures via create multiple and verify expected raids were
created.

Change-Id: I874b76644ccfc7d9dd307fbb179b35673b7bacc9
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/454508
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-05-20 17:02:35 +00:00
paul luse
9790bba16d bdev/compress: add UT for vbdev_init_compress_drivers()
Change-Id: I714aad7f82050b5ca480d2c51cffaaee8f059495
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452739
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-05-20 16:59:43 +00:00
paul luse
eb71364bce bdev/compress: remove unused test function, test_error_paths()
I'm testing the error paths within each function under test, this
was here simply because I copy-n-pasted much of the UT framework
from crypto to get it jump started.

Change-Id: Id7705bcaec87e93b589360e520b330f024d3293d
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452723
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-05-20 16:59:43 +00:00
paul luse
22760306a2 bdev/compress: add UT for vbdev_compress_submit_request
Note that much of the vbdev module is still WIP including this
function (ex there are rc checks at the end when nobody yet sets
an rc) but I want to catch up on UT before adding more new code,
or at least get close.

Change-Id: I4617c215fed9fe35a68dcc5e7ebc93e48588cf5b
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452715
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-05-20 16:59:43 +00:00
paul luse
3671369a1f bdev/compress: Add UT coverage for _compress_operation()
Aas well as general test setup required for this one, additional
mocks, etc.

Had to add some asserts to make scan-build happy although I don't
see how these changes stimulated those failures.

Change-Id: Ief08f9b71ee7a836f6026d26517f5faa5f9d51ce
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/451688
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-05-20 16:59:43 +00:00
lorneli
598ba408a2 bdev: add test case for splitting iovs that exceeds child_iov
Test multi vector command that needs to be split by strip and then
needs to be split further due to the capacity of child iovs.

Add a case that was not tested before. In this case, the length of the
rest of iovec array with an I/O boundary is the multiple of block size.
Expect the rest of iovec array to be submitted in the completion of
previous iovec array.

Change-Id: I5b95b1f1884a73b31709b2fd9187a8a9e9b2cd0b
Signed-off-by: lorneli <lorneli@163.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452414
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-05-20 09:37:59 +00:00
paul luse
871c805373 bdev/raid: gather all DEFINE_STUB macros and put them up front
For readability.

Change-Id: I3ce73bb4ed1dceb45420caecf67b6f775e4c6aad
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/454519
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-05-16 20:29:28 +00:00
paul luse
0cbd052925 bdev/raid: remove randomness from test_write_io()
Test 2 fixed IO sizes instead.

Change-Id: Idc909a6054d99cef74dd2ddad5680ef371d6f649
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/454512
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-05-16 20:29:28 +00:00
paul luse
3a1f7cb923 bdev/raid: remove randomness from test_read_io()
Use 2 fixed IO sizes instead.

Change-Id: I4f82108393703860c62fcde2f7d822a97cf2bb40
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/454513
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-05-16 20:29:28 +00:00
paul luse
3ac37f5db0 bdev/raid: remove unused stubs
Unclear why they were here to begin with.

Change-Id: Ic9b896796a9c56552972e3ef8075934d4b3f6497
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/454515
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-05-16 20:29:28 +00:00
paul luse
41730a8dbe bdev/raid: remove random IO sizes from test_io_failure()
Use 2 fixed sizes instead.

Change-Id: I617835311d3649e03b115bf528ac2a625ff5ec04
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/454514
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-05-16 20:29:28 +00:00
paul luse
4c3c840127 bdev/raid: use fixed blockcnt instead of random
Change-Id: I89cba64a265705aff20e647e7064f70516f8b4c7
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/454510
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-05-16 20:29:28 +00:00