Commit Graph

298 Commits

Author SHA1 Message Date
paul luse
ca1a3bf18e bdev/compress: use base bdev alias as base for comp_bdev name
If available, if not use the bdev unique name. This results in
a much friendlier comp_bdev naem for the user.  For example,
now it would look something like this: COMP_lvs0/lvs as opposed
to like this COMP_0b149b31-b66b-4cf7-ab39-a55b50788cd1

Change-Id: I319a141221ed8880edcec930ca5be9f256b105e0
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/463246
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-07-26 19:47:33 +00:00
Darek Stojaczyk
b701087f87 ut/bdev: silence scan-build warnings about garbage value comparison
bdev_ut.c:1300:17: warning: The left operand of '!=' is a garbage value
        CU_ASSERT(desc != NULL);
                  ~~~~ ^

Change-Id: Ie87cd4d7218a380e61616a83f3a9bceccc0798cb
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/463258
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-07-26 17:47:34 +00:00
Darek Stojaczyk
26f6c2a03d ut/bdev: silence scan-build warnings about histogram memory leaks
bdev_ut.c:1863:2: warning: Potential leak of memory pointed to by
'histogram'
        poll_threads();
        ^~~~~~~~~~~~~~

We used to free g_histogram instead of histogram. Those
should be the same thing, but scan-build gets confused.

Change the code to free histogram - this should also make
it slightly easier to read.

Change-Id: Ifaad1c2d9c7f9cc1a106f6edf3d22b54e42fe867
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/463256
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-07-26 17:47:34 +00:00
yidong0635
e3b90fe2d5 bdev: Fix warning about potenial memory leak about histogram.
Pointer passthrough g_histogram and histogram implicitly ,but the checker
still complains about this, change to spdk_histogram_data_free(histogram)
directly, no need any assert any more.

bdev_ut.c:1486:2: warning: Potential leak of memory pointed to by 'histogram'
        poll_threads();
        ^~~~~~~~~~~~~~
1 warning generated.

Change-Id: I609c1c92643ad03b8e3c62cd8434882e7942efa4
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462789
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-23 03:50:05 +00:00
yidong0635
2ab927030d bdev/vbdev_lvol: Fix scanbuild error about Null poniter parameter.
Issue reorts:
vbdev_lvol_ut.c:97:7: warning: Null pointer passed as an argument to a 'nonnull' parameter
                if (strncmp(alias, tmp->alias, SPDK_LVOL_NAME_MAX) == 0) {
                    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Change-Id: I0cc95c735ea7486263842f91a5866710f9ff61a5
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462598
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-07-23 00:17:25 +00:00
yidong0635
a1de2fe007 compress: Clear some simple scanbuild errors.
On Fedora30, GCC9+ reports errors:
Issues report:
vbdev_compress.c:482:17: warning: Assigned value is garbage or undefined
        comp_op->m_src = src_mbufs[0];
                       ^ ~~~~~~~~~~~~
vbdev_compress.c:502:17: warning: Assigned value is garbage or undefined
        comp_op->m_dst = dst_mbufs[0];
                       ^ ~~~~~~~~~~~~
vbdev_compress.c:491:3: warning: 1st function call argument is an uninitialized value
                rte_pktmbuf_attach_extbuf(dst_mbufs[iov_index],

compress_ut.c:826:2: warning: Attempt to free released memory
        free(g_mbuf_mp);
        ^~~~~~~~~~~~~~~
Should remove free(g_mbuf_mp), for pointer exchange.

Change-Id: I1a48b7c309a4e2aa0aa513f9be44ae3504d9e385
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462574
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-22 14:10:32 +00:00
Darek Stojaczyk
fa3e0a68c1 ut: move all spdk_env_* mocks to test_env.c
Cleanup.

Change-Id: If6c24ef6fcea5531007e6d4f10ee896939ffaa12
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/460906
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-07-18 04:21:08 +00:00
paul luse
06f6c90626 bdev/crypto: add IO queueing for out of mem condition via bdev layer
Also made on the prints a DEBUG message instead and noticed the really
name that was being registered by this component so updated it to
make it look like the rest of SPDK.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I747a846cb365e7db49be50db941e83fb1b265ea0
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/460244
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-07-08 09:24:29 +00:00
paul luse
f499ef7191 ut/compress: misc unit test fixes
Required due to recent changes to functional code, unclear how
the UT passed without these.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I3ea4401ee1842ef577793338f1567d20d73ad8bc
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459868
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-07-05 11:56:03 +00:00
Pawel Kaminski
c72e3bfe5b ut/raid: remove negative tests for spdk_jsonrpc_begin_result()
spdk_jsonrpc_begin_result() will be refactored to always
return non-null in subsequent patch and those raid tests
will no longer make sense, so we remove them now.

Change-Id: Ifd4570af4daab9c96cd8701440366e292f879204
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459659
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-04 09:05:19 +00:00
yidong0635
90513d3c5f bdev_ut.c: Fix scanbuild issue about potential memory leak.
Issue reports:
bdev_ut.c:1486:2: warning: Potential leak of memory pointed to by 'histogram'
        poll_threads();
        ^~~~~~~~~~~~~~
1 warning generated.

Add Tear down the channels like other tests in this file.

Change-Id: Iaeff93ec6f17716dfb7638cf4dd5f6cfe2c2de06
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459939
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Hailiang Wang <hailiangx.e.wang@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-07-04 00:41:01 +00:00
Tomasz Zawadzki
dae8e89253 test/ut: initialize spdk_bdev_desc to NULL
This fixes error on scan-build on Fedora 30, which assumes spdk_bdev_open()
is called from non-SPDK thread - failing before allocating spdk_bdev_desc:

bdev_ut.c:1766:17: warning: The left operand of '!=' is a garbage value
        CU_ASSERT(desc != NULL);

This is related to issue #822.

Change-Id: I8ee2112bb4d71aafc93d63bddca083c009ec11f3
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459755
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-07-04 00:26:17 +00:00
Hailiang Wang
9c3774681b test/bdev_raid_ut: fix a warning of a null pointer dereference
Compilation Warning on fedora30.
bdev_raid_ut.c:659:13: warning: Array access
(via field 'base_channel') results in a null pointer dereference
CU_ASSERT(ch_ctx->base_channel[index] == g_io_output[index].ch);
      ^~~~~~~~~~~~~~~~~~~~~~~~~~~
bdev_raid_ut.c:2181:14: warning: Array access
(via field 'base_channel') results in a null pointer dereference
CU_ASSERT(ch_ctx->base_channel[j] == (void *)0x1);
      ^~~~~~~~~~~~~~~~~~~~~~~
This is related to issue #822.

Change-Id: I9fd066b16e9f4c1fedc95403d2ac6e609f862c65
Signed-off-by: Hailiang Wang <hailiangx.e.wang@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/460094
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-07-04 00:21:43 +00:00
Changpeng Liu
efd7b514d4 bdev: rewind child offset to last block size aligned iov
Here is the an example to describe existing issue:

There is a Write request with 64KiB data length, and this IO is cross the IO
boundary.  We assume that the parent IO will have 2 children requests, one is
33KiB length, the other one is 31KiB.  Here is the view of parent iovs, the
first 33KiB length data has 33 iovs:

iov.[0].iov_length = 1024;
.
.
iov.[31].iov_length = 256;
iov.[32].iov_length = 768;
.
.
iov.[64].iov_length = 1024;

In function _spdk_bdev_io_split(), then you can see that for the 33KiB length
child request, exiting code will run out of child child_iov space and return
error due to last one data buffer is not block size aligned.

Here we can rewind the existing offset to last block size aligned buffer to
avoid the error case, for backend which need aligned data buffer such as
AIO backend, the request will go through spdk_bdev_io_get_buf() again to
do the data copy, otherwise for those backend devices such as NVMe with
hardware SGL support, 256 data segment is fine for them.

Change-Id: I96ebdf29829d86f9b38fab28a7406eedc9fa44ef
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453604
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-07-01 04:20:21 +00:00
Changpeng Liu
45f480535b UT/bdev: add IO boundary splitting unit test
Change-Id: If7c74dec31468b2fcd5fb761e5bc4f35a8d52f3a
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453195
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-06-24 04:51:46 +00:00
Konrad Sztyber
e2918289a1 lib/bdev: clear metadata when emulating write_zeroes
If bdev is configured to use separate buffers for metadata transfer,
but it doesn't support the write zeroes command, pass empty zeroed
buffer to make sure the metadata is cleared out as well.

Change-Id: If6f024266067e5764a28a276296f651d31da4792
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457628
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-06-19 03:53:02 +00:00
Maciej Szwed
622127d7e1 blobstore: Make possible to remove snapshot if there is only one clone
Starting with this patch it is possible to remove a
snapshot if there is only a one clone created from it.

In such case snapshot can be removed without any data
copying. This is achieved with following steps (case
with only one clone):
1. Open snapshot (Snapshot1) that shall be removed
2. Check if the Snapshot1 has no more than 1 clone (Clone1)
3. Remove Clone1 entry from Snapshot1
4. If the Snapshot1 has a parent snapshot (Snapshot2):
 4a. Add Clone1 entry to the Snapshot2 clones list
 4b. Remove Snapshot1 entry from Snapshot2 clones list
5. Open Clone1 blob
6. Freeze I/O operations on Clone1
7. Temporarily override md_ro flag for Snapshot1 and Clone1
   for MD modification
8. Merge Snapshot1 and Clone1 clusters maps into Clone1
   clusters map
9a. If Snapshot2 is present switch parent ID and backing
    bs_dev on Clone1
9b. If Snapshot2 is not present set parent ID to
    SPDK_BLOBID_INVALID and backing bs_dev to zeroes_dev
10. Sync MD on Clone1
11. Sync MD on Snapshot1
12. Restore MD flags for Clone1 and Snapshot1
13. Unfreeze I/O on Clone1
14. Close Clone1 blob
15. Remove Snapshot1

Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I800724b981af894e01e1912d0077c5b34a2ae634
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/445576
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-06-18 13:19:32 +00:00
paul luse
d5d7c57043 bvdev/compress: move comp transform structs to static globals
Code cleanup.

Change-Id: I9f96a889ed39afbd14a42b50e2537b1c4844f514
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456517
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-06-05 16:48:36 +00:00
paul luse
5dc25edde0 bdev/raid: fix order of operations bug in UT
Identified in earlier unrelated patch.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I890ce3d3411fd4e4b3d5d8a15616b5b39dc13a0e
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455733
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-06-04 02:57:18 +00:00
paul luse
cac906f322 bdev/raid: remove 1 bogus LOC from UT
Change-Id: I96925dbc9bc8d987801addfbbc62c2f0805ebc72
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/454654
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-06-04 02:57:18 +00:00
Changpeng Liu
9da1c7384d bdev: don't call spdk_bdev_free_io() for the error case
Existing code in spdk_bdev_write_zeroes_blocks() will call spdk_bdev_free_io()
for the error case, which will cause assertion because the bdev_io isn't
submitted to the backend yet, so we will check the condtion first to
avoid the error case.

Change-Id: If27d78217f709a3315e74c00869d345abd6b9a69
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453491
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-05-31 18:53:48 +00:00
paul luse
9fb414e2af bdev/compress: add missing .gitignore file for UT binary
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Id0e7707f61a920f5de08c860f5a533b73afc611a
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455274
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-05-30 17:56:54 +00:00
paul luse
4397e38782 bdev/compress: add UT for the poller
Adding these tests identified a small fix in the code under test
that is also included here.  The compressdev 'produced' field
is unsigned and reduce is expecting a negative errno in the
callback.

Change-Id: I28ab11ee3ef54768a9d6ccd26282cf7dd022be43
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/454806
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-05-22 15:36:48 +00:00
Changpeng Liu
c55c85f807 bdev: complete parent IO only when all the children IO are finished
When parent IO was splitted into several children requests, SPDK
may return parent completion callback with error status before
all the children requests are finished.

Change-Id: I63221a0ae1a5925a7fcd9744b4f5d8079c641252
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453611
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-05-22 08:18:40 +00:00
paul luse
45e17cb242 bdev/raid: drop max raids to 2
For UT there's no additional value in having more than 2 raid
devices, the value in having > 1 is in confirming the array
structures via create multiple and verify expected raids were
created.

Change-Id: I874b76644ccfc7d9dd307fbb179b35673b7bacc9
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/454508
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-05-20 17:02:35 +00:00
paul luse
9790bba16d bdev/compress: add UT for vbdev_init_compress_drivers()
Change-Id: I714aad7f82050b5ca480d2c51cffaaee8f059495
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452739
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-05-20 16:59:43 +00:00
paul luse
eb71364bce bdev/compress: remove unused test function, test_error_paths()
I'm testing the error paths within each function under test, this
was here simply because I copy-n-pasted much of the UT framework
from crypto to get it jump started.

Change-Id: Id7705bcaec87e93b589360e520b330f024d3293d
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452723
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-05-20 16:59:43 +00:00
paul luse
22760306a2 bdev/compress: add UT for vbdev_compress_submit_request
Note that much of the vbdev module is still WIP including this
function (ex there are rc checks at the end when nobody yet sets
an rc) but I want to catch up on UT before adding more new code,
or at least get close.

Change-Id: I4617c215fed9fe35a68dcc5e7ebc93e48588cf5b
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452715
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-05-20 16:59:43 +00:00
paul luse
3671369a1f bdev/compress: Add UT coverage for _compress_operation()
Aas well as general test setup required for this one, additional
mocks, etc.

Had to add some asserts to make scan-build happy although I don't
see how these changes stimulated those failures.

Change-Id: Ief08f9b71ee7a836f6026d26517f5faa5f9d51ce
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/451688
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-05-20 16:59:43 +00:00
lorneli
598ba408a2 bdev: add test case for splitting iovs that exceeds child_iov
Test multi vector command that needs to be split by strip and then
needs to be split further due to the capacity of child iovs.

Add a case that was not tested before. In this case, the length of the
rest of iovec array with an I/O boundary is the multiple of block size.
Expect the rest of iovec array to be submitted in the completion of
previous iovec array.

Change-Id: I5b95b1f1884a73b31709b2fd9187a8a9e9b2cd0b
Signed-off-by: lorneli <lorneli@163.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452414
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-05-20 09:37:59 +00:00
paul luse
871c805373 bdev/raid: gather all DEFINE_STUB macros and put them up front
For readability.

Change-Id: I3ce73bb4ed1dceb45420caecf67b6f775e4c6aad
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/454519
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-05-16 20:29:28 +00:00
paul luse
0cbd052925 bdev/raid: remove randomness from test_write_io()
Test 2 fixed IO sizes instead.

Change-Id: Idc909a6054d99cef74dd2ddad5680ef371d6f649
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/454512
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-05-16 20:29:28 +00:00
paul luse
3a1f7cb923 bdev/raid: remove randomness from test_read_io()
Use 2 fixed IO sizes instead.

Change-Id: I4f82108393703860c62fcde2f7d822a97cf2bb40
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/454513
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-05-16 20:29:28 +00:00
paul luse
3ac37f5db0 bdev/raid: remove unused stubs
Unclear why they were here to begin with.

Change-Id: Ic9b896796a9c56552972e3ef8075934d4b3f6497
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/454515
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-05-16 20:29:28 +00:00
paul luse
41730a8dbe bdev/raid: remove random IO sizes from test_io_failure()
Use 2 fixed sizes instead.

Change-Id: I617835311d3649e03b115bf528ac2a625ff5ec04
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/454514
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-05-16 20:29:28 +00:00
paul luse
4c3c840127 bdev/raid: use fixed blockcnt instead of random
Change-Id: I89cba64a265705aff20e647e7064f70516f8b4c7
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/454510
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-05-16 20:29:28 +00:00
paul luse
32e69f2968 bdev/raid: remove randomness from test_multi_raid_with_io()
No additional value in randomizing read vs write, which raid and
which channel. Still do read and write but fix the other values.

Change-Id: I5a4f023731119230d3eb49ae28421819144b90bc
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/454509
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-05-16 20:29:28 +00:00
paul luse
c009c2eb4a bdev/raid: remove queue depth param from unit tests
Having a varying QD in unit test isn't adding any value, the same
code is being executed N times with no specific assertions on the
difference between queue depths. Qdepth testing will be looked at
in system test as an opportunity to increase SW quality.

Change-Id: I5328a43f57cbfa2afc9145c828b6e968dcab310d
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/454507
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-05-16 20:29:28 +00:00
paul luse
17de9bfdd6 bdev/raid: remove random test case values
For test setup only, each time a UT would run several params, like
the number of drives in the test, would be based on a random value
which is not optimal for UT. This patch just selects a reasonable
fixed values and uses them every time.

This can save up to 4 seconds per run depending on the values that
were randomly chosen (worst case) vs what is in this patch.

There is value in some read/write test cases in using different
strip sizes however. Enabling multiples values for the tests where
it adds value will come in following patches as will removal of
more random numbers like # of channels.

Change-Id: I1b9d44b70ae81af0dcf53ae4a56e31c9c9add39e
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/454505
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-05-16 20:29:28 +00:00
paul luse
ad7f7bf587 bdev/raid: use mock macros in UT
General cleanup, first in a series of patches coming soon...

Change-Id: Ibed727559af22c6febb1aad2bb8320a765814144
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/454371
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-05-16 20:29:28 +00:00
Tomasz Zawadzki
d83a3489cc ut/bdev: change module initialization to async
This patch changes UT so that bdev modules have async init.
It demonstrates issue with triggering assert in init path,
when bdev modules are reinitialized multiple times.

Change-Id: I9b2b16d8ac53bac6a929f6929ceedb70b250c500
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/454618
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-05-16 08:29:19 +00:00
paul luse
7b641c757d bdev/compress: change how we manage mbufs
In working on a future patch to queue compress operations, I found
some corruptions in our mbuf pool stemming from an inconsistency
in using mbuf macros when constructing a compress operation and
how we allocated and freed both the pool and the mbufs.

This also fixes an issue that I hadn't addressed yet with freeing
chained mbufs.  Use of the mbuf macro to free them instead of the
generic spdk functions takes care of that for us.

Change-Id: I7b29a0d740ab745574698c721d575d8a735ad5cb
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453028
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-05-13 15:41:35 +00:00
paul luse
024e89afb3 bdev/compress: Populate params structure on load
On init, we pass reducelib the address of our params structure
so that the volume size is updated accordingly.  On load however,
there was nothing to set the volume size so any load of a previously
created compress volume would result in a 0 length bdev.

Change-Id: I7643c79fa2e664115fd25df064ff1c74d82e358b
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452906
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-05-13 15:41:35 +00:00
Mike Carlin
a6524cd1f8 raid bdev: Change product_name to "Raid Volume"
The product_name for raid bdevs was "Pooled Device" which was a legacy
naming convention that hadn't been cleaned up yet. This changes the
naming to be up to date with the current naming scheme.

Signed-off-by: Mike Carlin <mikefcarlin@protonmail.com>
Change-Id: I9092a2b793e48bb9ec0349087a31fdcde17ed9cd
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452269
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-05-02 09:02:52 +00:00
Jim Harris
617184be3b trace: remove short_name
This was not used by any of the trace register descriptions.
Let's remove it rather keeping it around if we don't need it.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Idda809e2911db5be555ff6aa13695484a14bf665

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452734
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2019-05-02 08:41:56 +00:00
Jim Harris
dd1c38cc68 trace: allow arg1 to represent a string
This requires changing arg1_is_ptr to arg1_type.
We will use this to print the first 8 characters of
a blobfs filename when collecting event trace data.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I1b321d99145e82b42dcf6d901ce9d6298158edae

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452259
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-05-01 17:10:43 +00:00
paul luse
933270eafd bdev/crypto: add new uninit call needed for dpdk 19.02
This patch won't fix issues related to the need for the latest
ipsec with 19.02 but it is needed for that patch to pass.

Change-Id: I8e2d984d85b355d88edfb9c90c159c8aa0a5362d
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/451788
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-04-24 22:00:10 +00:00
Darek Stojaczyk
283abcb9a2 bdev: temporarily allow bdev descriptors to be closed from any thread
Bdev descriptors could be closed only from the same thread
that opened them. This restriction was suddenly introduced
at one point without making sure all the SPDK code respects
it. Vhost can still close descriptors from any arbitrary
thread and fixing that would require some more effort.

With this patch we remove the thread-specific assert from
spdk_bdev_close() and hence allow vhost to work properly
in debug builds. Vhost can still have a possible data
race with bdev hotremove notification, but let's get rid
of the abort() from the usual code path first.

Change-Id: I6fac66a5ebc907b1c5418fff618f0b64cd9b69f4
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/451561
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-04-23 17:06:37 +00:00
Darek Stojaczyk
f93b6fb0a4 bdev: defer unregister untill *all* descriptors are closed
We used to wait only for those descriptors which
specified the hotremove notification callback. The
bdev could've been removed before the descriptor
was closed and the subsequent spdk_bdev_close would
simply segfault.

This patch modifies spdk_bdev_unregister to always
wait for all descriptors to be closed before actually
unregistering the bdev. This consolidates the bdev
unregister behavior for descriptors with and without
the hotremove callback.

Change-Id: I9b358209c6abd301b6fe8660e27bc6fa4ef485d6
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/450175
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-04-23 17:06:37 +00:00
paul luse
766c536184 bdev/compress: add missing mocks and re-enable UT build for comp
Disabled temporarily earlier to get to basic functionality, circling
back now to begin work on UT again.

Change-Id: Ie7606f91072257f392727bdecc5f1eac26380453
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/451063
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-04-22 15:44:38 +00:00