Commit Graph

1024 Commits

Author SHA1 Message Date
Pawel Wodkowski
c4fee1e970 mk: don't use '-include spdk/config.h'
Each file that need to check SPDK_CONFIG_* options need to include
spdk/config.h explicitly.

Change-Id: If9f2a91ac4c2b1a300dcf88ec3e2a12714ad344a
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/427221
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-10-02 23:13:32 +00:00
GangCao
7d030ef7fc QoS/Bdev: add the QoS related structure and enumeration
This patch is to introduce the specific QoS related structure
and the enumeration for types of QoS rate limits. Later new
types of QoS rate limits can be supported easily.

Change-Id: Idb8d2e7627fd145bf2b0ddb296c968b6b068f48c
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.gerrithub.io/424459
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-10-02 22:10:13 +00:00
paul luse
7bcdc55926 crypto: Use bulk API to return mbufs and crypto ops in poller
Performance testing shows much better CPU scalling with FIO as
DPDK will no longer take multiple locks with each loop.  Instead,
just 2 at the end to perform the bulk returns.

Change-Id: Ie4d556264302f01c2a9498be83b7bccbaf16b287
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/427158
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-10-01 15:55:36 +00:00
Shuhei Matsumoto
6fa7e38667 bdev: Avoid assert and factor out queue IO operation in _bdev_write_zero_buffer_next
Currently write to bdev fails in _spdk_bdev_write_zero_buffer_next
due to other than -ENOMEM, assert is called.

RAID bdev using this feature is generally availale now and it will be
OK to remove this assert and return error instead.

Additionally, applying the factored function to _bdev_write_zero_buffer_next
will improve readability slightly.

These two changes are done in this patch.

Change-Id: I462630a71e57e2e5146b085b215d62a378ea9402
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/427186
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-09-29 03:12:24 +00:00
Shuhei Matsumoto
9d4708f35e bdev: Factor out queueing IO operation in _spdk_bdev_io_split
This patch factors out that operation into a function and adds
error handling to that operation to improve readability slightly.

Change-Id: Ic24df0c0a9abbebc38d30fc17779dc5a5f6138a6
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/427026
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2018-09-29 03:12:24 +00:00
Shuhei Matsumoto
9872b99206 bdev: Avoid assert when read/write to bdev fails in _spdk_bdev_io_split
Currently when read/write to bdev fails in _spdk_bdev_io_split_with_payload
due to other than -ENOMEM, assert is called.

RAID bdev that utilizes the split IO feature is generally availale now
and it will be OK to remove this assert and return error instead.

Change-Id: I6ea6fd45b94bff0ea84e498e0c4dfd1dd31e0260
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/427025
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-09-29 03:12:24 +00:00
Shuhei Matsumoto
0df515a842 bdev: Remove limitation of child iov size in bdev_io_split_with_payload()
When a bdev IO is split, if iovec size in a strip is more than 32,
the IO will fail.

Remove the limitation by spliting the split IO further.

Change-Id: I962ad86dfe63ea1fcd86ffa52ead7452fb80e53d
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/425876
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2018-09-29 03:12:24 +00:00
paul luse
b5c5d14506 crypto: add error handling to claim function
When adding RPC support, did some refactoring and totally missed
error handling in the claim function.  This patch adds it.

Addresses github issue #445≠

Change-Id: Ie2df279af7bfa3660a4a9fe2fd8bcbf7b5fd07fd
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/427170
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-09-28 20:21:02 +00:00
paul luse
6de9698dec crypto: gather loop condition variables into one spot in _crypto_operation
Readability changes in _crypto_operation() as suggested in a previous
review.

Change-Id: I8e71be297ae42d78e5c98302838625b80f319848
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/426598
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-09-28 19:35:57 +00:00
paul luse
18aeb14fe0 crypto: fix file permissions
Somehow I ended up with x permissions on some source files,
maybe copied them to a windows box or something, don't know..

Change-Id: I2c40d9fa0094a3ae56332ce165750f1fcc644b15
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/427276
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-09-28 18:50:16 +00:00
paul luse
369719c26b crypto: move PMD driver input validation to common function
Previously was only checking the crypto driver name in the conf
file parsing and not when it came in via RPC so a bogus name
could result in nasty stuff.  Moved the check to the common
function used by both paths so a bad name will fail gracefully.

Addresses github issue #444

Change-Id: Id881d9d448d0bb6935162484154964a1d5d59a0b
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/427164
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-09-28 18:43:04 +00:00
paul luse
e42bb743f9 crypto: Misc cleanup from final review
* fixed typo in comment
* cleanup up some current_len logic that was no longer needed
after changing to LBA as IV
* moved a few src_mbufs assignments so they are grouped together
for easier reading
* updated vbdev_crypto_io_type_supported() to positively decode
supported IO types
* updated vbdev_crypto_dump_info_json() to use json _named_* API

Re-tested with overnight bdevperf OIO 1...256 sizes 512..65536
as well as bdevio and fio (basic multi-threaded tests)

Change-Id: I86fa177927ce61d12c86d352b00d850ffa878929
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/425994
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-09-28 18:30:20 +00:00
paul luse
cafd537c7d test: exercise crypto rpc calls
Most vbdevs have complete test suites that include rpc calls
to set them up and teare them down.  With crypto there's not
a lot to do w/test other than make sure it's configured and
run IO so right now that amounts to tools that use a config
(ini) file.  Adding create/delete calls here to get basic rpc
coverage in CI.

The fixes/udpates in the vbdev module and the .py files were
fixes found in enabling testing of the rpc calls.

Change-Id: I5d7d25e1c96dcf76e0055f23d27781b858e290b0
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/425875
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-09-27 22:07:02 +00:00
Dariusz Stojaczyk
1e4f9974a7 bdev: do not finish unitialized modules
To achieve its goal, this patch changes the order
in which bdev modules are finished. All modules
that examine bdevs (e.g. lvol,split,...) will be now
finished last. It should not cause any issues though,
since all bdevs are already removed at the time when
any module finish is called

Fixes #387

Change-Id: Id60c375eb5c3d7306b69cdce86bded77354868d8
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/421158
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-09-26 21:11:51 +00:00
Shuhei Matsumoto
ef80470338 bdev/raid: Add .INI config dump
Add dump the current configuration of raid bdevs to the specified
file for .INI config file.

.INI config file will be deprecated but this addition will be helpful
for now.

Change-Id: I35582ceaff18d81b066e3e63e0c8c22525ec6b9e
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/426299
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Hailiang Wang <hailiangx.e.wang@intel.com>
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-09-26 15:39:37 +00:00
Shuhei Matsumoto
ed6a63d569 bdev/raid: Add JSON config dump
Change-Id: I5b2832aad585145e021e26ac288da91ece2153a9
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/426289
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-09-26 15:39:37 +00:00
Shuhei Matsumoto
9d1b8599ea bdev/raid: Remove duplicated error check in construct RPC
This part was duplicated but was not removed during refactoring caressly.

Additionally, the variable iter was not changed to the single character
variable i too.

Fix them in this patch.

Change-Id: I1e026434ce48e9e8d492a3aa772c3be6df0b1d37
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/425528
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-09-26 15:39:37 +00:00
Shuhei Matsumoto
bc3dfe3043 bdev: Fix the second parameter success of spdk_bdev_io_completion_cb
The type of the second parameter `success' of spdk_bdev_io_completion_cb
is bool. Hence change the code to use bool type success or failure.

Change-Id: I9e93f4ccbb085e8e184f209e706915dcd34aa966
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/426648
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-09-26 09:03:15 +00:00
Ni Xun
1bc3837cdf vbdev_passthru: add support for bdev_io_wait
according to commit:
     bdev: add spdk_bdev_queue_io_wait
This patch will make io_wait to support vbdev_passthru

Change-Id: I282b52a7dc0e0fdbfe79e570b0c6a84b01c390ea
Signed-off-by: Ni Xun <nixun@baidu.com>
Signed-off-by: Li Lin <lilin24@baidu.com>
Signed-off-by: Zhang Yu <zhangyu31@baidu.com>
Reviewed-on: https://review.gerrithub.io/426163
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-09-25 17:56:39 +00:00
Wael Halbawi
e80b3d5f13 bdev: Fix spdk_bdev_part_io_type_supported()
The function spdk_bdev_part_io_type_supported() internally invokes
io_type_supported() as provided in the base bdev's function table.
This function should be called with the base bdev's context instead
of the bdev itself.

Change-Id: I5fbda0521f05b86dd81725614c5e45ee170cd8f5
Signed-off-by: Wael Halbawi <waelhalbawi@gmail.com>
Reviewed-on: https://review.gerrithub.io/426306
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-09-25 05:07:45 +00:00
Jim Harris
1d64773c2a raid: enable by default
We kept this disabled by default originally because it lacked
multiple iov support needed for vhost and nvmf in-capsule data.
Now that support has been added, there is no reason to not
enable it automatically - it has no extra package dependencies,
so requiring users to specify --with-raid to configure is just
an extra unneeded step.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ia9796370e12a6f93993e70083de36114a37851bd

Reviewed-on: https://review.gerrithub.io/426003
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-09-19 17:22:31 +00:00
Jim Harris
2484d0d1cd bdev/nvme: write TransportID to config file
It was writing TransportId (lower-case d) instead.

While here, fix similar issue in etc/spdk/nvmf.conf.in.

lib/conf uses strcasecmp(), meaning TransportID and
TransportId are effectively the same, but it's better
to reduce confusion and just use the same in all places.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I2996ba46a5c2a239650e841d5b57424ff00f47e6

Reviewed-on: https://review.gerrithub.io/426001
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: GangCao <gang.cao@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-09-19 17:22:31 +00:00
Pawel Wodkowski
4dc0ef6f45 scripts: fix executable attribute for includes
rpmbuild pointed out that this is wrong to have executable files without
shebang. Taking the opportunity and fix this for other files too.

Change-Id: Ib21f436672150edc0aff511bff2eb6839870cf79
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/425382
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-09-18 16:34:45 +00:00
Paul Luse
51606ed402 bdev: Add crypto virtual bdev module
Initial support for softare AESNI_MB DPDK driver only.

Have tested (both aesni and QAT seprately and concurrrently) on underlying NVMe devices
with bdevio and a bdevperf script that runs IOs from 512B to 128K each with Q depths from
1 to 512 in powers of 2 for 30 seconds each run.

QAT can be included in the code (but not makefile) and marked as experimental
until we are ready to test in CI.  It works well on 2 systems but is a big PITA to get
the hardware setup and configured for use with DPDK (IMHO).

Change-Id: If518c3df8e74e00efa18afdf194824c5e69778fc
Signed-off-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/403107
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-09-17 21:23:14 +00:00
Piotr Pelplinski
fbb599288f lvolstore: Modify vbdev_lvol to use io_unit size instead of page size
Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Change-Id: I0276528c360874eefa7f2f1ecb8a35fa57cee669

Reviewed-on: https://review.gerrithub.io/425357
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-09-14 22:46:27 +00:00
Jim Harris
10ee6eecb3 bdev/raid: optimize for single base bdev case
We do not need to split I/O to single base bdev
RAID, since strips do not really apply.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I0fa741f627b5aeb15d2ab19365afac4f48248654

Reviewed-on: https://review.gerrithub.io/425322
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Kunal Sablok <kunal.sablok@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-09-12 13:18:12 +00:00
Jim Harris
13c3b147dc bdev/raid: rename some of the I/O submission path functions
These new names are a bit more clear after some of the recent
I/O splitting changes.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ibe265ca2f969ee1058316c519e5d679861e03713

Reviewed-on: https://review.gerrithub.io/425321
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-09-12 13:18:12 +00:00
Jim Harris
f2a9507af6 bdev/raid: enable multiple iovs
The bdevio tests will now work on RAID volumes, so enable
RAID testing there too.

It is probably safe now to just build RAID by default, and
not require --with-raid - but let's do some more testing
with vhost and NVMe-oF against it first before we do so.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I15697f035cb688574a7ecb8be24d0c84fc622d83

Reviewed-on: https://review.gerrithub.io/423408
Reviewed-by: Kunal Sablok <kunal.sablok@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-09-12 13:18:12 +00:00
Jim Harris
4c60fa994f bdev/raid: add reset handling
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I2732039bd062dba3fdbf1929f92073316fc6462b

Reviewed-on: https://review.gerrithub.io/423407
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Kunal Sablok <kunal.sablok@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-09-11 19:00:29 +00:00
Shuhei Matsumoto
6bc4a314ae bdev/raid: Update the comment of raid_bdev_add_base_bdevs()
Implementation was updated by the previous patch series but the
comment was not. Hence update the comment in this patch.

Change-Id: Ib2097561501984c4ad213652ba716ce195c7bea5
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/424737
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Kunal Sablok <kunal.sablok@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-09-11 19:00:05 +00:00
Shuhei Matsumoto
7f78f68ec8 bdev/raid: Use helper function to get config in destroy_raid_bdev RPC
Use helper function raid_bdev_config_find_by_name() in destroy_raid_bdev
RPC.

Besides, unify two iterator variables in construct_raid_bdev RPC.
This is too small to add in a separate patch.

Change-Id: If0bbae8993af4e402a82299db378e89245b8966e
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/424150
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Kunal Sablok <kunal.sablok@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-09-11 19:00:05 +00:00
Shuhei Matsumoto
dbc4e5094a bdev/raid: Refactor raid_bdev_config_destroy()
raid_bdev_config_destroy_check_raid_bdev_exists() and
raid_bdev_config_destroy() do almost same operation and can be
unified to raid_bdev_config_destroy().

Comment in spdk_rpc_destroy_raid_bdev() about raid_bdev_config_destroy()
are duplicated with the one on top of the raid_bdev_config_destroy().
Hence remove the former in this patch too.

Change-Id: I4a323bf24d2266e2da7f030c52771013d075eb31
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/424149
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Kunal Sablok <kunal.sablok@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-09-11 19:00:05 +00:00
Shuhei Matsumoto
3f578219be bdev/raid: Extract adding base bdevs from RPC and apply it to parse .INI config
Initialization of bdev modules don't have fixed ordering.
Hence both cases should succeed to create configured raid bdev
for .INI config file.

1. create raid bdev and then create base bdevs of it.
2. before creating raid bdev, create base bdevs of it.

To do this, factor out adding base bdevs to raid bdev
to a helper function raid_bdev_add_base_devices() first
and then apply it to construct_raid_bdev RPC and parse .INI
config file.

Change-Id: I4389fd4e7bd41296dfa9210bbe8b60348ee5bc9d
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/423769
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Kunal Sablok <kunal.sablok@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-09-11 19:00:05 +00:00
Shuhei Matsumoto
a3004632e2 bdev/raid: Change construct_raid_bdev RPC to be consistent to other bdev modules
Change the behavior of construct_raid_bdev RPC to be consistent with
other bdev modules.

Create a config of raid bdev first. Then create a raid bdev based
on the config.

If both succeed, the config and the raid bdev are not removed even if
any failure occur later in construct_raid_bdev RPC. Otherwise, both
are removed and return failure.

During iteration of adding base bdevs in construct_raid_bdev RPC,
- skip any nonexistent bdev and move to the next base bdev.
- if adding any base bdev fails, move to the next bdev.
- if adding base bdevs don't fail but any base bdev doesn't exist,
  the raid bdev is in the configuring state and return success.
- if adding base bdev fails, the raid bdev is in the configuring state
  and return failure.
- if all adding base bdevs succeed, configure the raid bdev. If configuring
  the raid bdev fails, move the raid bdev to the offline state and return
  failure. If configuring the raid bdev succeed, return success.

check_and_remove_raid_bdev() becomes unused in raid_bdev_rpc.c but
is still necessary in UT. Hence move this function to UT.

In UT, finding a raid bdev config by name becomes necessary. Hence
factor out the iteration to a function and use the function in UT.

Change-Id: Ifa36967bdc987d97030e3a4e36684cb37b329d4e
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/423622
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Kunal Sablok <kunal.sablok@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-09-11 19:00:05 +00:00
Shuhei Matsumoto
50b51bc831 bdev/raid: Create raid bdev before starting to add base bdevs to it
When construct_raid_bdev RPC is callled, even if all adding base bdevs
to it fail, a raid bdev with no base bdev should exist and be in
the configuring state.

Extract raid_bdev_create() from raid_bdev_add_base_bdev() and put it
before starting iteration of raid_bdev_add_base_bdev().

This change cam be applied to .INI config file too.

Change-Id: Ia71aca0bbdb33049fe5bd9b8fd8163c99c032401
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/424148
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Kunal Sablok <kunal.sablok@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-09-11 19:00:05 +00:00
Seth Howell
60a208c004 iscsi: free lun after bdev_destruct_done returns
This prevents us from using the bdev embedded in the lun after it has
been freed.

Change-Id: I780cf3eccca05a58d3461366fec024be42b8ff74
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/425174
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-09-10 21:08:39 +00:00
Jim Harris
01e7c02e15 bdev: call spdk_bdev_io_get_buf before splitting
We cannot split an iov if a buffer hasn't been
allocated yet.  So always call spdk_bdev_io_get_buf
on reads before trying to split.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I2c26efb9dc6cb2c7c3e3b7ae5bab2c37844b9113

Reviewed-on: https://review.gerrithub.io/424879
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-09-07 14:59:21 +00:00
Jim Harris
de4d961996 bdev: set iovs on correct bdev_io in spdk_bdev_io_put_buf
spdk_bdev_io_put_buf() is responsible for reclaiming
bdev-allocated buffers from a bdev_io.  If there are
bdev_ios waiting for one of these buffers, it calls
spdk_bdev_io_set_buf() on the next bdev_io in the queue.
This will set the iov_base and iov_len on the bdev_io
to point to the bdev-allocated buffer.

But spdk_bdev_io_put_buf() was calling spdk_bdev_io_set_buf()
on the just completed bdev_io, not the next bdev_io in the
queue.  So fix that.

Fixes: 844aedf8 ("bdev: Simplify get/set/put buf functions")
Reported-by: Alan Tu
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ibbcad6e35a3db6991bd7deb3516229572f021638

Reviewed-on: https://review.gerrithub.io/424880
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-09-06 23:42:32 +00:00
wuzhouhui
f118de60af bdev: fix race condition between spdk_bdev_close and _remove_notify
When new bdev was created, the struct spdk_bdev_module::examine_disk()
may open and close bdev. On the other hand, if something goes wrong,
the creation procedure may unregister new created bdev, so race
condition appeared between _remove_notify() and spdk_bdev_close().

Add the new field "closed" and "remove_notified" in struct spdk_bdev_desc,
so _remove_notify() and spdk_bdev_close() knows how to deal with this
situation.

Change-Id: Ibfe915a4d76096796b039a13a4f49f26669eba2c
Signed-off-by: wuzhouhui <wuzhouhui@kingsoft.com>
Reviewed-on: https://review.gerrithub.io/423369
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-09-06 20:59:05 +00:00
Ben Walker
898739fbac bdev: Enforce that spdk_bdev_close() is called on same thread as open()
spdk_bdev_close() must be called on the same thread as
spdk_bdev_open(). Further, the remove callback on the
descriptor will also be run on the same thread as
spdk_bdev_open().

Change-Id: I949d6dd67de1e63d39f06944d473e4aa7134111b
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/424738
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: GangCao <gang.cao@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-09-06 20:59:05 +00:00
Jim Harris
73c8b61cac bdev: free bdev_io in write_zeroes emulation
When emulating write_zeroes commands on device that
don't natively support it, we submit a write with
a zeroed buffer.  We used to just reuse the original
bdev_io, but that was recently changed due to other
splitting code added for iovs.  But when making those
changes, we forgot to free the bdev_io for the
write that was sent down to the device.

Fixes: 183f37e8 (bdev: do not reuse bdev_io when...)

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: If08782c65f6305c0a9f9d15d74fd8823e1158e9b

Reviewed-on: https://review.gerrithub.io/424733
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-09-06 20:42:47 +00:00
wuzhouhui
e133dcd850 bdev/gpt: vbdev_gpt_examine: no need to read gpt if bdev is too small
Change-Id: I28f21649feae8022cd0f0afd5b01ae3ee2b3592e
Signed-off-by: wuzhouhui <wuzhouhui@kingsoft.com>
Reviewed-on: https://review.gerrithub.io/424618
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-09-06 16:53:36 +00:00
wuzhouhui
823a9960b9 bdev/lvol: _create_lvol_disk(): unregister disk if something wrong
Change-Id: Ib2202ed920c76d6bf72e59eb7475582a0e2490fd
Signed-off-by: wuzhouhui <wuzhouhui@kingsoft.com>
Reviewed-on: https://review.gerrithub.io/422037
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-09-05 21:57:55 +00:00
Jim Harris
ba38785b7e bdev: remove get_bdevs_config RPC
This RPC does not work for a lot of bdev types.  For
example, NVMe namespaces and virtio scsi LUNs are not
explicitly constructed by an RPC - they are indirectly
constructed by an RPC associated with an NVMe controller
or virtio-scsi controller.

While here, remove spdk_bdev_config_json.  It was
only created to facilitate this get_bdevs_config RPC.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I537166d8f91ab458bd2000859d74f7254bfc9c0a

Reviewed-on: https://review.gerrithub.io/424584
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-09-05 19:46:17 +00:00
Jim Harris
30dc6a1893 bdev: don't output "name" when write_config_json not specified
This isn't valid RPC so it needs to be removed.  Bdev modules were
working around this issue by defining empty write_config_json
methods.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I3c4c20249eadfcfb4103430f5801190b14897249

Reviewed-on: https://review.gerrithub.io/424582
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-09-05 19:46:17 +00:00
Li Feng
3f6a54c1a7 bdev/rbd: fix memory leak and spdk_io_device_register fail
io_device is registered, but never release.
It will lead spdk_io_device_register to fail
because allocate new io_device may reuse the same memory address.

Change-Id: I3359ed783c1df43ef382669d9d73d06bfbc1fe2d
Signed-off-by: Li Feng <lifeng1519@gmail.com>
Reviewed-on: https://review.gerrithub.io/424235
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: GangCao <gang.cao@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-09-05 19:26:24 +00:00
Jim Harris
82c3c30f44 trace: remove alias concept
This was added a long time back for tracking an rte_mbuf
whose buffer was a different rte_mbuf - all related to
a userspace TCP stack that is no longer in development.
The concept isn't useful now, so remove it to reduce
the complexity of the tracing code.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I310e492eba7f55df242bb29d82fb19f6daee1f51

Reviewed-on: https://review.gerrithub.io/424565
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-09-05 18:03:43 +00:00
Ben Walker
c94020001a thread: Add a name parameter to spdk_register_io_device
This is a string name used for debugging only.

Change-Id: I9827f0e6c83be7bc13951c7b5f0951ce6c2a1ece
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/424127
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-09-05 16:00:54 +00:00
wuzhouhui
1062181443 bdev/lvol: fix error path of spdk_rpc_get_lvol_stores()
Free memory if spdk_jsonrpc_begin_result() failed.

Change-Id: Iad4656f455b5fff1c55adf1d4728517c6e5656b6
Signed-off-by: wuzhouhui <wuzhouhui@kingsoft.com>
Reviewed-on: https://review.gerrithub.io/424392
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-09-04 23:33:29 +00:00
Jim Harris
afaabcce23 bdev: add trace points
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I37e7f2fb19fecfe5933b4815d24240954b74b62b

Reviewed-on: https://review.gerrithub.io/424278
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-09-04 17:09:25 +00:00