Commit Graph

4270 Commits

Author SHA1 Message Date
Konrad Sztyber
22c1a00fa8 fio_plugin: use pollers' expiration for timedwait
Check next poller's expiration time to calculate the timeout for
pthread_cond_timedwait.

Change-Id: I96f81bab917c1bc628d94bba3c58d25066d4209a
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/437309
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-12-23 00:58:20 +00:00
Darek Stojaczyk
edaea9164d vhost: move vid to session struct
Each connection is created with the `new_connection`
rte_vhost callback with a unique vid parameter. Storing
the vid inside the device struct was sufficient until
we wanted to have multiple connections per device.

Change-Id: Ic730d3377e1410499bdc163ce961863c530b880d
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/437775
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-12-23 00:42:54 +00:00
Darek Stojaczyk
73844cccf4 vhost: introduce device sessions
Grouped a few spdk_vhost_dev struct fields into a new
struct spdk_vhost_session. A session will represent the
connection between SPDK vhost device (vhost-user slave)
and QEMU (vhost-user master).

This essentially serves two purposes. The first is to
allow multiple simultaneous connections to a single
vhost device. Each connection (session) will have access
to the same storage, but will use separate virtqueues,
separate features and possibly different memory. For
Vhost-SCSI, this could be used together with the upcoming
SCSI reservations feature.

The other purpose is to untie devices from lcores and tie
sessions instead. This will potentially allow us to modify
the device struct from any thread, meaning we'll be able
to get rid of the external events API and simplify a lot
of the code that manages vhost - vhost RPC for instance.
Device backends themselves would be responsible for
propagating all device events to each session, but it could
be completely transparent to the upper layers.

Change-Id: I39984cc0a3ae2e76e0817d48fdaa5f43d3339607
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/437774
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-12-23 00:42:54 +00:00
Ziye Yang
58f1624497 nvmf: add the transport shared buffer num configuration option.
Previously, we allocate the buffer size according
to the MaxQueueDepth info, however this is not exactly
a good way for customers to configure, we should provided
a shared buffer number configuration for the transport.

Change-Id: Ic6ff83076a65e77ec7376688ffb3737fd899057c
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/437450
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-12-20 19:55:57 +00:00
Shuhei Matsumoto
3947bc2492 util/crc16: Add spdk_crc16_t10dif_copy to use in read strip and write insert
For DIF and DIX, read strip and write insert operation will copy
data together with DIF generation and verification.

This patch adds spdk_crc16_t10dif_copy for those cases.

Change-Id: I9a77fa8b367486fe0b6704d58dcdf95d5210c875
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/437461
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-12-20 17:52:29 +00:00
Shuhei Matsumoto
f0e00ef287 dif: Inject bit-flip error for extended LBA payload
This patch adds APIs to inject bit flip error into any field
of the extended LBA payload.

Change-Id: I3ca601999e55ea6228bb525ac8c0744c7df32398
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/434292
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-12-20 17:52:29 +00:00
Shuhei Matsumoto
0a3cdcf3c3 dif: Factor out core logic to generate and verify DIF for each split block
This patch factors out the core logic to generate and verify DIF
for each split extended logical block.

This patch reduces nesting and clarify the core logic.

Change-Id: I6adf36fb86fafef8a4235021f77a1d99a0d63c8a
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/437795
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-12-20 17:52:29 +00:00
Shuhei Matsumoto
f9cbc493dc dif: Use simple helper functions to iterate iovec array
Introducing helper functions will simplify a little complex
implementation.

Change-Id: I94911497b8aecc0ffbd148519ce22501ef69c79d
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/435387
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-12-20 17:52:29 +00:00
Shuhei Matsumoto
8b6c22b1a0 dif: Generate DIF for extended LBA payload
DIF and DIX will be supported in SPDK throughout, e.g., NVMe
driver, NVMe-oF initiator and target, NVMe block device, malloc
block device, SCSI, iSCSI target, FIO plugin, and Perf.

Generic and common APIs to generate and verify DIF and inject
bit flip error to any field will be helpful for them.

This patch is the first in the patch series.

This patch adds APIs to generate and verify DIF for SGL extended
LBA payload as byte alignement and granularity.

Change-Id: Ie6588d960113761f10efbf2d2a3cae004af37ce8
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/432261
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-12-20 17:52:29 +00:00
Seth Howell
088379cf74 nvme_rdma: Add source addr info to RPC
Change-Id: Id6b5aee4b36d828a0f9e5ddc85293a15342d2aae
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/436220
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-12-20 17:31:43 +00:00
Seth Howell
b1ecb314b7 bdev_nvme: use spdk_nvme_connect in spdk_bdev_nvme_create
This makes it possible to configure the host address information over
RPC.

Change-Id: Icf0de200fbb8cb869c1408f12564e832d58fe00d
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/437571
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-12-20 17:31:43 +00:00
Seth Howell
675c5592e7 nvme_bdev: add parsing for hostaddr and hostsvcid
This allows us to specify the host-side configuration for each
controller to which we connect.

Change-Id: Iac2aed3934d4a326f45546f2f541e374308e2589
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/436219
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-12-20 17:31:43 +00:00
Ziye Yang
c9c4a3346d vpp: Change the type of sa into sockaddr_storage
This will fix the compilation issue when enabling VPP.

Change-Id: Id06c832ed45369448d127ec148e43fe3f9e1ce69
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/436909
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-12-20 01:49:29 +00:00
Ziye Yang
ef8e47571b net: make the net initialization in a correct way
Change-Id: If795ae841127a0742b366ac5df019e5a2d6e6b65
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/436901
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-12-20 01:37:50 +00:00
Seth Howell
672115fef4 nvme.c: break out parsing from trid parse
This code snippet will be generally useful for parsing information when
we add support for host address configuration.

Change-Id: Ic90f485de5a5db699901da029c9a29be4db477c7
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/437739
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-12-19 22:45:13 +00:00
GangCao
70a3488657 bdev/qos: enable and disable when the QoS thread is not set
In the case the QoS thread has not properly initialized yet,
needs to go through the regular QoS enabling process to notify
all the channels and also disable the QoS properly. The channel
and poller related staff also needs to be handled together
with the thread.

Change-Id: Ifc2b2cdfb1181aa6418ad1d43ae5905c0c317549
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.gerrithub.io/437519
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-12-19 18:17:02 +00:00
Wojciech Malikowski
d1a6901c61 event: reactor loop delay configurable from cmdline
Change-Id: I38ab7b60d528fa9e0161ddba48c723601d8a6b78
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/437142
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-12-19 17:22:33 +00:00
wuzhouhui
8db5ff2bdd reduce: _allocate_vol_requests: set pointer to NULL after freed
If _allocate_vol_requests() failed, the caller will call
_init_load_cleanup() to free memory that _allocate_vol_requests()
already freed, and cause segment fault. Setting pointer to NULL keeps
_init_load_cleanup() is safe to free it again.

Change-Id: I9ad09eabf6b65350f174bd5a4faf5ee643cbd23f
Signed-off-by: wuzhouhui <wuzhouhui@kingsoft.com>
Reviewed-on: https://review.gerrithub.io/437292
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-12-18 23:38:51 +00:00
wuzhouhui
3a4185be19 reduce: check strlen(SPDK_REDUCE_SIGNATURE) in buildtime
Change-Id: I4d12f2a693a4a34c840d972a4f60cbd0fc35011b
Signed-off-by: wuzhouhui <wuzhouhui@kingsoft.com>
Reviewed-on: https://review.gerrithub.io/437236
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-12-18 23:34:17 +00:00
wuzhouhui
7fbc5106e4 reduce: put pmem_unmap() in _init_load_cleanup()
In effect, this commit fixes error path of spdk_reduce_vol_init()
and spdk_reduce_vol_load(): forget to unmap pmem file after mapped.

Change-Id: I797f15e315fff3ff42c17509a73dc46e7f6bdb24
Signed-off-by: wuzhouhui <wuzhouhui@kingsoft.com>
Reviewed-on: https://review.gerrithub.io/437270
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-12-18 23:33:50 +00:00
Ben Walker
7b94053827 thread: Rename spdk_thread_msg to spdk_msg_fn
This is the callback type used for message passing. The
old name is easy to confuse with the callback to pass
a message to a thread or the upcoming callback to
spawn a thread.

Change-Id: I5fd63b57c4be2a4262a197850e6de4901be03ee7
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/435941
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-12-18 23:31:08 +00:00
Ben Walker
978725c020 thread: Minimize scope of internal locking
Now that the thread is stored in a thread local
variable, the scope of the locking can be greatly
reduced.

Change-Id: Ic44934fff352be0fa5d54746226ef707aceae7b5
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/435940
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-12-18 23:31:08 +00:00
Shuhei Matsumoto
e303567bc1 util/crc16: Add init_crc parameter as seed value to spdk_crc16_t10dif
Add init_crc parameter as seed value to spdk_crc16_t10dif API to generate
a CRC value spanning multiple separate buffers.

This will be necessary for upcoming DIF/DIX patches.

Having init_crc parameter is general, and so change the existing API
without adding seed version of the existing API.

Change-Id: I0ac7919b18013967e41829dcedd3e4e73204d5d6
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/437204
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-12-18 17:28:56 +00:00
Konrad Sztyber
62db4ac2cf util: added spdk_divide_round_up()
Replaced divide_round_up() from blobstore.c, lvol.c and reduce.c with
new spdk_divide_round_up() from util.h.

Change-Id: I013383ac286ca52b5c15c7fab4fb40ad97b92656
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/437649
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-12-18 17:26:49 +00:00
Ziye Yang
94cd652b18 nvmf/tcp: Add a poller to check the timeout of each qpair
This makes the timeout check for each qpair in the group
efficient. If there are many qpairs in the group, we
can scale.

Change-Id: I75c29a92107dc32377a2ef7edb5ac92868f1c5df
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.gerrithub.io/435277
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-12-18 08:34:30 +00:00
Ziye Yang
9d11abfd0e nvmf: Do not set the error state of the qpair
Reason: I checked the code in different transport,
the qpair is already freed, so we dot need to set
any state.

Change-Id: I3d78c259c3f79ea4426dc9408e5c3469bc171358
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/437493
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-12-18 04:00:59 +00:00
Ziye Yang
04d09f9207 nvmf/tcp: Use generic transport options structure
Remove the unnessary fields in spdk_nvmf_tcp_transport

Change-Id: I632608ba654b30f3511f5e1d925c6743c9100365
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/437271
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-12-17 04:25:25 +00:00
Liu Xiaodong
ae0aae1532 lib/trace: add 3 RPC method for tpoint_group_mask
Now trace facility can be enabled/disabled at runtime,
besides app starting parameters

Change-Id: I086564665ba4375b946a5339d5921d6584e9bfda
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/435470
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-12-14 21:29:12 +00:00
Ben Walker
605e530a4e thread: Use TLS to accelerate thread look up
Change-Id: I8136db265c9cb8d61de4845ce6eaff2351b5b597
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/435939
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-12-14 18:54:03 +00:00
Ben Walker
70642bcbe8 rocksdb: Use new threading API calls
Change-Id: I6ed98605512b675e1b6bde5b9644d6160802b893
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/436757
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-12-14 18:54:03 +00:00
Evgeniy Kochetov
d722a1742d nvmf: Improve error handling in spdk_nvmf_transport_poll_group_create
At least in case of RDMA transport, poll_group_create (spdk_nvmf_rdma_poll_group_create)
 can return error (NULL).

Change-Id: If1576b3515e7f9ede76af08bfa6b1c8399dcda09
Signed-off-by: Sasha Kotchubievsky <sashakot@mellanox.com>
Signed-off-by: Evgeniy Kochetov <evgeniik@mellanox.com>
Reviewed-on: https://review.gerrithub.io/436887
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-12-14 16:15:48 +00:00
Evgeniy Kochetov
7da9f8faba nvmf/rdma: Fix refcnt check on RDMA QP destroy
Check for QP reference counter in RDMA QP destroy function was wrong
and QP resources were never released.

Change-Id: I6ab0ce39452e8263f89589d138c90f749516ebb1
Signed-off-by: Evgeniy Kochetov <evgeniik@mellanox.com>
Signed-off-by: Sasha Kotchubievsky <sashakot@mellanox.com>
Reviewed-on: https://review.gerrithub.io/436974
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-12-14 16:15:12 +00:00
Wojciech Malikowski
212eb89376 util: added bit array bitmask load, store and clear
Change-Id: I7cd47184d0dfb038297c0ac04d7dfb5d31f96b6b
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/432537
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
2018-12-14 15:34:53 +00:00
Seth Howell
7245134ab0 memory: return first translation from mem_map_translate
This should have always been the case with spdk_mem_map_translate. For
some memory maps (like RDMA) this doesn't matter, but for others like
our virtual to physical map, this is critical for retrieving valid
translations.

This behavior change will only affect maps that have a registered
contiguous memory callback.

Change-Id: I67517667f01d974702d7daa7c81238281aae0cf6
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/436562
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-12-13 22:27:21 +00:00
Maciej Szwed
54f6083a81 blob: Remove snpashot from the list only on blob deletion
When last clone of a snapshot is being deleted
we remove that snapshot from snapshots list.
We should not do that as it still works as a
snapshot and it is read-only, but it does not list
as a snpashot from get_bdevs. Instead remove snapshot
entry from the list when blob that represents that
snapshot is being removed.

Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I8d76229567fb0d9f15d29bad3fd94b9813249604

Reviewed-on: https://review.gerrithub.io/436971
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-12-13 22:26:56 +00:00
Ziye Yang
ea8aa1bf0a nvmf: check the qpair->ctrlr
The ctrlr may be NULL, so we need to add a check here
to present segment fault.

Change-Id: I6c5361cc829af065082a95df0b8cc2f8d49a6002
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.gerrithub.io/436950
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-12-13 21:52:45 +00:00
Jim Harris
3d1c1f945c reduce: don't try to clear io units for empty chunk maps
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Reported-by: Shuhei Matsumoto
Change-Id: Idcf427e32962236acfeb7dab793e27c44cdc6151

Reviewed-on: https://review.gerrithub.io/436878
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-12-13 14:55:39 +00:00
Jim Harris
0b0af8f30b reduce: calculate vol size from backing dev size
This more closely matches the sequence of events
a user will take in creating a compressed volume.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I2311aed49e2a623feb9c5fbdbd25fb3201aeeeb5

Reviewed-on: https://review.gerrithub.io/436065
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-12-13 14:55:39 +00:00
Jim Harris
8c2cf577b3 reduce: remove extra size check
This was the result of a previous merge conflict.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ic657e7f9e16b6a073669b078b4323f7abfef95b9

Reviewed-on: https://review.gerrithub.io/436063
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-12-13 14:55:39 +00:00
Jim Harris
1c68d71107 reduce: change some member names to make them more descriptive
Based on feedback from Paul Luse in some earlier reviews.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I1930a3de05ada15d3ac4530c081c2f951b96fe97

Reviewed-on: https://review.gerrithub.io/435368
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-12-13 14:55:39 +00:00
Jim Harris
383b117309 reduce: implement read and write with data
For write operations, copy data to req->buf and write
to disk.

If chunk already specified in logical map, read the
chunk first into req->buf, and overwrite with data
specified by the write operation.

If chunk not specified in logical map, fill logical
blocks not specified by the write operation with
zeroes.

For read operations, read chunk into req->buf first,
then copy relevant data into the buffers specified
by the read operations.

These operations are all functional, but have room
for future improvement.  For example, this patch
will issue a separate backing read/write operations
for each backing block in the chunk - this could be
optimized to coalesce operations where the backing
blocks are contiguous.

While here, clean up freeing bufspace in one of
the error paths - this needs to be freed using
spdk_dma_free instead.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I6dbf4fc9a8fdf0f5424b1f1f9178c79891c96d0d

Reviewed-on: https://review.gerrithub.io/434116
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-12-13 14:55:39 +00:00
Jim Harris
84f8b6339f reduce: start implementation of spdk_reduce_vol_writev
Focus of this patch is adding the foundations of manipulating
the chunk_map and backing_block bit arrays and persisting
chunk maps and the logical map to the pm_file.

No data is writting the backing device yet.  That will come
in later patches.  This also does not rebuild the bit arrays
from the pm_file during spdk_reduce_vol_load - that will also come
in a later patch.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ib5336dbe907c253e545704471de8c4e812bbc157

Reviewed-on: https://review.gerrithub.io/434115
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-12-13 14:55:39 +00:00
Jim Harris
3213962e91 reduce: replace ziperrno with reduce_errno
ziperrno was left over from earlier versions of this
patch set that used "sbzip" instead of "reduce".

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I4159141b92727bb0a24db5f9aefcccaadc360d9e

Reviewed-on: https://review.gerrithub.io/436399
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-12-13 14:55:39 +00:00
wuzhouhui
fdaea1fa5a lvol: lock g_lvol_stores_mutex when access g_lvol_stores
Since we lock g_lvol_stores_mutex when add lvs to g_lvol_stores, so
logically speaking, we shoud lock g_lvol_stores_mutex whenever access
g_lvol_stores_mutex. Although I can't figure out specific scenario that
will cause contention if we don't lock it.

Change-Id: If3bdba91407f5c9d09fc16c5ec7dcb919ff9647d
Signed-off-by: wuzhouhui <wuzhouhui@kingsoft.com>
Reviewed-on: https://review.gerrithub.io/426147
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-12-13 08:10:51 +00:00
Ziye Yang
527c825c81 nvmf: Re-add spdk_nvmf_transport_poll_group_remove
For TCP/IP transport, we need to remove the socket
from the polling group since we do not want to keep the
tgroup info in the NVMe/TCP qpair, it should be general.

Change-Id: I4b064d8378f66ea5d91ac554fe628d9ccebd07f4
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/434128
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-12-13 02:41:14 +00:00
Ziye Yang
5f03a9c1f3 nvmf/tcp: remove the unnecessary check.
Since we already make the recv state handling in a correct
way, so we do not need this check any more.

Change-Id: Id71ab2e0ef60be302f8cf6ea776259d7312663ec
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/436896
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-12-12 20:45:32 +00:00
Ziye Yang
b7b2c972c9 nvme/tcp: solve the adminq is not created issue.
If the adminq is not created, we should call
nvme_transport_ctrlr_destruct (i.e., nvme_tcp_ctrlr_destruct
in tcp transport).

Change-Id: If2362c47ac89fd80289ddd8402cf9f74d7bb5801
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/436613
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-12-12 20:44:01 +00:00
Jim Harris
20c69bbc30 env: remove default pre-reserved memory size
Much of this was originally applied by Darek in
6f5840d22 but then had to be reverted due to NVMe-oF
MR issues in a817ccf5.  Now that the NVMe-oF MR issues
have been resolved, we can reapply these changes.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Change-Id: I58d603523901c78dbc81279078a09b2a24c091e6

Reviewed-on: https://review.gerrithub.io/436547
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-12-12 11:00:57 +00:00
Lance Hartmann
e865a52415 nvme: Eliminate identify errors to Discovery ctrlr
The nvme/identify cmd issued some cmds to a ctrlr irrespective
of its type, and when the target was a Discovery ctrlr which only
accepts a very limited cmd set, that would result in errors observable
both on the initiator side (from nvme/identify) and in the output on
the target (nvmf_tgt).  Introduce new API, spdk_nvme_ctrlr_is_discovery(),
and alter identify to make use of that in determining which commands
to send to the target.

Change-Id: I974a569843f1d2b9e1ece7bd3bf9ceee1bfae872
Signed-off-by: Lance Hartmann <lance.hartmann@oracle.com>
Reviewed-on: https://review.gerrithub.io/436225
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-12-11 17:39:52 +00:00
Shuhei Matsumoto
37af0cc930 iscsi: Propagate SCSI error status to split SCSI read/write commands
For split SCSI read command, if there is any failure in the sequence of
it, the first error must be propagated to all subtasks of it.

For split SCSI write command, if there is any failure in the sequence
of it, the first error must be propagated to the primary subtask.

Before this patch,
for read task:
- any failure is propagated to already completed subtasks, but
  is not propagated to any subtasks not completed yet, and
- if any failure occurs in non-primary subtasks, it is not propagated
  to the primary subtask.

for write task:
- if the primary subtask completes after any failure of non-primary
  subtasks, the failure will be overwritten by the success of the
  primary task.

This patch fixes these issues.

Change-Id: I2d878798cbb40a8c5bd6a6fe5efb32b8de4a8ecd
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/436673
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-12-11 16:37:55 +00:00