Commit Graph

9344 Commits

Author SHA1 Message Date
Ziye Yang
ba3adc4367 iscsi: return the correct error code in spdk_iscsi_read_pdu
We shoud not always return SPDK_ISCSI_CONNECTION_FATAL
in spdk_iscsi_read_pdu function.

Reason: In iscsi_conn_handle_incoming_pdus, the loop
directly return only rc==SPDK_ISCSI_CONNECTION_FATAL.
But it masks all the necessary information. So we would like
to keep some information of the return value for spdk_iscsi_read_pdu,
and we can use error log to track those information.

Then we can return SPDK_ISCSI_CONNECTION_FATAL as the error
return value for iscsi_conn_handle_incoming_pdus function.

Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I4c40fcb27052b55cb92e06273701a881def18e12
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457078
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-06-13 06:04:31 +00:00
yidong0635
858718c31d target/filesystem: Add partprobe after parted
Partprobe is a program that informs the operating system kernel
of partition table changes, by requesting that the operating system
re-read the partition table. For instance, if you create a new
partition on one of your disks using parted, you should run
partprobe afterwards to make the kernel aware of the new partition
configuration.

This is useful for issue #799.And it's compatible with other systems.

Change-Id: Icd4c85193bd9d9e6c2b32b8463e75c7a6ff06f34
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457735
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-06-13 00:49:52 +00:00
Shuhei Matsumoto
c758dc088a nvmf: Reject bdev with separate metadata to attach to subsystem
NVMe bdev module support separate metadata now but NVMf subsystem
cannot process bdev with separate metadata yet.

Hence reject any bdev with separate metadata to be attached
explicitly by this patch.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I793c6c5f61deb766d7bf427ff67ccc57a48974cf
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457167
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-06-13 00:48:11 +00:00
Changpeng Liu
a47e7de332 bdev: rename _spdk_bdev_io_split to spdk_bdev_io_split
_spdk_bdev_io_split() will be used to requests that have IO buffers.

Change-Id: I5ba0b83e47b84066e6c5032de2e02404155896d3
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457549
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-06-13 00:44:34 +00:00
Darek Stojaczyk
59c8eeddf6 test/vhost: remove virtio-pmd tests
Those tests are absolutely awful. For the last few days
they've been randomly failing way more often than usual
and we have no way of figuring out what the root cause is.
We know it's 'discover_bdevs' that hangs, but since it
doesn't produce any output (all redirected to null), we
can't get any relevant information out of it. The only
way to move forward is to rewrite this entire test from
scratch.

Change-Id: Ie874110008f82f3ca0dd62f733b0dac73160fbb8
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457766
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-06-12 10:17:58 +00:00
Darek Stojaczyk
4cc2d1d814 test/iscsi: don't use set +e
We used `set +e` before bdev deletion RPCs and
`wait $fio_pid`. Disabling errors from those RPCs
doesn't make sense. We expect FIO to fail due to
bdev hotremoval, but the bdev hotremoval itself
must succeed.

Besides, it's easy to forget to undo `set +e` so
don't use it. For `wait` we can use the following
paradigm instead:

```
rc=0
wait $pid || rc=$?
```

Change-Id: I8716bd861eae85d7a8590d4bd26a65cfa10e0dc1
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457469
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-06-12 05:29:25 +00:00
Darek Stojaczyk
1a8ee925b0 rte_vhost: fix compilation against dpdk master
struct ether_addr was renamed to struct rte_ether_addr
in latest DPDK master, but our internal fork of rte_vhost
still used the old name, which can be now a non-defined type.

Together with the struct, the RTE_ETHER_ADDR_LEN define
was renamed as well, so we'll now check if it's defined and
we'll manually define struct ether_addr to keep the old
rte_vhost working.

Change-Id: I78b8104ed3bfe03397881a94f0f8bee14f9efae8
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457609
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-06-12 05:29:02 +00:00
Changpeng Liu
3ec061800f nvmf: add a persist through power loss configuration file when constructing NS
For reservation feature in NVMoF, we can't support the persist through
power loss feature, now we will add the configuration file parameter
with Namespace, after users set the configuration file parameter with
one NS, then the PTPL feature can be enabled.

Change-Id: Id72699093f7e68318b9529f7bacc5c9804f7f86b
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455905
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-06-12 00:30:03 +00:00
Shuhei Matsumoto
37bdd0e87f scsi: Add data offset to DIF context separately from start block address
This patch uses the change by the last patch to initialize DIF
context in SCSI layer. Besides this patch changes the name of a
parameter from offset to data_offset to clarify the meaning.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I54bf1168ec5959432aa15dae0360c0640138b033
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457541
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-06-11 18:53:58 +00:00
Shuhei Matsumoto
73204fe2e5 dif: Add data offset to DIF context separately from start block address
Data offset are intended to correspond to DATAO in NVMe/TCP and
Buffer Offset in iSCSI.

Previously for iSCSI, buffer offset had been merged to start block
address, but passing buffer offset separately from start block address
clarifies the logic more.

On the other hand, for NVMe/TCP, passing DATAO separately from start
block address will be critically important because DATAO will bave any
alignment and will be necessary to use for not only reference tag
but also guard computation.

This patch adds data_offset to struct spdk_dif_ctx and adds it to the
parameters of spdk_dif_ctx_init(). ref_tag_offset is also added to struct
spdk_dif_ctx and it is computed by dividing data_offset by data_block_size
and is used to compute reference tag.

The next patch will use this change when getting DIF context in SCSI.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Id0e12ca9b1dc75d0589787520feb0c2ee0f844a5
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457540
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-06-11 18:53:58 +00:00
Shuhei Matsumoto
d9ec66dfb6 dif: Use variables instead of numerics to test DIF insert/strip in UT
DIF UT code had used numerics extensibly for complex DIF insert/strip
UTs. But numerics was error prone and difficult to follow the logic.
Hence this patch changes numerics to variables in DIF insert/strip UTs.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I40cb27167c068855dc5d1eb9e0f36818d39c73fa
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457539
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-06-11 18:53:58 +00:00
Shuhei Matsumoto
3e52bdf722 dif: Fix wrong data_offset of spdk_dif_generate_stream in UT
data_offset to spdk_dif_generate_stream() was wrong but the function
worked as expected because the wrong data_offset was in the same
data block as the correct data_offset.

But the subsequent patches will refine spdk_dif_generate_stream()
and this bug will cause UT failure. Hence fix the bug up front.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I45f2302deaf377fda0ec5454c61fb463231a39db
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457538
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-06-11 18:53:58 +00:00
Darek Stojaczyk
e83d5c4a2d test/ocf: disable persistent metadata test
It's randomly times out on waitforlisten. See #815

Change-Id: Ib9a75d0a96756810cb25f2a1ddf72bc800ea4b5c
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457483
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-06-11 18:16:55 +00:00
paul luse
12f7aeb4ce lib/reduce: avoid extra operation in write path
Change-Id: I92355d3deb25f818e6d4c7a72d2f4fd45e6879ac
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457523
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-06-11 18:13:10 +00:00
Richael Zhuang
6708b70938 lib/ftl: fix the unclear PAGE_SIZE
In lib/ftl PAGE_SIZE is used in many places. But I think there's no relation with
the host memory PAGE_SIZE.
In most changes, PAGE_SIZE is replaced by FTL_BLOCK_SIZE for ftl block size.
In ftl/ftl-anm.c, PAGE_SIZE is replaced by 4096 for alignment.

Change-Id: I72e7c65c83b9fe1a4e50944dc5d90b9459b4e593
Signed-off-by: Richael Zhuang <richael.zhuang@arm.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455347
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-06-11 18:12:27 +00:00
Alexey Marchuk
53777de855 rdma: Unset IBV_SEND_SIGNALED flag for RDMA_WRITE operations
Unsetting this flag will decrease the number of WRs retrieved during CQ polling and will decrease
the oeverall processing time. Since RDMA_WRITE operations are always paired with RDMA_SEND (response),
it is possible to track the number of outstanding WRs relying on the completed response WR.
Completed WRs of type RDMA_WR_TYPE_DATA are now always RDMA_READ operations.

The patch shows %2 better peformance for read operations on x86 machine. The performance was measured using perf with the following parameters:
-q 16 -o 4096 -w read -t 300 -c 2
with nvme null device, each measurement was done 4 times

avg IOPS (with patch): 865861.71
avg IOPS (master): 847958.77

avg latency (with patch): 18.46 [us]
avg latency (master): 18.85 [us]

Change-Id: Ifd3329fbd0e45dd5f27213b36b9444308660fc8b
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Signed-off-by: Sasha Kotchubievsky <sashakot@mellanox.com>
Signed-off-by: Evgenii Kochetov <evgeniik@mellanox.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456469
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-06-11 18:07:28 +00:00
Maciej Szwed
1900c8c261 bdev: Move spdk_bdev declaration higher in the bdev.h file
It is required for upcoming patches.

Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I836a1118309cc3ef4168d834697109200cf18553

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455646
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-06-11 18:06:58 +00:00
Changpeng Liu
14f6d724ac scsi: add additional check for initiator port
Vhost-scsi target will not set task's initiator port parameter,
so reservation commands sent from Guest will cause segment fault.

Change-Id: Ifc175effded5371eca08d5bfe7e4aa977dd4b1c6
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457550
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-06-11 18:05:55 +00:00
Mateusz Kozlowski
8cdb3d330e bdev/ftl: Add driver specific data to get_bdevs rpc
Added basic configuration details (transport type and address,
parallel units, cache info) as well as most important OCSSD geometry
data to get_bdevs driver_specific section.

Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: I00c50706bd9203bcef1701be1b7d87e93c10e57f
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456790
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-06-11 06:25:36 +00:00
Wojciech Malikowski
675c4aa743 lib/ftl: Read partially lba map during relocation
Read requested range of lba map during relocation
instead whole lba map.

To read partial lba map range single move now has
three states: read, read_lba_map and write.

Free queue and write queue were merged to a single
move queue.

Change-Id: I86839f2286d42d4debf87cea40091370e5283b15
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/454747
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-06-11 06:21:46 +00:00
Wojciech Malikowski
a1f1581968 lib/ftl: Remove FTL_IO_KEEP_ALIVE flag from ftl_io_flags
FTL_IO_KEEP_ALIVE flag is deprecated.

Change-Id: Id074f45a3e9ecb6d05c6a88a834476e2c19288a3
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/454326
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-06-11 06:21:46 +00:00
Wojciech Malikowski
4769f058e0 lib/ftl: Stop using ftl_io for internal relocation logic
Added ftl_reloc_move structure to internally track
relocation requests. This patch remove unnecessary
dependency between relocation module and ftl_io.

Change-Id: I9dd1bca6039b7c8961cb22a9f85ea1c5737a0a6f
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/454006
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-06-11 06:21:46 +00:00
Wojciech Malikowski
d23cc88d73 lib/ftl: Track cached lba map segments
Keep track of read lba map segments to avoid
unnecessary traffic in case ANM events.

Lba map is divided on 4KB segments which can
store 512 lba entries.

In case multipe read request on same segment
keep pending request list and process it in
read_lba_map() completion callback.

Change-Id: I2661bdb716ab7c975140e0b37aebcb17aa23901d
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453371
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-06-11 06:21:46 +00:00
GangCao
7729225fa8 nightly: fix vhost migration issue
This is to solve below issue:

error 'Fio job '\''/migration-tc1.job'\'' does not exist'

Change-Id: I60dc6254d238878b03c5faf341b3201643286dfe
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457453
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-06-11 05:55:00 +00:00
Darek Stojaczyk
fadcb08db3 autotest/common: define xtrace_restore as an alias
This doesn't make difference right now, but would be
useful if we printed $BASH_SOURCE or $LINENO in xtrace
(via PS4). Those two will now point to the original
line where xtrace_restore was called rather than
always the single line inside autotest_common.sh.

Change-Id: Idf3ac8d00ad9610960678351014334013149b88d
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456964
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-06-11 05:17:58 +00:00
Tomasz Zawadzki
416c5e29ab bdevperf: move checking test parameters to verify_test_params()
Moving verifying test parameters to single function function.

This is start of series that will add RPC to start bdevperf tests,
allowing configuration beforehand.

In future verify_test_params() will be used to verify parameters
passed via RPC as well.

Change-Id: I0aff7d9fa9f99205b3559b0ea081aa3690013faa
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457223
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-06-11 05:13:48 +00:00
Wojciech Malikowski
ef084d199e lib/vmd: Hooking devices behind VMD endpoint to PCI subsystem
Create PCI hook for each detected PCI device
behind VMD endpoint.

Change-Id: I0bec674b859b02e5338eb407dc6c5f83cf1e8978
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Signed-off-by: Orden Smith <orden.e.smith@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456632
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-06-11 04:35:31 +00:00
Wojciech Malikowski
4a3d5418c5 lib/vmd: Replace spdk_vmd_probe(bdf) with spdk_vmd_init()
spdk_vmd_init() will attach all VMD devices that were
unbinded from system. There is not need to specify VMD
bdf in VMD public interface since it can be controlled
by setup.sh script.

Change-Id: Ifc45c32dc7e11b59429a41ddfdd596db30e27731
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456631
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-06-11 04:35:31 +00:00
Wojciech Malikowski
fa64709373 examples/vmd: VMD device usage example
This example enumarates NVMe disks behind VMD.

Change-Id: I2e38aa3b3b73db1cc3b3bc79128071800cbf5fa9
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Signed-off-by: Orden Smith <orden.e.smith@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455829
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-06-11 04:35:31 +00:00
JinYu
8fc9ac7b0e nvmf: complete all I/Os before changing sgroup to PAUSED
For the nvme device, I/Os are completed asynchronously. So we
need to check the outstanding I/Os before putting IO channel
when we hot remove the device. We should be sure that all the
I/Os have been completed when we change the sgroup->state to
PAUSED, so that we can update the subsystem.

Fix #615 #755

Change-Id: I0f727a7bd0734fa9be1193e1f574892ab3e68b55
Signed-off-by: JinYu <jin.yu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452038
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-06-11 01:51:56 +00:00
Darek Stojaczyk
17e0283db1 configure: use the external rte_vhost lib by default
It is supported with DPDK 19.05+, so `configure` will now
automatically fall back to the internal rte_vhost copy if
the external one is not available.

Change-Id: I83746154f07a907d361a7511478112082710174f
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456190
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-06-11 01:14:45 +00:00
Darek Stojaczyk
4fcec18d38 vhost: don't interrupt uninitialized virtqueues
rte_vhost_vring_call() from upstream DPDK can read some
unitialized memory and crash if it's called on invalid
queue ids. The implementation in our internal rte_vhost
fork ends up wiritng to a random descriptor number, which
doesn't cause any crashes but is a bug nevertheless.

To fix it, just check if the queue is initialized before
interrupting it during the session start. It's not a hot
I/O path and there's no performance impact.

Change-Id: I830c1be98ef00d4ece9a6bd88cf79b9dfe29d2a9
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457247
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-06-11 01:14:45 +00:00
Darek Stojaczyk
9c8937cfd8 test/json_config: disable vhost-nvme tests
We're about to make SPDK vhost work against the upstream
rte_vhost from DPDK 19.05+ and vhost-nvme tests currently
stay in the way. vhost-nvme is only supported in our
modified rte_vhost fork and while it could be implemented
entirely in SPDK against the latest upstream rte_vhost
APIs, there are just no plans to do that now.

For that reason we disable all vhost-nvme tests.

Change-Id: I2d43bbe75dcab7535e7b1254ca75cb78b6d446fa
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456960
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-06-11 01:14:45 +00:00
Chunyang Hui
dd26583316 Opal: Add opal_create_key function
Change-Id: Id1705636e25fe3ad90ff60a57aca7b1e4c2ef687
Signed-off-by: Chunyang Hui <chunyang.hui@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453972
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2019-06-11 01:12:24 +00:00
Chunyang Hui
9f988238fc Opal: Refactor and clean functions
Delete opal_next, introduce opal_add_tokens.
Delete spdk_opal_cmd, seperate cmds to new APIs.

Change-Id: Ide56817eec7fde7b110818966ebf10e65a952fc9
Signed-off-by: Chunyang Hui <chunyang.hui@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/454433
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-06-11 01:12:24 +00:00
yidong0635
4913ed4dc5 bdev: add uring in bdev Makefile
We had "./configure --with-uring" , but actually, it wasn't compiled.
we need add uring to Makefile that code will add this.
then we can continue to verify this module.

Change-Id: I8b98825f6795eb9f9e7b4947d1c7c3a44a6f0f64
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457081
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-06-11 00:26:48 +00:00
Shuhei Matsumoto
3cfb1fe8dd iscsi: Schedule the connection only after completing login processing
Previously the connection was scheduled to another core just
after setting conn->full_feature to 1, but conn->state was still
ISCSI_CONN_STATE_INVALID. The connection started to run on another
core but could not run normally until conn->state became
ISCSI_CONN_STATE_RUNNING. conn->state was changed late to
ISCSI_CONN_STATE_RUNNING after sending the first login reponse.

This gap window had caused intermittent critical failures.

Based on this analysis, this patch changes to call
spdk_iscsi_conn_schedule() just after sending the final login response.
Whether any login response is final or not can be known by checking
conn->full_feature is not zero.

    Fixes #785

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I8ea55fef27e2f332fcd789d32daf479a24c0588d
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457414
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-06-10 09:43:09 +00:00
Konrad Sztyber
fc3e1b6fb0 bdevperf: target bdev selection
Allow the user to select target bdev to be used by bdevperf. It's useful
when the config contains multiple bdevs, but only one is supposed to be
utilized by bdevperf directly.

Change-Id: I51aa645dcf60a4413057d86f68fe24442b280367
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456787
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-06-10 04:15:10 +00:00
Konrad Sztyber
5012fcc148 bdevperf: update target list in bdevperf_construct_target
The target list is now updated inside bdevperf_construct_target. That
way, if the target is successfully created, it'll already be on the
target list once this function finishes.

Change-Id: I364c45005c28deadae37615b6ac85f3860fc5080
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457089
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-06-10 04:15:10 +00:00
Karol Latecki
535654f826 scripts/rpc: add try-except around call_rpc_func
Missing try-except block around call_rpc_func().
If called function returned error there was an uncaught exception
which resulted in backtrace printed to screen.

Change-Id: Ifb37c29d70b93ef648a0503643a51cc1967ef8d0
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456956
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-06-08 11:16:07 +00:00
wanghonghui
77509767c0 bdev/uring: adjust to the latest io_uring API
Upstream liburing rename the io_uring_get_completion to
io_uring_peek_ceq

Signed-off-by: Honghui Wang <wanghonghui@ucloud.cn>
Change-Id: I8f207b7b1255b44304f4eca4101b939847767179
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456959
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-06-07 18:43:07 +00:00
Changpeng Liu
a8c32ed5fe blob: return error for write_zeroes and unmap requests
Actually write/writev/write_zeroes/unmap are never be called, and we add
the error code here to keep it same style with snapshot bs_bdev.

Change-Id: I32ad051c1902bd7080b894e36f7c89f1c8d27434
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456924
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-06-07 18:37:53 +00:00
yidong0635
fdb03395db nvmf_vhost: Check qemu before and in vhost/common.sh.
Normal return for qemu checking , this should check before vhost/common.sh.
For qemu not installed on this machine. It may be a VM. Skipping nvmf_vhost test.
Otherwise, code will exit 1 before this normal return.
And in vhost/common.sh should with qemu checking.

Change-Id: I2285fffa6fc4384b4b172e5c443019e9ab391fa0
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456123
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-06-07 18:35:12 +00:00
JinYu
2ec93bc316 unit/nvme: add wait for completion unit test
Add the completion and completion_timeout unit test.
Increase the code coverage of nvme.c

Change-Id: I89ff744955c83dc64a4678633a00c1bb166414d6
Signed-off-by: JinYu <jin.yu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455233
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-06-07 18:31:47 +00:00
Liang Yan
985169456b doc/vpp: update vpp document and configuration
Update the configure and doc for VPP.

Change-Id: I922c02de108abd4bd4ac0c853b49080e449539cd
Signed-off-by: Liang Yan <liang.z.yan@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453758
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2019-06-07 18:27:53 +00:00
Mateusz Kozlowski
42fe2e9b24 lib/ftl: Add config to construct_ftl rpc for dirty shutdown restore
Gives the ability to change behavior of restore after dirty shutdown
without recompiling ftl library. User can define if partial recovery
or error should be returned after such a scenario.

Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: I6dda40df7b92d6a377957e4a70a3eab91a6ac4a9
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456185
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2019-06-07 18:26:57 +00:00
Mateusz Kozlowski
a9ce7b8551 lib/ftl: Add ability to ignore endmeta errors on restore
If a band wasn't closed during shutdown (ie. after dirty shutdown),
the start md contains valid data, however end md does not (or it may
not be written at all).
The config gives the user the ability to specify if encountering
this case should result in an error, or if ftl should pad the band
with data and recover from closed bands only.

Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: If02f7ca8bc90bb61698fb710fee2274af6af01e4
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455513
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-06-07 18:26:57 +00:00
Mateusz Kozlowski
9939830608 lib/ftl: Add padding when open band is found on restore
If an open band is found during recovery (after dirty shutdown)
ftl will attempt to pad it, instead of returning a recovery error.
Any previously written data on that band is lost, as the line is
treated as being in free state after the pad.

Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: Ie48fa9fa37a3853f41921b9417c69301cf47e673
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455512
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-06-07 18:26:57 +00:00
Mateusz Kozlowski
4214803c16 lib/ftl: Add public get chunk info admin command
The patch also changes, so the command calculates the needed
chunk offset based on PPA, instead of relying on the user to
provide one.

Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: Ic4eec1b86ded4eb71de860015403294ed0c8c266
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455973
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2019-06-07 18:26:57 +00:00
Mateusz Kozlowski
baa06190d8 lib/ftl: Add direct wptr and IO
Direct wptr allows for writing to band with externally
provided PPA, rather than one assigned by the wptr. This
can be useful when padding open bands during recovery or
bdev creation, as ftl can find partially written end meta
with no way to close the band with valid data, that would
conform to an already written CRC value.

This does mean that I/O sender is responsible for calculating
and obeying per-chunk write pointer arithmetics.

Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: Ifcbc4e8ffb69c829469a571bed8e27b148d7a651
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455511
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-06-07 18:26:57 +00:00