Commit Graph

2629 Commits

Author SHA1 Message Date
Karol Latecki
f6f434b4b4 test/vhost: Update vhost test plan
Also move the test plan file to a more appropiate location.

Change-Id: I462058279d4d2dc03790657b82b3cfeb00c8a93a
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/394180
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-01-25 13:59:47 -05:00
Jim Harris
33e88d790f env: bump max virtual address map from 128tb to 256tb
At least on some ARM platforms, virtual address map
uses lower 48 bits of address space, compared to just
47 on x86-64.

This does double the size of the top level map from
1MB to 2MB.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I18f95a10b4913dca0256cbbd8d05f3baa5ab69fa

Reviewed-on: https://review.gerrithub.io/396318
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-01-25 11:05:59 -05:00
Daniel Verkamp
23fd31f9cc bdev: convert RPC calls to use SPDK_COUNTOF()
These two RPCs were missed in the initial COUNTOF conversion.

Change-Id: Ia40eb0cf3b5c859e2b63148e442fd7d52d1032a5
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/396231
Reviewed-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-01-25 10:48:09 -05:00
Shuhei Matsumoto
8f9028deb3 iscsi/rpc: Fix completeness of dump of iSCSI global params
To use JSON dump of iSCSI global parameters as configuration file,
at bootup, it must have completeness.

This patch fixes the following:
- DefaultTime2Wait was not contained.
- MinConnectionsPerCore was not contained.
- Converting only 0 to "None" for dump is not efficient.

Change-Id: I740e6938c216c5c62df6ee70b0ceac40fd0cac00
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/396100
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2018-01-24 18:32:37 -05:00
Changpeng Liu
53c24913f2 vhost_lib: enable virtio configuration space feature
New added vhost user messages: GET_CONFIG/SET_CONFIG are
used for get/set virtio device's configuration space, this
commit enable the new added vhost messages.

Change-Id: I5c3e3f8fb6ed55e99299323c39658765b1724bb8
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/386545
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-01-24 16:48:48 -05:00
Daniel Verkamp
88da45bfd0 scsi: ensure bdlen is initialized before use
This fixes a false positive warning when building on GCC 7.2.1 with
CONFIG_COVERAGE=y.

bdlen is always initialized on the path where it can be used, but the
compiler seems to get confused when coverage is enabled, so zero out the
value at the top of the function.

Change-Id: Ifc13abff80124cad3d26286ffebf84f967141d13
Reported-by: John Meneghini <johnm@netapp.com>
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/396244
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-01-24 16:45:49 -05:00
Piotr Pelplinski
074f4d7627 blob: set md_ro and data_ro fields only after persisting data
Currently, there is no possibility to save read only blob to disk.
This patch modifies behaviour so that read only flags are applied after syncing blob.
This is analogy to resize, set xattr and remove xattr operations.

Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Change-Id: Iffed601c78cb83231bb20e7ef05b73847dc3c95a
Reviewed-on: https://review.gerrithub.io/394243
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2018-01-24 15:56:15 -05:00
Ben Walker
be774bf664 nvmf: Check subsystem state when modifying listeners or hosts
Change-Id: Idadd4eb7574b005cce8b888b28c8c3b6993343a4
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/395555
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-01-24 15:52:48 -05:00
Tomasz Zawadzki
b6aaba0852 bdev: remove vbdevs during spdk_bdev_unregister()
spdk_vbdev_unregister() is part of internal bdev API,
yet bdev module that uses spdk_vbdev_register() directly
will not be removed correctly when using delete_bdev RPC.
spdk_vbdev_unregister() is now consolidated with
spdk_bdev_unregister().

This comes up when deleting lvol bdev, as it does not use
spdk_bdev_part_* functions.
base_bdev->vbdevs entry was not removed for bdev that lvs
is created on.

Additionally patch expands test to create lvol bdev,
after removing it using delete_bdev RPC.
With ASAN enabled this would report accessing
already freed memory previously.

Change-Id: I9547e83862e2daa50355d56a1c9f453aaa6cfdb8
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/395711
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-01-24 15:50:04 -05:00
Jim Harris
22bdcdc39b iscsi: reuse spdk_iscsi_conn_stop_poller
Similar functionality is needed when migrating a connection
to its new core after login.  So make spdk_iscsi_conn_stop_poller
reusable for that case, and call it in the login migration path.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ie014a2e50d6ed02165317deab85c943225953197

Reviewed-on: https://review.gerrithub.io/395857
Reviewed-by: <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-01-24 15:39:05 -05:00
Jim Harris
20528e262d iscsi: simplify spdk_iscsi_conn_stop_poller()
All calls to this function now use spdk_env_get_current_core()
as lcore, so remove this parameter.

All calls to this function also use _spdk_iscsi_conn_free() as
the stop_fn, so remove that parameter as well - just call that
function directly from the callsites and eliminate the extra
event.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I639e63d26fbfcc054137d9b453be2927e2c1f625

Reviewed-on: https://review.gerrithub.io/395854
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-01-24 15:39:05 -05:00
Jim Harris
5aa90453cb iscsi: move spdk_iscsi_conn_get_migrate_event inline
This function is now only called from one place, and moving
this inline will simplify some future patches which will
consolidate the login and full-feature pollers.

No functional change.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I7d4e6973882c7b579e341283648317e8e06d998a

Reviewed-on: https://review.gerrithub.io/395853
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-01-24 15:39:05 -05:00
Jim Harris
f570aa654a vhost: only split on 2MB boundaries when necessary
vhost I/O only need to be split on 2MB boundaries if
there is a break in the VM's memtable at that 2MB
boundary.

This should drastically reduce (if not eliminate)
the intermittent test pool failures seen recently.
virtio limits number of segments to 128, but this
2MB splitting could introduce additional segment
breaks which we do not allocate IOVs for.  In almost
all cases, there are no memtable breaks except at
low 2MB, so most of the extra segment breaks we are
adding are unnecessary.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I12d85c289ad80c7bb65e3d2030a2405092b19deb

Reviewed-on: https://review.gerrithub.io/396058
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
2018-01-24 10:33:43 -05:00
Maciej Szwed
68b8237cdd blob: assign iovcnt value in spdk_bs_user_op_alloc function
iovcnt value was set to 0 instead of being assigned from
input argument.

Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I0f7e89871357f5db9a5a34c801176bc5c7870021

Reviewed-on: https://review.gerrithub.io/395959
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-01-23 23:41:57 -05:00
Dariusz Stojaczyk
fb12bbecd2 virtio: move vdev->name allocation to generic virtio
Previously vdev->name was being allocated/freed
separately in virtio_pci and virtio_user backends.
Now it's all done in generic virtio library and
cleans up some code.

Change-Id: I810e976d09781c0c9b25c6f7fd957a83aad6c7b8
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/394704
Reviewed-by: <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-01-23 22:49:21 -05:00
Dariusz Stojaczyk
e6da08c28a virtio/pci: detach pci device on virtio-pci destroy
Fix memory leak on virtio-pci device destruct.

Change-Id: I7d33802bc15c15d6a2dbac3e83456c9cf23b6004
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/394703
Reviewed-by: <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-01-23 22:49:21 -05:00
Dariusz Stojaczyk
0d6a37c7d9 bdev/virtio/rpc: add RPC to attach virtio-pci device
`rpc.py construct_virtio_pci_scsi_bdev <bdf> <name>`

The RPC internals reuse the struct definition and callback
functions from virtio-user equivalents

Change-Id: I0c6c49d8481565a49ec9460a633696d27d55367a
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/394447
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-01-23 22:49:21 -05:00
Ben Walker
2bbc59fa79 nvmf: Fix bug when accessing realloc'd pointer
Caught by the latest scan-build.

Change-Id: I606a30447e5221c86e79accb9a2c1fe76e9eeee0
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/396054
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-01-23 17:20:05 -05:00
Jim Harris
4d48d87a7f iscsi: remove idle connection handling
Some upcoming changes will effectively render this moot
anyways by adding an epoll/kqueue descriptor to poll
on in all cases (not just connections that have been
idle for 5ms).

The epoll/kqueue code was just ifdef'd out instead
of removed - some of this code will be useful
and reusable with minimal changes in the upcoming
patches.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I0c354390537e6369cb3c32e78a59c300dec6d098

Reviewed-on: https://review.gerrithub.io/395553
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: <shuhei.matsumoto.xt@hitachi.com>
2018-01-22 23:22:17 -05:00
Jim Harris
8c2b8ca7d7 iscsi: put spdk_iscsi_conn_write_pdu() in lib/iscsi/conn.c
This function was originally named spdk_iscsi_write_pdu()
in lib/iscsi/iscsi.c.  Since this is an operation on a
connection, add "conn" to the name and move it to conn.c.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iaad022907d43788108d5b2660306abcf5e94040d

Reviewed-on: https://review.gerrithub.io/395522
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-01-22 23:22:17 -05:00
Jim Harris
936eb59d8d iscsi: use return code to determine if all PDUs were flushed
This will enable some future enhancements where the main
iSCSI connection poller will not try to flush PDUs, we will
just start a separate poller to periodically flush PDUs in that
case.

This is all part of broader scheme to enable epoll() for
iSCSI.  Making these changes allows us to avoid using epoll()
to signal when TCP buffer space is available.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I6fe9314e386673fcc87bb15580ef808838c55a7b

Reviewed-on: https://review.gerrithub.io/395521
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: <shuhei.matsumoto.xt@hitachi.com>
2018-01-22 23:22:17 -05:00
Jim Harris
902f8719f2 vhost: bump SPDK_VHOST_IOVS_MAX to 129
Currently we test up to 512KB I/O sizes, and this I/O
size could fragment to 129 IOVs if split on 4KB
boundaries if the buffer start is not 4KB aligned.

This is likely the cause of a recent slew of failures
in the test pool which coincide with updating to
fio 3.3.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I189157af578e75e025ca8e3420712739e604fca7

Reviewed-on: https://review.gerrithub.io/395872
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-01-22 23:19:43 -05:00
Jim Harris
b2503cb335 blob: add spdk_bs_user_op_t
This allows a channel's request_set resources to be
used for queuing I/O requests.  This is needed
for upcoming thin provisioning functionality,
where we must queue I/O requests that need to
allocate a cluster, if another cluster allocation
is in progress.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ie8d3e799afc0b56bc95ba5ecab11253d8bc8608f
Reviewed-on: https://review.gerrithub.io/395037
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-01-22 16:00:09 -05:00
Jim Harris
4547f9bdf8 blob: split readv/writev as separate blob calls
Now all operations (both single buffer and iov-based
payloads) which span a cluster boundary get split into
separate blob calls for each cluster.

This will simplify upcoming patches that need to do
special operations if a cluster needs to be allocated.
This code can now be added to just the single cluster
operations and not have to worry about splits.  It
will also simplify the code that will eventually queue
requests which require a cluster allocation if an
allocation is already in progress.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I22a850e6e23e3b3be31183b28d01d57c163b25b1

Reviewed-on: https://review.gerrithub.io/395035
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2018-01-22 16:00:09 -05:00
Jim Harris
7ba04e589a blob: move batch creation into single/split non-iov functions
A future patch will queue an operation in
_spdk_blob_request_submit_op_single if the cluster is not allocated
and another allocation is already in progress.  If the queueing fails
because of no channel resources, we want to fail the callback and
need to do this before creating the batch.

This causes a bit of code duplication, but in the end should make the
code easier to read.

While here, pass spdk_blob instead of spdk_blob_data to
_spdk_blob_request_submit_op_single.  This simplifies a future patch
which will need the spdk_blob when queueing an operation.  It also
incidentally makes it consistent with _spdk_blob_request_submit_op_split.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ib0ec0e5138eac5bf208fcde6676cd2a77a1a663f

Reviewed-on: https://review.gerrithub.io/395196
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-01-22 16:00:09 -05:00
Dariusz Stojaczyk
568cf46c5d bdev/virtio/rpc: cleanup construct_virtio_user_bdev code
Make it non vhost-user specific. The same
code will be later reused by upcoming
construct_virtio_pci_bdev RPC call.

Change-Id: I6964fedea2a1d52c9b06d5fd8cafe745e06df0ab
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/394504
Reviewed-by: <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-01-22 15:57:11 -05:00
Dariusz Stojaczyk
225ef46bc0 bdev/virtio: allow specifying a name for virtio-pci devices
The default name remains as sprintf("VirtioScsi%d", counter++).

Change-Id: I244c1389f8dfac16f61aaf1d610a7888dd55e56c
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/394446
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-01-22 15:57:11 -05:00
Dariusz Stojaczyk
37fb62224a bdev/virtio: add API to attach particular PCI devices
This is just the API introduction. See subsequent
patches for its RPC usage.

Change-Id: Iadb7c9bf6a56ab4330c9f2215c6006a2935d208d
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/394445
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-01-22 15:57:11 -05:00
Shuhei Matsumoto
632f0ced75 iscsi/rpc: Set processor affinity of connection through JSON-RPC
Currently setting cpumask to portal is possible only through
iSCSI.conf. This patch makes possible for any user to set cpumask
through JSON-RPC too.

The following are done in this patch:
- To keep compatibility, cpumask parameter handled as optional.
- Python test code is added.
- Current python script for JSON-RPC does not work correctly for
  IPv6 and the issue is fixed.

Change-Id: I42ef397ce95040a36db4430417a35e9e97527477
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/391728
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-01-22 15:54:48 -05:00
Jim Harris
7bae25dfcd iscsi: remove flush timeout
This concept was added very early in the SPDK iSCSI target
development process when there was a high focus on maximizing
throughput on a single iSCSI connection with 512-byte reads.

Realistically, in multi-connection environments focused on
predominantly 4KB (or more) workloads, this concept loses its
effectiveness - it is relatively rare that PDUs from multiple
I/O would coalesce within the default 8us flush timeout period.

There were no users of flush timeout in the SPDK tree - it was
not even documented in the iSCSI configuration file example and
there was no way to modify it at run time.  So leave the change
out of the CHANGELOG.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I08c3e959fb3945dc2c9cb89248305d0c88aa778f

Reviewed-on: https://review.gerrithub.io/395520
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-01-22 13:37:31 -05:00
Shuhei Matsumoto
f587197d21 iscsi&iscsi/ut: Unify cpumask decode between iSCSI.conf and JSON-RPC
Currently cpumask cannot be specified for each portal when it is
created by JSON-RPC and portal group creation is not unified
between iSCSI.conf and JSON-RPC.

This patch does the following:
 - cpumask string is decoded in spdk_iscsi_portal_create() which
   is common between iSCSI.conf and JSON-RPC.
 - parsing configline of portal is difficult to understand and
   hence it is refactored.
 - UT code is added.

 JSON-RPC will be added by the next patch.

Change-Id: I13b9989263fae5facff260de32a55ec99f5d5a06
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/392447
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-01-19 15:47:57 -05:00
Jim Harris
f536eb583c blob: turn non-iov splitting into additional blob calls
For I/O that do not span a cluster boundary, just issue
a single batch command to underlying block device.

For I/O that do span a cluster boundary, issue a batch
command for each against the blob (not the block device)
for each cluster accessed by the I/O.

This is all in preparation for upcoming patches which
enable thin provisioning and hence cluster allocation
in the I/O path.  It will simplify implementation of
the cluster allocation path since now that code only
needs to be concerned with a single allocation at once.

Splitting for readv/writev will be handled in a
later patch.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ia2341abbda599dace3357c4eec06ab6602ef81a8

Reviewed-on: https://review.gerrithub.io/395027
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-01-18 16:39:50 -05:00
Jim Harris
96d11441be blob: add _spdk_blob_request_submit_op_impl()
This breaks out the logic for building a batch for
non-iov operations to a separate function.  Future
patches will do further modifications on this
new function - separating it out into two separate
functions, one for operations that span a cluster boundary
and one for those that do not.

No functional change here - this is just moving code
around to reduce size of upcoming patches.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Id1e67e6305d7ba2317700e1477e12c749ebf664c

Reviewed-on: https://review.gerrithub.io/395026
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-01-18 16:39:50 -05:00
Jim Harris
a5ecbadb2c blob: add spdk_bs_batch_xxx_blob variants
These will be used to implement user request splitting
on cluster boundaries.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I29e00ae9555fdd8a149e92be3cf88a2e528f5c0e

Reviewed-on: https://review.gerrithub.io/395021
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-01-18 16:39:50 -05:00
Jim Harris
c46591393b blob: add _dev prefix to sequence/batch operations
This clarifies that the read/write/etc. operation is
being performaned on the block device.  This
clarification will be important in some upcoming
patches which will batch similar operations on a
blob (as part of splitting a user request into smaller
operations if it spans a cluster boundary).

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ic574804c15f769ee80c1b6f68ca3b77ec910f0f6

Reviewed-on: https://review.gerrithub.io/395017
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-01-18 16:39:50 -05:00
Jim Harris
a2a29abcfb blob: remove sequence/batch operations for flush
These were unused in blobstore so remove them for now.
This reduces number of code changes in some upcoming
patches.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I02e04f7d84fff1d2a70f371d222425f373a5d4ff

Reviewed-on: https://review.gerrithub.io/395016
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-01-18 16:39:50 -05:00
GangCao
6d5c2d2bd1 io_channel: add the check for the existence of poller_fn
There existing some cases like in our UT code, the
start_poller_fn and stop_poller_fn is not configured.
Add a check here and properly handle these cases.

Change-Id: Iaac83d78547432584f6de0c54e9d2e06b9b35741
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.gerrithub.io/392996
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-01-18 13:15:17 -05:00
Dariusz Stojaczyk
583a24a489 bdev: share nomem_io data between bdevs built on the same device
When there are two bdevs built on the same io_device,
it is possible that one bdev entirely saturates
underlying queue, not letting the second bdev issue
a single I/O. The second bdev will silently fail any
subsequent I/O and append it to the nomem_io list.
However, since we resend I/O only from I/O completion
callback and there's no outstanding I/O for that bdev
(io_outstanding==0), the I/O will never be resent.
It'll be stuck in nomem_io forever.

This patch makes nomem_io list to be shared between
bdevs built on the same device. It is now possible
that I/O completion callback from one bdev will retry
sending I/O from other bdev.

The shared bdev data is based on thread-local
bdev_mgmt_channel, so doesn't need any external
synchronization.

Change-Id: Ia5ac3a1627ce3de4087e43907c329aa7d07ed7c7
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/394658
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ziye Yang <optimistyzy@gmail.com>
2018-01-18 00:29:46 -05:00
Ziye Yang
25f26f38b8 lvol: restore the channel operation number into 512
We would still like to keep up this Macro, then if users
want to change this value with their own ones,
they can change it.

Change-Id: Ic13c9ba0889955cc41a75f700c58822728782f6c
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/394311
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2018-01-18 00:29:19 -05:00
Ben Walker
6d4d942b2a nvmf: Remove master_thread concept
Individual threads can now pause and resume subsystems
as needed, so the master_thread isn't used.

Change-Id: I49ccb1f87d3e5956db7bd8fc9da0542c5c487923
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/394260
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-01-17 17:28:13 -05:00
Ben Walker
7358fb6f85 nvmf: Hot plug notifications now use the subsystem pause functionality
Use the general purpose mechanism to pause a subsystem when a bdev
is hot removed.

Change-Id: I3cd4fb15140d6682707b7840c6a9c3f63fe48e55
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/394259
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-01-17 17:28:13 -05:00
Ben Walker
d346b9c5a2 nvmf: State change functions now have a return code
When the state change is known to fail immediately, use
a return code instead of calling the callback.

Most of the callbacks didn't actually check for errors,
so this patch also doesn't check for errors on return
codes. That should be done in the future.

Change-Id: I67e03f93d7f53892473dfc073f4150e7e620cad2
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/394281
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-01-17 17:28:13 -05:00
Ben Walker
6711e5a525 nvmf: Fail incoming commands to an inactive subsystem
Eventually this should probably queue, but for now
just fail these commands.

Change-Id: Ie1b1088aa38198c17062ac9874539ad6d99b01eb
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/394125
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-01-17 17:28:13 -05:00
Ben Walker
823b565b5f nvmf: No longer route fabrics/admin commands to a single thread
These commands can now pause a subsystem if they need to operate
on it. We don't currently implement any of the NVMe commands
that would need to pause, so this patch is simpler than most
would expect.

Change-Id: I25bfdf8e7577cda2bb0ce248d2889447032b9b4c
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/394121
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-01-17 17:28:13 -05:00
Shuhei Matsumoto
afc97ddcf7 iscsi/rpc: Configure Header/Data digest through JSON-RPC
This patch makes possible to set and get Header Digest and Data
Digest of target.

Change-Id: I1d1b892f9dfb747c0f5ad8fc4fddef40929b4143
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/394482
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2018-01-17 12:41:29 -05:00
Shuhei Matsumoto
de70d712fb iscsi: Check CHAP params when a target is created by JSON-RPC
When a target is created by iSCSI.conf, only valid CHAP params
are passed to spdk_iscsi_tgt_node_construct().

When a target is created by JSON-RPC, help information encourages
users to specify valid CHAP params but
spdk_iscsi_tgt_node_construct() does not check CHAP params and
users can create targets whose CHAP params are invalid.

Change-Id: I7e9057a982f21f04782481cda74208a139c1fdad
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/394481
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-01-17 12:41:29 -05:00
Shuhei Matsumoto
f04569140c rpc/iscsi: Adding an LUN to an existing target
Add JSON-RPC to add an LUN to an existing target. The parameter
lun_id is optinal and if skipped, the lowest free LUN ID will be
assigned.

This JSON-RPC is tested in iscsi_tgt/rpc_config.

Change-Id: I010619f2d4ccec89c589bb0618466b4d15949ebb
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/385181
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-01-17 12:41:29 -05:00
Shuhei Matsumoto
3b3c6002c9 iscsi: Add LUN to an existing target (not runtime)
Removing an LUN from an existing iSCSI target is possible by
removing the corresponding BDEV. However adding an LUN to an
existing iSCSI target is not possible yet.

Add a new function spdk_iscsi_tgt_node_add_lun() and related
functions first toward supporting this function.

JSON-RPC for this operation will be submitted an another patch.

Informing the newly added LUN to the initiator is not included
in this patch. Hence this operation is possible only for any
inactive target.

Change-Id: I3a28f4d75a17126e49c9d12ce64c3ad68f231840
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/385180
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-01-17 12:41:29 -05:00
Maciej Szwed
0e91725631 blob: add spdk_bs_batch/sequence_read/readv_bs_dev functions
This prepares for some future blobstore.c code that will use these
bs_dev functions directly making possible to read from different device
than the one in the current context.

Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I55b15544ea4b53475c4b72ee5c92ff1c0a2b8c88
Reviewed-on: https://review.gerrithub.io/393781
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-01-17 07:58:57 -05:00
Maciej Szwed
bf2d02b6ef blob: add _spdk_bs_allocate_cluster function
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I35779dc547e0c084086ec6d9bf44f86850cb7f05
Reviewed-on: https://review.gerrithub.io/393780
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-01-17 07:58:57 -05:00