Commit Graph

328 Commits

Author SHA1 Message Date
Changpeng Liu
90520634bd bdev/nvme: configure the number of requests allocated for one NVMe I/O queue via RPC
A single I/O may allocate more than one request, since splitting may be
necessary to conform to the device's maximum transfer size, PRP list
compatibility requirements, or driver-assisted striping.  Very big
I/O request sent from application may get error due to limited resources
in NVMe driver layer, so here we add an optional parameter to make the
parameter can be configured by users.

Fix issue #745.

Change-Id: I7824232c54865b052dcd0ec6e91484c3837fc2c4
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/461182
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-07-29 22:52:39 +00:00
Wojciech Malikowski
825ae66bd0 event/subsystem/vmd: RPC support
Added enable_vmd RPC call.

Change-Id: Ib0bea12f097a46a8be537a1db8aace6cb3b9b699
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459949
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-07-26 18:27:40 +00:00
Karol Latecki
4462653272 scripts/rpc: change construct_crypto_bdevs args to positional
Arguments for this function were added as optional arguments
and were listed as such by argparse when using rpc.py --help.

This is confusing as all of these arguments are obligatory
in order to create a crypto bdev in SPDK configuration.

Change-Id: I7c31413314f28e44ee008d3d7c28759063700b61
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/460968
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-07-24 18:23:02 +00:00
Mateusz Kozlowski
cd115ecfcd bdev/ftl: Add defrag parameters to construct_ftl_bdev rpc
Added parsing and saving of (optional) parameters which are responsible for
relocation in FTL. Changing the parameters may affect Write
Amplification Factor and overall performance, especially during random
write workloads. If parameters are not specified, default values will be used.

Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: Ic0229e39109460f3541c31e1cbf2a485efe408c6
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/460504
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-24 18:21:07 +00:00
Ziye Yang
6ad6a1131b nvmf/tcp: Add a feature to allow set the sock priority of the connection.
This priority is used to differentiate the sock priority on the TCP connections
between  NVMe-oF TCP target and other TCP based applications.

Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I6ee294e647420b56d1d91a07c2e37bf34ce24e03
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/461801
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-07-19 06:30:19 +00:00
Ziye Yang
4739f62365 nvmf: enable transport based scheduler decode in RPC.
Purpose: To eanble the transport based scheduler in RPC.
Previously, we only support it with configuration file.

Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I02ae9b1b316d4fec8b28b550e70dcdc78ce78722
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/461645
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-17 15:01:20 +00:00
Seth Howell
e2c0d9a294 rpc: fix nvmf_create_transport c2h_success option.
This option was intended to enable the user to disable the c2h_option,
but because of the way arguments filter down through the python script,
it was actually impossible to disable the optimization.
My understanding is that typically we will want to keep this
optimization enabled on most systems, but the option should work like it
says it does.

Also, align the implementation of this function with the other ones on
the RPC i.e. use the store_true, store_false paradigm.

Change-Id: I59f0e9a573abf1d567e5539294c63c68899b35f1
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/461737
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-16 01:25:54 +00:00
Evgeniy Kochetov
fca6ff8f75 rpc: Add nvmf_get_stats RPC method
This patch adds nvmf_get_stats RPC method and basic infrastructure to
report NVMf global and per poll group statistics in JSON format.

Signed-off-by: Evgeniy Kochetov <evgeniik@mellanox.com>
Change-Id: I13b83e28b75a02bc1dcb7b95cbce52ae10ff0f7b
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452298
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-07-12 12:46:29 +00:00
Evgeniy Kochetov
7535cdbd62 rpc: Add thread_get_stats RPC method
SPDK threads collect busy and idle time statistics. This commit adds
thread_get_stats RPC method to retrieve these values.

Signed-off-by: Evgeniy Kochetov <evgeniik@mellanox.com>
Change-Id: I8ed8041c6164eb0c0a9336f4e50b5f26a3f20190
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/445285
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2019-07-10 04:28:14 +00:00
Changpeng Liu
5317a9f795 rpc/nvmf: add RPC support to add the persistent configuration file for one NS
Change-Id: Ic4963d3e55cffceca35d18ba8d406658e51a189a
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455913
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-07-10 01:40:26 +00:00
Shuhei Matsumoto
390cffb64e rpc: Add dif_insert_or_strip parameter to nvmf_create_transport RPC
Add an new optional parameter dif_insert_or_strip to
nvmf_create_transport RPC.

.INI config file will be deprecated and dif_insert_or_strip is not
supported in .INI config file.

Change-Id: Ibf38b599cff75eeb0056dd2125d6ec10d444f339
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458927
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-07-10 00:43:02 +00:00
Darek Stojaczyk
c47d6541e0 scripts/rpc.py: rename mock_ functions
There will be one more "mocked" implementation soon,
so make existing names more descriptive now.

Change-Id: I30daf2b60ab683d5befb06eca50d455712858d61
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459390
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-07-04 07:16:52 +00:00
Darek Stojaczyk
31c25ab3ae scripts/rpc.py: rename print_string to print_json
The previous name wasn't very accurate. The function
accepts a json object, so call it print_json().

Change-Id: If1cae587fd13f0a8a2761049125fc7fb234c0360
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459388
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-07-04 07:16:52 +00:00
paul luse
bfda995be2 bdev/compress: add RPC to specify PMD
By default QAT will be selected if available however a new RPC
can be used to either auto-select (default) or specify either
ISAL or QAT.

Change-Id: I37cf7640bbd8cef455583e1eccb8adb59cc419d8
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456693
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-06-26 07:57:09 +00:00
Or Gerlitz
6629202cbd nvmf/tcp: Use the success optimization by default
By now (5.1 is released), the Linux kernel initiator supports the
success optimization and further, the version that doesn't support
it (5.0) was EOL-ed. As such, lets open it up @ spdk by default.

Doing so provides a notable performance improvement: running perf with
iodepth of 64, randread, two threads and block size of 512 bytes for 60s
("-q 64 -w randread -o 512 -c 0x5000 -t 60") over the VMA socket acceleration
library and null backing store, we got 730K IOPS with the success
optimization vs 550K without it.

IOPS           MiB/s    Average       min      max
549274.10     268.20     232.99      93.23 3256354.96
728117.57     355.53     175.76      85.93   14632.16

To allow for interop with older kernel initiators, we added
a config knob under which the success optimization can be
enabled or disabled.

Change-Id: Ia4c79f607f82c3563523ae3e07a67eac95b56dbb
Signed-off-by: Or Gerlitz <ogerlitz@mellanox.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457644
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-06-26 06:24:03 +00:00
paul luse
8d2455caa3 bdev/compress: always use QAT if available
The next patch in this series introduces and RPC that makes the
selection more flexible.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I55617d3c37b51cf9474c358cb92f5218397c0c58
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457157
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-06-21 07:53:39 +00:00
Pawel Kaminski
6e4ea8fdb4 rpc.py: add --dry-run option to print the json request and exit
Change-Id: If20c10cb74e3825fc29b9d45524353f77b832e4f
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457145
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-06-19 03:50:30 +00:00
Karol Latecki
535654f826 scripts/rpc: add try-except around call_rpc_func
Missing try-except block around call_rpc_func().
If called function returned error there was an uncaught exception
which resulted in backtrace printed to screen.

Change-Id: Ifb37c29d70b93ef648a0503643a51cc1967ef8d0
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456956
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-06-08 11:16:07 +00:00
Mateusz Kozlowski
42fe2e9b24 lib/ftl: Add config to construct_ftl rpc for dirty shutdown restore
Gives the ability to change behavior of restore after dirty shutdown
without recompiling ftl library. User can define if partial recovery
or error should be returned after such a scenario.

Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: I6dda40df7b92d6a377957e4a70a3eab91a6ac4a9
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456185
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2019-06-07 18:26:57 +00:00
Vitaliy Mysak
1e0e0ea1fc ocf: RPC add WriteBack to allowed cache modes
Make construct_ocf_bdev call accept 'wb'
  as allowed cache mode.
Previusly user could only create OCF bdevs
  in WriteThrough or PassThrough modes.

Change-Id: Ic2e1c8d2905cb51fc13c080aa7b7a4dfd7d9387f
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455628
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2019-06-07 18:18:42 +00:00
Pawel Kaminski
3bf82af866 script/rpc: Display request when rpc throws exception.
This change will allow easier debugging of
what parameters are sent to spdk target.
It will also help to analyse which commands will fail
if multiple commands are sent to rpc.py.

Change-Id: I8e75b5edce791a1fc41e83664add8893f4c1bbfb
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455636
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-06-04 03:36:26 +00:00
Seth Howell
3fb4a66d7d bdev: Create a new delay vbdev module
This module simply sits on another virtual bdev
and adds a simulated average and p99 latency to that drive.

Change-Id: Ie9fc91e27585fd0636cb7dc845cb41744bf24625
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453594
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-05-31 14:37:26 +00:00
Jim Harris
6ee44c694e rpc: rename RPC get_rpc_methods to rpc_get_methods
Make the old name a deprecated alias.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ibbf50676e0d989b67121e465fc140f94faec46ed

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453033
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-05-27 12:52:53 +00:00
Jim Harris
caec89289f scripts/rpc.py: allow empty lines when executing script
This is a bit more flexible in case script accidentally
has an extra newline at the end of the rpc.py input.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I1ebf08681e5900a4817712c7b62e4d5c21d43e54

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453956
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-05-14 19:07:31 +00:00
Jim Harris
e2d7d02d65 log: use new framework for marking deprecated RPCs
We marked the old set/clear/get_trace_flags as
deprecated in v19.01.  These are now called
set/clear/get_log_flags (to reduce confusion with the
tracing framework).  Use the new methods in this
patch series to mark these RPCs as deprecated.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I5ce8992ce8f6c1de5d5596b5f94a1587555e8546

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453563
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-05-09 04:37:08 +00:00
Jim Harris
c2b5a3f6b4 rpc.py: add framework for detecting deprecated aliases
This will get used in upcoming patches that actually
create RPC aliases.

Thanks to Ben Walker for help on the deprecated_alias
helper.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ica4432888b1ecb9bd234690758f67ef82794c50a

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453035
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-05-09 04:37:08 +00:00
Karol Latecki
22c3befd3d scripts/rpc.py: fix pipe for whitespace separated list arguments
Ordinary python str.split() will by default split on each whitespace.
This includes arguments for rpc.py which expect a whitespace separated list,
eg. construct_raid_bdev -b "Bdev_1 Bdev_2".
shlex.split() prevents splitting on quoted arguments.

Change-Id: If37eb87cf82a161263f2b247baff5c8a77c43efc
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453652
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-05-08 14:43:59 +00:00
Jim Harris
ac0824b2d8 rpc.py: detect stdin pipe
If user calls rpc.py with no parameters, it currently
hangs, waiting for input.  It should print help in this
case instead.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I5939999c57b8db0916267935d79ccc1a3201748b

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453431
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-05-07 15:48:22 +00:00
Seth Howell
350e429a57 rdma: add a flag for disabling srq.
There are cases where srq can be a detriment. Add a flag to allow users
to disable srq even if they have a piece of hardware that supports it.

Change-Id: Ia3be8e8c8e8463964e6ff1c02b07afbf4c3cc8f7
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452271
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-05-06 18:11:13 +00:00
Ben Walker
af8affd1a9 iscsi: Deprecate MinConnectionsPerCore
iSCSI is currently being adapted to SPDK's new threading model,
and this parameter doesn't make sense anymore. Remove it for
now until something functionally equivalent is added later.

Change-Id: Ia0e2f5aa81b72d99467c5a900619fbeeb42b2069
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452779
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-05-06 17:10:48 +00:00
Jim Harris
363fe506bc scripts/rpc.py: allow users to pipe multiple requests
Users can now pipe a large number of requests to
rpc.py, separated by newlines - each line will be
executed as if it was passed to rpc.py individually.

There is significant savings using this feature when
executing multiple requests through rpc.py.  On my
system, a loop of 30 RPCs related to setting up
10 NVMf subsystems takes 5 seconds when executed
one at a time.  With this new feature, it takes
less than 1 second.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iec957ca67461af8e8c41aee47e1d113714b22d3d

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452770
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-05-02 08:53:56 +00:00
Jim Harris
6895f0f27e scripts/rpc.py: add call_rpc_func()
This makes no functional change to the script, but
it prepares for an upcoming change allowing callers
to pipe a newline delimited list of rpc calls.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I695ca61510f9edd336713e88d134e42c1d78a1b4

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452769
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-05-02 08:53:56 +00:00
paul luse
600341b6c5 bdev/compress: replace RPC parm name with pm_path
We decided to name compress bdevs with COMP_ and then the name of
the underlying bdev so the RPC parm name was stale.  Previously
hardcoded the path to PM for early dev so replaced name with pm_path.

Note that a sample create looks like this now:

rpc.py construct_compress_bdev -b NVMe0n1 -p ~/pm_files -d compress_isal

And that devs need to pay attention for the pathname they provide, its
easy to leave orphaned pm files around and they can get big.

Change-Id: Ifb5245c922461bbecec4bef266bdeb25e8b87f31
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452235
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-04-26 20:07:19 +00:00
Gregory Shapiro
14032a984c NVMF: Add model number as parameter to construct_nvmf_subsystem (-d option).
Change-Id: Ia1a458a0ac1c5a17d2955a3f31c6dfe77538eb17
Signed-off-by: Gregory Shapiro <gregory.shapiro@kaminario.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/438562
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-04-23 16:51:16 +00:00
Pawel Wodkowski
2cccea55ed rpc/notifications: add notification support to RPC client and SPDK host.
Two new RPC calls get_notification_types and get_notifications added.

Change-Id: Ia5288d83cc5e9f18ef1622d3f15b9fe3bbf640ed
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/436530
Reviewed-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-04-09 23:01:52 +00:00
Xiaodong Liu
efdd6edba2 nbd: optional nbd_device in start_nbd_disk
Enhance RPC method start_nbd_disk to take nbd_device as
one optional parameter. If it is not assigned, automaticly
choose an available nbd device path from /dev/nbd0 to
/dev/nbdN.
For github issue #324:
https://github.com/spdk/spdk/issues/324

Change-Id: I72c064d8bd476df342f5aa0af4d6120eb021c7ed
Signed-off-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/440453
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-04-04 08:35:50 +00:00
Konrad Sztyber
a0cb5e9d77 lib/ftl: retrieve caching bdev from configuration
Added means to configure libftl to use (optionally) another bdev as
persistent write buffer cache.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I97028a681be168d9386eac8a226631ff772f803b
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/448629
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-03-29 04:42:48 +00:00
Paul Luse
db541f8eb3 bdev/compress: add compression vbdev module
Just the starting point, base functions and structs.

Change-Id: I2ff2b672aa675a19b89e4449f1cc4aa664007f6f
Signed-off-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/435747
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-03-27 14:51:22 +00:00
Vitaliy Mysak
bef0c6edd2 ocf: rpc: extend get_ocf_bdevs for multicore cases
Extend existing get_ocf_bdevs call to make it easier
  to get information about attached cores.

Implementation is based adding additional optional argument "name"
  to existing call. Based on "name" bdevs are filtered.
  Backward compatability of RPC interface is preserved.

This patch also adds tests for the case when name is given.

Change-Id: I4300ebe37e936bc5cca8e066b5f09db462a87cf7
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/444841
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2019-03-26 23:30:31 +00:00
Chunyang Hui
38902a5a27 rpc: add get_spdk_version rpc method
Change-Id: Iacc1766a60fb1e15a1013475f83f7f13cd45fe12
Signed-off-by: Chunyang Hui <chunyang.hui@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/448613
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-03-26 22:54:28 +00:00
Shuhei Matsumoto
c781fdee2f scripts/rpc: Change the separator of portal list parameter of add_portal_group
The python script for add_portal_group had required each portal is
an argument and portal list are made of remaining all arguments.

However this prevents adding new parameters to add_portal_group.

This patch changes portal list to be a single argument for python,
and each portal is separated by a whitespace in the argument.

Currently there will be very rare that a portal group has multiple
portals. Hence this change will not break any compatibility.

Updating CHANGELOG will be done together with other updates later.

Change-Id: I8c597a1194fff338afb371e6d24f9273122b753c
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/446379
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-03-20 07:44:54 +00:00
Ben Walker
a741e34180 bdev/nvme: Add configuration parameter to slow down polling
For NVMe devices, in conjunction with the new batching options,
it can be advantageous to artificially delay between polling
for completions. Add an option to slow this rate down.

Change-Id: I0fc92709ff45ead0beb388dda60694bf1ed8b258
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/447716
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-03-19 07:27:44 +00:00
Evgeniy Kochetov
ed0b611fc5 nvmf/rdma: Add shared receive queue support
This is a new feature for NVMEoF RDMA target, that is intended to save
resource allocation (by sharing them) and utilize the
locality (completions and memory) to get the best performance with
Shared Receive Queues (SRQs). We'll create a SRQ per core (poll
group), per device and associate each created QP/CQ with an
appropriate SRQ.

Our testing environment has 2 hosts.
Host 1:
  CPU: Intel(R) Xeon(R) CPU E5-2609 0 @ 2.40GHz dual socket (8 cores total)
  Network: ConnectX-5, ConnectX-5 VPI , 100GbE, single-port QSFP28, PCIe3.0 x16
  Disk: Intel Optane SSD 900P Series
  OS: Fedora 27 x86_64
Host 2:
  CPU: Intel(R) Xeon(R) CPU E5-2630 v2 @ 2.60GHz dual-socket (24 cores total)
  Network: ConnectX-4 VPI , 100GbE, dual-port QSFP28
  Disk: Intel Optane SSD 900P Series
  OS : CentOS 7.5.1804 x86_64
Hosts are connected via Spectrum switch.
Host 1 is running SPDK NVMeoF target.
Host 2 is used as initiator running fio with SPDK plugin.

Configuration:
- SPDK NVMeoF target: cpu mask 0x0F (4 cores), max queue depth 128,
  max SRQ depth 1024, max QPs per controller 1024
- Single NVMf subsystem with single namespace backed by physical SSD disk
- fio with SPDK plugin: randread pattern, 1-256 jobs, block size 4k,
  IO depth 16, cpu_mask 0xFFF0, IO rate 10k, rate process “poisson”

Here is a full fio command line:
fio  --name=Job --stats=1 --group_reporting=1 --idle-prof=percpu \
--loops=1 --numjobs=1 --thread=1 --time_based=1 --runtime=30s \
--ramp_time=5s --bs=4k --size=4G --iodepth=16 --readwrite=randread \
--rwmixread=75 --randrepeat=1 --ioengine=spdk --direct=1 \
--gtod_reduce=0 --cpumask=0xFFF0 --rate_iops=10k \
--rate_process=poisson \
--filename='trtype=RDMA adrfam=IPv4 traddr=1.1.79.1 trsvcid=4420 ns=1'

SPDK allocates the following entities for every work request in
receive queue (shared or not): reqs (1024 bytes), recvs (96 bytes),
cmds (64 bytes), cpls (16 bytes), in_capsule_buffer. All except the
last one are fixed size. In capsule data size is configured to 4096.
Memory consumption calculation (target):
- Multiple SRQ: core_num * ib_devs_num * SRQ_depth * (1200 +
  in_capsule_data_size)
- Multiple RQ: queue_num * RQ_depth * (1200 + in_capsule_data_size)
We ignore admin queues in calculations for simplicity.

Cases:
1. Multiple SRQ with 1024 entries:
   - Mem = 4 * 1 * 1024 * (1200 + 4096) = 20.7 MiB
     (Constant number – does not depend on initiators number)
2. RQ with 128 entries for 64 initiators:
   - Mem = 64 * 128 * (1200 + 4096) = 41.4 MiB

Results:
FIO_JOBS   kIOPS     Bandwidth,MiB/s  AvgLatency,us  MaxResidentSize,kiB
       RQ       SRQ     RQ      SRQ    RQ       SRQ      RQ       SRQ
1      8.623    8.623   33.7    33.7   13.89    14.03    144376   155624
2      17.3     17.3    67.4    67.4   14.03    14.1     145776   155700
4      34.5     34.5    135     135    14.15    14.23    146540   156184
8      69.1     69.1    270     270    14.64    14.49    148116   156960
16     138      138     540     540    14.84    15.38    151216   158668
32     276      276     1079    1079   16.5     16.61    157560   161936
64     513      502     2005    1960   1673     1612     170408   168440
128    535      526     2092    2054   3329     3344     195796   181524
256    571      571     2232    2233   6854     6873     246484   207856

We can see the benefit in memory consumption.

Change-Id: I40c70f6ccbad7754918bcc6cb397e955b09d1033
Signed-off-by: Evgeniy Kochetov <evgeniik@mellanox.com>
Signed-off-by: Sasha Kotchubievsky <sashakot@mellanox.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/428458
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-03-15 19:19:17 +00:00
Pawel Wodkowski
5c6ca5b6e0 jsonrpc: add Python with statement support
The JSONRPCClient class now support the with statement. From now on the
__del__() method is changed to close() and user need to call it manually
at object disposal or use the 'with' Python statement.

Change-Id: I74afd93d411b9596ab8f9523c54a4a7523eec5e5
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/444686
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Pawel Kaminski <pawelx.kaminski@intel.com>
2019-03-15 05:01:27 +00:00
Pawel Wodkowski
27cc63ec68 rpc: extend logging in JSONRPCClient class
Add extra long argument '--verbose <VERBOSE LEVEL>' to add more log
levels Default is error. Legacy'-v' parameter is translated into INFO
log level.

Some parts of code are logged using DEBUG level.

Change-Id: I2aaa6e2fbaf9b2101c6eeaec0cef6141636f135b
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/436528
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-03-15 05:01:27 +00:00
Jim Harris
db524432dc bdev: remove delete_bdev RPC
This RPC was deprecated a couple of releases ago.
bdev modules now each have their own RPC for deleting
bdevs.  Due to how bdevs are created differently on
different modules, it is simply not possible to
have one delete_bdev RPC that would work for all bdev
types.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ia46c95dce6e35f7557e8d41c41b8fea382924547

Reviewed-on: https://review.gerrithub.io/c/442615
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-03-01 08:50:07 +00:00
Tomasz Zawadzki
ca87060dcc lvol: add option to change clear method for lvol store creation
Default 'unmap' option stays as it was.

'Write_zeroes' comes useful when one wants to make sure
that data presented from lvol bdevs on initial creation presents 0's.

'None' will be used for performance tests,
when whole device is preconditioned before creating lvol store.
Instead of performing preconditioning on each lvol bdev after its creation.

Change-Id: Ic5a5985e42a84f038a882bbe6f881624ae96242c
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/442881
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-02-28 20:50:27 +00:00
Jim Harris
e632df9d4f rpc.py: cleanup some help text
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I36f80fd936a5afefd79721b4c0ed077410fb6798

Reviewed-on: https://review.gerrithub.io/c/445935
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-02-25 07:06:04 +00:00
Shuhei Matsumoto
260f9a77c3 bdev/nvme: Set per-controller PRCHK options by JSON RPC
Add prchk_reftag and prchk_guard to construct_nvme_bdev RPC.
In spdk_rpc_construct_nvme_bdev, create prchk_flags based on them
and pass it to spdk_bdev_nvme_create, and in spdk_bdev_nvme_create,
pass it to create_ctrlr.

A single option enable_prchk may be enough but add separate options
for reftag and guard to clarify that apptag is not supported yet.

The next patch will make per-controller PRCHK options configurable
by .INI config file.

Change-Id: I370ebbe984ee83d133b7f50bdc648ea746c8d42d
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/c/443833
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-02-12 09:14:31 +00:00
paul luse
8a1acca65d bdev/raid: Add strip_size_kb rpc param for create
strip_size as an rpc param is now deprecated and can be removed
in a future release.  Either strip_size or strip_size_kb can be
used but only one of them or the rpc will fail.

Internally we maintain both fields because strip size always
comes in as KB but we convert it to blocks so having both elements
makes it clear for developers what they're looking at.

JSON output includes both strip_size and strip_size_kb.

Fixes #550

Change-Id: I5dc51e8af22eae3d56af8f8d37a564dbaae228fa
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/437873
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-02-11 22:46:33 +00:00
Ben Walker
ad272579b9 nvmf: Clarify wording of buf cache size parameter
I think this wording is slightly more clear.

Change-Id: I9fc500b8b87c0520bdf96e6c5db490bb98121da3
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/442435
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-02-04 19:16:03 +00:00
Pawel Wodkowski
48834f0daa vhost-scsi: use first free SCSI target ID if -1 specified
Fixes #328

Change-Id: I34b816a31a51d1a8aa4c61285e01d0249c283f53
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/442434
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-02-04 18:16:35 +00:00
Jim Harris
51725c486c nvmf: remove deprecated construct_nvmf_subsystem RPC
This was marked deprecated in the v18.10 release, so
remove it now before v19.01 is tagged.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I57673a5ab475b97c812bebcefd77ff90d9305d1c

Reviewed-on: https://review.gerrithub.io/c/442412
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-01-30 00:48:23 +00:00
Ben Walker
d1f943239e nvmf: Remove deprecated set_nvmf_target_options rpc
Marked as deprecated in 18.10.

Change-Id: I40d0e6103623aee6e6a0b9fa6e82f7b826ca1fe6
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/442420
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-01-29 01:30:30 +00:00
Ben Walker
b5c7ae072d bdev/virtio: Remove deprecated remove_virtio_scsi_bdev rpc
Marked as deprecated in 18.10

Change-Id: I67eb3e7b83995c2e53905f9a6bc5b8562a3dffc4
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/442418
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-01-28 22:18:53 +00:00
Ben Walker
9a016643dc bdev/virtio: Remove deprecated construct_virtio_pci_blk_dev rpc
Marked as deprecated in 18.10 release.

Change-Id: Icbe3ef087ba681189f9b440e4b9cecaad02eb874
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/442417
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-01-28 22:18:53 +00:00
Ben Walker
766b7ef2ed bdev/virtio: Remove deprecated construct_virtio_user_blk_dev rpc
This was marked deprecated in 18.10

Change-Id: Id47e770b0388c935fe684aeef7a9824f24cef47f
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/442416
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-01-28 22:18:53 +00:00
Ben Walker
63edd8bad0 bdev/virtio: Remove deprecated construct_virtio_pci_scsi_bdev rpc
Marked as deprecated in 18.10

Change-Id: If7e105194e2ca8227b3b50b8e7b02c633c27daaf
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/442415
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-01-28 22:18:53 +00:00
Ben Walker
8683a0fda1 bdev/virtio: Remove deprecated construct_virtio_user_scsi_bdev RPC
Marked as deprecated in 18.10.

Change-Id: Iac9f0033204dd468c2df11653ae46410dfbb1f7a
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/442414
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-01-28 22:18:53 +00:00
Vitaliy Mysak
199080cfa2 OCF: rpc: add get_ocf_bdevs method
Add new RPC method for OCF bdev: get_ocf_bdevs
It is useful in respect to not registered OCF bdevs
  which do not appear in standard get_bdevs call

Change-Id: I8a5fc86a880b04c47d5f139aa5fa4d07ca39c853
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.gerrithub.io/c/441655
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-01-28 18:53:31 +00:00
Vitaliy Mysak
27e0190b84 OCF: rpc: add stats method
Add get_ocf_stats rpc method that
  returns available OCF statistics for specific OCF bdev

Change-Id: If043a18c847fbeeddd8fdde1af9397e24bd90718
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.gerrithub.io/c/438411
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-01-28 18:53:31 +00:00
Vitaliy Mysak
35366140d8 OCF: add RPC
Add rpc methods for vbdev_ocf:
  * construct_ocf_bdev
  * delete_ocf_bdev

Change-Id: Ie75a2b4bf4f88462c00651d195d22d156478abf8
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.gerrithub.io/c/435710
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2019-01-28 18:53:31 +00:00
Wojciech Malikowski
7c233adeff bdev_ftl: Added FTL bdev module
This patch adds FTL bdev. RPC scripts have been updated to allow for
creation and removal of FTL bdevs.

Change-Id: I82a5c5033b65bbeb67c238cae969a68cff767dcc
Signed-off-by: Jakub Radtke <jakub.radtke@intel.com>
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/431329
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-01-24 18:55:56 +00:00
Piotr Pelplinski
e5e427c94b histograms: add rpc calls
This patch adds RPC calls for histograms in bdev layer.
Following calls are added:
 - enable_bdev_histogram - enable/disable histogram structures for specified bdev and each of its channels.
 - get_bdev_histogram - merges histograms from all channnels and encodes histogram as base64


Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Change-Id: Ib423a919dc1cde7dd7d92247db5482cfb9d66956
Reviewed-on: https://review.gerrithub.io/c/433573
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: wuzhouhui <wuzhouhui@kingsoft.com>
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-01-24 14:14:32 +00:00
Maciej Szwed
adb39585ef lvol: add option to change default data erase method
Some users require to do write zeroes operation when
erasing data on lvol. Currently the default method is
unmap. This patch adds flag to spdk_rpc_construct_lvol_bdev
call that changes default erase method. This is also a base
implementation for possible future function for erasing
data on lvol bdev.

Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I8964f170b13c2268fe3c18104f7956c32be96040

Reviewed-on: https://review.gerrithub.io/c/441527
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2019-01-23 22:25:37 +00:00
GangCao
05b43152b2 QoS/Bdev: add RPC support for Read/Write separate bandwidth limits
This patch adds the RPC support for the Read/Write separate
bandwidth limit controls. The basic usage as following:

usage:
rpc.py set_bdev_qos_limit [-h] [--rw_ios_per_sec RW_IOS_PER_SEC]
                               [--rw_mbytes_per_sec RW_MBYTES_PER_SEC]
                               [--r_mbytes_per_sec R_MBYTES_PER_SEC]
                               [--w_mbytes_per_sec W_MBYTES_PER_SEC]
                               name

positional arguments:
  name       Blockdev name to set QoS. Example: Malloc0

optional arguments:
  -h, --help show this help message and exit
  --rw_ios_per_sec RW_IOS_PER_SEC
             R/W IOs per second limit (>=10000, example: 20000).
             0 means unlimited.
  --rw_mbytes_per_sec RW_MBYTES_PER_SEC
             R/W megabytes per second limit (>=10, example: 100).
             0 means unlimited.
  --r_mbytes_per_sec R_MBYTES_PER_SEC
             Read megabytes per second limit (>=10, example: 100).
             0 means unlimited.
  --w_mbytes_per_sec W_MBYTES_PER_SEC
             Write megabytes per second limit (>=10, example: 100).
             0 means unlimited.

Change-Id: I822ec4814d21adff9826ce03a6af3783b1b98f44
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.gerrithub.io/c/417650
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-01-22 23:21:43 +00:00
Tomasz Zawadzki
3bb815ae9f lvol: exposed marking lvol bdev as read only
Added set_read_only_lvol_bdev() RPC that marks
lvol bdev as read only.
This enables to create clones off of a base lvol,
without having to create additional lvol bdev with snapshot.

Change-Id: Ic20bbcd8fbebcef157acce44bfa1da035b0da459
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/440534
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-01-18 19:02:36 +00:00
Seth Howell
2740f26841 nvmf_rpc: add control for rdma buffer cache size
Change-Id: I63e433a7e0aa5374b12ca6c12a4855900cf2d2c6
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/439431
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-01-18 16:57:37 +00:00
Ziye Yang
58f1624497 nvmf: add the transport shared buffer num configuration option.
Previously, we allocate the buffer size according
to the MaxQueueDepth info, however this is not exactly
a good way for customers to configure, we should provided
a shared buffer number configuration for the transport.

Change-Id: Ic6ff83076a65e77ec7376688ffb3737fd899057c
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/437450
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-12-20 19:55:57 +00:00
Seth Howell
cc7a099c0c rpc: reconcile options for construct_nvme_bdev
Change-Id: I7a13559312979f5019b42305ebe02ec102dda2a1
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/437574
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-12-20 17:31:43 +00:00
Seth Howell
088379cf74 nvme_rdma: Add source addr info to RPC
Change-Id: Id6b5aee4b36d828a0f9e5ddc85293a15342d2aae
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/436220
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-12-20 17:31:43 +00:00
Liu Xiaodong
ae0aae1532 lib/trace: add 3 RPC method for tpoint_group_mask
Now trace facility can be enabled/disabled at runtime,
besides app starting parameters

Change-Id: I086564665ba4375b946a5339d5921d6584e9bfda
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/435470
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-12-14 21:29:12 +00:00
Patrick Ohly
4cfae03606 bdev/rbd: full control over client configuration
When the caller of the RPC API has all the necessary information about
how to access a Ceph cluster, then having to create configuration
files before calling the RPC API is problematic (has to touch files
owned by a local admin, changes must be removed again).

But having to encode support for certain configuration options in SPDK
is also problematic, because that might change depending on the
librados version.

The approach taken here is to merely pass through arbitrary key/value
config options. Existing config files are ignored when that happens.
The caller of the RPC then has full control over the connection setup
and can be sure that he does not inherit settings from a local file
accidentally.

In addition, user management is supported now, with or without a
config. This is useful for accessing a volume with a less privileged
user. Previously, passing NULL to rados_create implicitly chose the
"admin" user.

Change-Id: I5e7f36092df663a3d7ac503c04fc624a8fe1208e
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Reviewed-on: https://review.gerrithub.io/430460
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-12-05 17:49:57 +00:00
Pawel Wodkowski
f7cb69cb60 scripts/rpc: remove call_cmd decorator
This decorator is not needed. This also removes one exit point in case
of exception.

Change-Id: I7e1492cf9cccdb891dd70494652e0bc86a8bbcca
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/435469
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-12-05 00:42:37 +00:00
Jim Harris
870b0e5b29 log: remove "trace" from log flag RPCs
We will keep the RPCs for now but mark them deprecated.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I0407dcb392ea0c9e89c0f26cd5670aed2dbfadef

Reviewed-on: https://review.gerrithub.io/435345
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-12-03 19:50:15 +00:00
Seth Howell
b7f54bd66e app: RPC to wait for app subsystem initialization.
Change-Id: I66a9f9c1a92c8b9185a0c4c882e61b028e1da261
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/432491
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-11-12 22:21:33 +00:00
Seth Howell
962ba4e89a nvmf: remove tgt_opts from nvmf_tgt
This option is deprecated. Also, rename the rpc and configuration
options for setting the opts to reflect that they now only set the max
number of subsystems

Change-Id: Iaabcbf33dd0a0dc489d81233fda74e9e7f3e0d2e
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/430161
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-11-08 23:08:26 +00:00
paul luse
164d79143b passthru: add RPC testing
And change names of parms to be consistent with others used in
RPC testing.

Change-Id: I8331c6a22866d89a2a4ffb5fc8d41d74b4b7b07d
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/428724
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-11-06 17:32:08 +00:00
Liu Xiaodong
98187ed9a7 rpc: revise comments for construct_nvme_bdev
Just considering the situation of multi-namespaces device.

Change-Id: I1b0d502874ab53891060c973a3ae75e81ec591f2
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/431698
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-11-02 19:02:43 +00:00
Seth Howell
c89b7bac82 nvmf: add rpc for getting transport information
Change-Id: I8c3cbc1b08a219844c9b25f8c8494efd09e347f6
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/430623
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-11-02 18:04:06 +00:00
Liu Xiaodong
43b335f917 scripts/rpc.py: add method "get_nvme_controllers"
"get_nvme_controllers" can be used to list NVMe controllers.
Names and transport IDs of the NVMe controllers will be
reported by this method.

Change-Id: Ie59b567afc09e70475f97939e86a872af39c5d8a
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/419094
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: GangCao <gang.cao@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-10-25 17:26:25 +00:00
Liu Xiaodong
e66ea62412 scripts/rpc.py: add method "send_nvme_cmd"
An example to send nvme_cmd rpc request:
scripts/rpc.py send_nvme_cmd -n Nvme0 -t admin -r c2h \
-c <base64_urlsafe encoded cmdbuf> -D 4096

nvme-rpc will be processed internally by bdev_nvme.

Change-Id: I6e731b76be0f503d48154a8b34a1e81b4b454396
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/417962
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-10-22 22:01:14 +00:00
Pawel Kaminski
09a9130ed2 test/spdkcli: Add load and save config commands.
Change-Id: I499f54b025080ad1916acc0cf265a58c806da002
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.gerrithub.io/428494
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-10-15 17:32:09 +00:00
GangCao
868c28cd13 QoS/Bdev: add the RPC support for the bandwidth rate limit
This patch added the support of RPC method to enable,
adjust, disable the bandwidth rate limit on the bdev.

And it can work together with the existing IOPS rate limit.

The RPC method has been consolidated to support both IOPS
and bandwidth rate limits as below:

usage:
rpc.py set_bdev_qos_limit [-h]
                          [--rw_ios_per_sec RW_IOS_PER_SEC]
                          [--rw_mbytes_per_sec RW_MBYTES_PER_SEC]
                          name

positional arguments:
  name       Blockdev name to set QoS. Example: Malloc0

optional arguments:
  -h, --help show this help message and exit
  --rw_ios_per_sec RW_IOS_PER_SEC
             R/W IOs per second limit (>=10000, example: 20000).
             0 means unlimited.
  --rw_mbytes_per_sec RW_MBYTES_PER_SEC
             R/W megabytes per second limit (>=10, example: 100).
             0 means unlimited.

Change-Id: I9c03cd635280add01801a81c6a6c02f0cf85bee1
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.gerrithub.io/416511
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-10-09 18:26:44 +00:00
wuzhouhui
74a28c199f vhost: rpc: add an optional parameter -n/--name for get_vhost_controllers
User can use this optional parameter to get the information of specific
vhost controller.

Change-Id: I3911c6c7d4e7b75e82277d1e4690d5e40019aa06
Signed-off-by: wuzhouhui <wuzhouhui@kingsoft.com>
Reviewed-on: https://review.gerrithub.io/425451
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-10-04 23:33:04 +00:00
paul luse
cafd537c7d test: exercise crypto rpc calls
Most vbdevs have complete test suites that include rpc calls
to set them up and teare them down.  With crypto there's not
a lot to do w/test other than make sure it's configured and
run IO so right now that amounts to tools that use a config
(ini) file.  Adding create/delete calls here to get basic rpc
coverage in CI.

The fixes/udpates in the vbdev module and the .py files were
fixes found in enabling testing of the rpc calls.

Change-Id: I5d7d25e1c96dcf76e0055f23d27781b858e290b0
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/425875
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-09-27 22:07:02 +00:00
Ben Walker
2eacfd8786 nvmf: Generate a reasonable default subsystem serial number
If the user does not specify a serial number, just set it
to all 0.

Change-Id: Ie1aaa4c1bf5f9eaf76fcc9ed6f6bd6ded2aeb200
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/425324
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ziye Yang <optimistyzy@gmail.com>
2018-09-20 18:35:27 +00:00
Paul Luse
51606ed402 bdev: Add crypto virtual bdev module
Initial support for softare AESNI_MB DPDK driver only.

Have tested (both aesni and QAT seprately and concurrrently) on underlying NVMe devices
with bdevio and a bdevperf script that runs IOs from 512B to 128K each with Q depths from
1 to 512 in powers of 2 for 30 seconds each run.

QAT can be included in the code (but not makefile) and marked as experimental
until we are ready to test in CI.  It works well on 2 systems but is a big PITA to get
the hardware setup and configured for use with DPDK (IMHO).

Change-Id: If518c3df8e74e00efa18afdf194824c5e69778fc
Signed-off-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/403107
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-09-17 21:23:14 +00:00
John Barnard
183d81d0c6 nvmf: Move target opts to transport opts (part 2)
- Add independent functions to create transport with specific opts
  and add to target while maintaining backward compatibility with
  current apps and rpc configuration that still use the add listener
  method to create a transport.
- Add new rpc function to create transport and add to target.
  + Update json reporting to include new rpc function.
  + Update python scripts to support new rpc function.
  + New nvmf test script (cr_trprt.sh) to test new rpc function.

Change-Id: I12d0a42e34c9edff757755f18a78b722d5e1523e
Signed-off-by: John Barnard <john.barnard@broadcom.com>
Reviewed-on: https://review.gerrithub.io/423590
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-09-17 20:42:16 +00:00
Pawel Wodkowski
b96f97cf29 scripts: use python3 in all scripts
This it to prepare for RPM package. Also lower number of dependencies
needed by SPDK tools.

Update changelog to deprecate Python 2 and explicit interpeter invoking
in scripts.

Change-Id: I2497cca721cbcbadc1c99c675f8b8b7f682d5efa
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/425233
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-09-14 22:24:30 +00:00
Ben Walker
aedac1a878 nvmf: Add an nvmf_subsystem_create rpc
Add a new nvmf_subsystem_create RPC. The current RPC,
construct_nvmf_subsystem, is old and jumps through a lot
of hoops to maintain backward compatibility. Leave it in,
but create a new RPC as the main mechanism going forward.

Change-Id: Ia99cb5994a90579956136dac1800ed1fdce482dd
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/425169
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-09-12 13:20:32 +00:00
Avinash M N
1f3f160581 nvmf: Add IP based load balancing
This patch adds support for IP based load balancing. All the rdma connections
coming from a particular IP will be assigned the same core on target. The load
balancing policy - roundrobin/hostip can be configured through rpc during startup only.
Roundrobin is the default policy.

Change-Id: I7bb216cb4cb2639fa401e23090143c4cf1d7bb5a
Signed-off-by: Avinash M N <avinash.m.n@wdc.com>
Reviewed-on: https://review.gerrithub.io/422190
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-09-11 23:55:25 +00:00
Jim Harris
55014ddf53 rpc: remove file parameters from save/load_config in rpc.py
User can pipe or redirect data as needed.  As features like
bdev crypto are added, storing config data in files needs to
be carefully scrutinized - don't make it easy by providing
ways to write data to files that may not be sufficiently
protected.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I24dd32f81b965e275e1bac875f6d008d1b68007a

Reviewed-on: https://review.gerrithub.io/425165
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-09-11 21:11:51 +00:00
Jim Harris
ba38785b7e bdev: remove get_bdevs_config RPC
This RPC does not work for a lot of bdev types.  For
example, NVMe namespaces and virtio scsi LUNs are not
explicitly constructed by an RPC - they are indirectly
constructed by an RPC associated with an NVMe controller
or virtio-scsi controller.

While here, remove spdk_bdev_config_json.  It was
only created to facilitate this get_bdevs_config RPC.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I537166d8f91ab458bd2000859d74f7254bfc9c0a

Reviewed-on: https://review.gerrithub.io/424584
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-09-05 19:46:17 +00:00
Shuhei Matsumoto
3a08001dd4 iscsi&scripts/rpc: Add get_iscsi_auth_groups RPC to get current configuration
Add an new RPC to get current authentication group configuration.
This patch is utilized in the next patch to support JSON config dump
for authentication group configuration.

Change-Id: I34be9e196f8e7a484bcd316da54f05d0f6ee0300
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/421468
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-08-29 21:40:31 +00:00
Shuhei Matsumoto
ad323b8064 iscsi&scrpts/rpc: Require to specify CHAP secret file explicitly to load it
Previous patches enabled users to configure CHAP secrets dynamically
by RPCs. Subsequent patches will enable users to load CHAP secrets
from JSON config file.

Loading CHAP secret file is done by default and this will conflict to
JSON config file.

Hence the path to CHAP secret file is required to specify in the config
file or JSON RPC set_iscsi_options explicitly after this patch.

Users who have used CHAP secret file are expected to specify it explicitly
and this will be no harm for them.

Besides, CHAP secret file is not oly for discovery sessions but also for
login to iSCSI targets. However there were wrong description to make user
misunderstand. Hence remove these wrong description in this patch too.

Change-Id: Ic4093cabc0c14b87e26baef4bba6b0d292e40c06
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/421467
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-08-29 21:40:31 +00:00
Shuhei Matsumoto
750e5d25fb iscsi&scripts/rpc: Add add/delete_secret_to/from_iscsi_auth_group RPCs
Add RPCs to add/delete a secret to/from an existing authentication group
dynamically.

Use mutex to ensure exclusive access to CHAP secrets.

Tries to use descriptive message in RPCs when error occurs.

Change-Id: I59650ae11a2fe675d03b90bbd4d2dc5b9c0160ed
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/421465
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-08-27 21:13:49 +00:00
Shuhei Matsumoto
dcafd5b1ea iscsi&scripts/rpc: Add add/delete_iscsi_auth_group RPCs
This patch adds new RPCs add/delete_iscsi_auth_group to add and
delete authentication group dynamically.

Mutex is added to ensure exclusive access to CHAP secrets.

Additionally provide descriptive message in RPC when error occurs.

Change-Id: Iaddfbdd5688ca7907d2c7d859835faa056deecd1
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/421464
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-08-27 21:13:49 +00:00
GangCao
d393983d74 lib/iscsi: export FirstBurstLength as user configurable parameter
According to https://tools.ietf.org/html/rfc3720, the
default value for the FirstBurstLength is 65536 bytes
while SPDK iSCSI target picks the smaller 8192 as the
default setting. This value is the communication for
the iSCSI initiator to send the unsolicited data and
instead of having a fixed setting here, expose it as a
user configurable parameter to fit the real use case,
especially for the data out iSCSI write.

Example of usage as following in the iSCSI.conf:

FirstBurstLength 8192

Change-Id: I71690c7c48aa0875f1f975c0ea935389de6d1e6d
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.gerrithub.io/421142
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-08-27 19:42:07 +00:00
Shuhei Matsumoto
844735c928 iscsi: Add set_iscsi_target_node_auth RPC to configure CHAP dynamically
An new RPC set_iscsi_target_node_auth enables users to configure CHAP
authentication for the target node dynamically.

New setting is used for new iSCSI sessions. For existing iSCSI sessions,
new setting is not used until user logout and login again.

Try to use descriptive message in the RPC when error occurs.

Change-Id: I0bd40d92262d708c1f7de0effb208078bdf8cc41
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/420974
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-08-22 22:39:26 +00:00