So, in an attempt to be a “good SPDK citizen,” when I noticed that
check_format.sh suggested I install shellcheck, I did and re-ran
my commit.
Got bunches of errors like this on unmodified SPDK bash scripts:
In scripts/ceph/start.sh line 58:
echo -e "\tosd data = ${mnt_pt}" >> "$ceph_conf"
^-- SC1117: Backslash is literal in "\t".
Prefer explicit escaping: "\\t".
Adding SC1117 to the list of ignored warnings lets check_format
run clean.
ShellCheck is at version 0.5.0, Ubuntu at 19.04.
Signed-off-by: James Bergsten <jamesx.bergsten@intel.com>
Change-Id: I1705b3d15338a18a3fde3ae796a7c46bbf92b1cd
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/465448
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
The fact that all latencies were in microseconds was not explicit in all
of the help/docs for this bdev.
Change-Id: Id13fbb55af62c5f14fd2790e894c4c0646eb25f7
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/465169
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
This is patch that implements changes according to proposal
submitted in https://review.gerrithub.io/c/spdk/spdk/+/453036
Change-Id: I5423cd34cb5fc111b34cab2b7f7c6c5f11898af8
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/464677
Reviewed-by: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Fix SC2064 issue.
Use single quotes, otherwise trap expands now rather than
when signalled.
Change-Id: I0b3a9157f52eed037e8d217f639c64d6876ec1e1
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/464655
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Pawel Kaminski <pawelx.kaminski@intel.com>
This will allow us to do interesting things in tests like configure a
bdev with a really long delay, and allow I/O to build up on a target to
see what happens, then decrease the latency to allow traffic to flow
through normally without ever having to delete and reconfigure the bdev.
Change-Id: Ibcb1101f8eed9fe3094ba239110cb4e49ace6554
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/464454
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Fix SC2007: Use $((..)) instead of deprecated $[..].
SC2007 removed from check_format.sh exclude list.
Change-Id: Ifd858857e461d785d6d6f101acca13c326ee637e
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/464172
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Fall back to tty output in case diff cannot be used for
not auto-fixable errors.
Change-Id: I481ad0dbe8e42cf510fa3a02636bc0117630da14
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/464171
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Removing parentheses around if condition.
This unnecessarily spawns a subshell.
Removing SC2233 (Remove superfluous (..) around condition)
from check_format.sh exclusion list.
Change-Id: I0a4c266667264220a898ce80f587ae61bc113821
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/463627
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Changing according to styling check done by ShellCheck.
Removing from check_format.sh exclusion list:
SC2236 - Use -n instead of ! -z
SC2070 - -n doesn't work with unquoted arguments. Quote or use [[ ]]
Change-Id: Ia9d645b9d0ce31b67c4de682395cf36f4ddc8d1f
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/463180
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Fixing errors reported by ShellCheck:
SC1009: The mentioned syntax error was in this if expression.
SC1073: Couldn't parse this test expression. Fix to allow more checks.
SC1019: Expected this to be an argument to the unary condition.
SC1020: You need a space before the ].
SC1072: Missing space before ]. Fix any mentioned problems and try again.
Removing mentioned errors from check_format.sh exlusion list, as
all of these errors should be tested for regularly.
Change-Id: I19402baba7bb8e59ed707de97133bd0a579333cf
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/463174
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Fix shellcheck styling error SC2006. Use of backtics is deprecated
and dollar-parentheses are encouraged instead.
Removing SC2006 from check_format.sh error exlusion list.
Change-Id: I8ef9d782839ff4361386720c39a28e449b5efab9
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/463801
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Add Bash styling checks using shellcheck:
https://github.com/koalaman/shellcheck
Shellcheck can be used in check_format just like astyle
and pep8 to improve our bash scripts quality by checking
for syntax errors.
It could also allow us to enforce (at least to some
extent) consistent coding style - using $() instead of
backticks for command calls, string quoting, etc.
Shellcheck maintains it's own wiki with error descriptions:
https://github.com/koalaman/shellcheck/wiki/
To check for specific error go to, for example:
https://github.com/koalaman/shellcheck/wiki/SC2006
Each wiki entry includes an example of problematic code, an
example of suggested correct code and a rationale.
Currently all of the found problems are excluded in check_format.sh
and will be fixed incrementally.
Change-Id: Ib1d6f628e101e0e2b2d56956b679942630a73f95
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/463172
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Seth Howell <seth.howell@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Add shellcheck package for bash syntax and styling checks.
Enabling in check_format.sh done in separate patch.
Change-Id: I654f4168d5fab55dc7e1eeaa353ea6da24d04dd7
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/463866
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Seth Howell <seth.howell@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Connections will soon be assigned to poll groups, which will be
dynamically moved between CPU cores based on load. It no longer makes
sense to restrict certain portal groups to specific cpu cores in this
model.
Change-Id: Iee983d75febc9797aa60021c5bc0680335e895cd
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/463358
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
A single I/O may allocate more than one request, since splitting may be
necessary to conform to the device's maximum transfer size, PRP list
compatibility requirements, or driver-assisted striping. Very big
I/O request sent from application may get error due to limited resources
in NVMe driver layer, so here we add an optional parameter to make the
parameter can be configured by users.
Fix issue #745.
Change-Id: I7824232c54865b052dcd0ec6e91484c3837fc2c4
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/461182
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
On multiple development systems (including mine), this results
in new error messages when running scripts/setup.sh. This has
the potential of causing lots of questions when users upgrade
to SPDK 19.07.
I understand the desire for the error messages, but I think we have
to come up with a smarter check than just against "unlimited".
This reverts commit 0abee610f0.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I2854af556b67a9a4e5f686c72c4407962808f964
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/463357
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This internal tool can now be passed an optional --cross-prefix,
which sets up the prefix used for a cross compiler.
Change-Id: Ia4bbbae3bd5a2e4ddc9da342cd03d600e9ee6099
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/463016
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
We need to know this up front.
Change-Id: I3a9ceb90cf62eacbf3fdf518a9ccb4c4978b3a05
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/463014
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Arguments for this function were added as optional arguments
and were listed as such by argparse when using rpc.py --help.
This is confusing as all of these arguments are obligatory
in order to create a crypto bdev in SPDK configuration.
Change-Id: I7c31413314f28e44ee008d3d7c28759063700b61
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/460968
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Added parsing and saving of (optional) parameters which are responsible for
relocation in FTL. Changing the parameters may affect Write
Amplification Factor and overall performance, especially during random
write workloads. If parameters are not specified, default values will be used.
Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: Ic0229e39109460f3541c31e1cbf2a485efe408c6
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/460504
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Catch exceptions during RPC client initialization
to display meaningful error message.
Withouth this change, user gets stacktrace
when e.g. SPDK application is not running.
Before:
```
Traceback (most recent call last):
File "scripts/rpc/client.py", line 53, in __init__
raise socket.error("Unix socket '%s' does not exist" % addr)
OSError: Unix socket '/var/tmp/spdk.sock' does not exist
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "scripts/spdkcli.py", line 74, in <module>
main()
File "scripts/spdkcli.py", line 47, in main
with rpc.client.JSONRPCClient(args.socket) as client:
File "scripts/rpc/client.py", line 56, in __init__
"Error details: %s" % (addr, ex))
rpc.client.JSONRPCException: Error while connecting to /var/tmp/spdk.sock
Error details: Unix socket '/var/tmp/spdk.sock' does not exist
```
After:
```
Error while connecting to /var/tmp/spdk.sock
Error details: Unix socket '/var/tmp/spdk.sock' does not exist. SPDK not running?
```
Change-Id: I65862965b68acf3bd4709de598f04de49da27de2
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462020
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
RPC client constructor accepts optional port,
but does not check if it's None in case of
remote connection.
This patch adds the check and new error message
related to it.
Case where port is unexpectedly None
can be reproduced by starting spdkcli
when SPDK socket does not exist.
Change-Id: I46e0b99547204c6fdeac421e5de9d6991387e207
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/460974
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
This priority is used to differentiate the sock priority on the TCP connections
between NVMe-oF TCP target and other TCP based applications.
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I6ee294e647420b56d1d91a07c2e37bf34ce24e03
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/461801
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
JSONRPCClient.call recieved optional `params` value,
but then `params` was used in `json.dumps` function (line #159)
as a dictionary, so `None` was treated as dictionary
in some cases, which resulted in TypeError.
This patch fixes above issue by changing
the default value from `None` to `{}`.
Change-Id: I27ca0ccc2d970c0ff9e8117f02203c03e417adf0
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462070
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Currently we do not switch driver for NVMe that is mounted.
The problem is that we take into consideration only the
first namespace, but any other namespace can be still mounted.
In such case we should not switch driver.
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: Idd13edccd0929574f2914a71e841e67871dd2e9f
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457762
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Purpose: To eanble the transport based scheduler in RPC.
Previously, we only support it with configuration file.
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I02ae9b1b316d4fec8b28b550e70dcdc78ce78722
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/461645
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This option was intended to enable the user to disable the c2h_option,
but because of the way arguments filter down through the python script,
it was actually impossible to disable the optimization.
My understanding is that typically we will want to keep this
optimization enabled on most systems, but the option should work like it
says it does.
Also, align the implementation of this function with the other ones on
the RPC i.e. use the store_true, store_false paradigm.
Change-Id: I59f0e9a573abf1d567e5539294c63c68899b35f1
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/461737
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
`pmem_pool_info` was incorrectly set to invoke
`delete_pmem_pool` RPC method.
The name of command was also incorrect - "info_pmem_pool"
instead of "pmem_pool_info", where
latter is the original name of RPC method.
This patch fixes both issues with `pmem_pool_info`.
This bug has not been caught because
there are no tests for `pmem_pool_info` command yet.
Change-Id: Icc207bc3a7cf6c817cb079137d7b473f52186a84
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/460840
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
This patch adds nvmf_get_stats RPC method and basic infrastructure to
report NVMf global and per poll group statistics in JSON format.
Signed-off-by: Evgeniy Kochetov <evgeniik@mellanox.com>
Change-Id: I13b83e28b75a02bc1dcb7b95cbce52ae10ff0f7b
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452298
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
This patch excludes *.patch files from some checks in check_format.sh:
- trailing spaces,
- POSIX includes
Change-Id: Ic55ce7f4128ddc946d235b4ca487061075edc03b
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/460939
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Add an new optional parameter dif_insert_or_strip to
nvmf_create_transport RPC.
.INI config file will be deprecated and dif_insert_or_strip is not
supported in .INI config file.
Change-Id: Ibf38b599cff75eeb0056dd2125d6ec10d444f339
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458927
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Scripts were using a mix of two approaches, lets unify that so
just dollar-parenthesis syntax is used.
Also update two scripts from spdk/test which we previously missed.
Change-Id: I429f9bc158076462b419fae597f716c329f9b7aa
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/460344
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
There will be one more "mocked" implementation soon,
so make existing names more descriptive now.
Change-Id: I30daf2b60ab683d5befb06eca50d455712858d61
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459390
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
The previous name wasn't very accurate. The function
accepts a json object, so call it print_json().
Change-Id: If1cae587fd13f0a8a2761049125fc7fb234c0360
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459388
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Also, construct this path by grabbing the home directory
of the current user.
Change-Id: Ia8d5cc2be40c0c3ac693c80a3132b970f7124183
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456704
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Changed initialization of the ftl lib when using an rpc call to
allow for usage of any default configuration parameters (currently
only allow_open_bands is exposed).
Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: I73457dfcacc6b1adeffd13ecc6e98001749e00cf
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459741
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
By default QAT will be selected if available however a new RPC
can be used to either auto-select (default) or specify either
ISAL or QAT.
Change-Id: I37cf7640bbd8cef455583e1eccb8adb59cc419d8
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456693
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
By now (5.1 is released), the Linux kernel initiator supports the
success optimization and further, the version that doesn't support
it (5.0) was EOL-ed. As such, lets open it up @ spdk by default.
Doing so provides a notable performance improvement: running perf with
iodepth of 64, randread, two threads and block size of 512 bytes for 60s
("-q 64 -w randread -o 512 -c 0x5000 -t 60") over the VMA socket acceleration
library and null backing store, we got 730K IOPS with the success
optimization vs 550K without it.
IOPS MiB/s Average min max
549274.10 268.20 232.99 93.23 3256354.96
728117.57 355.53 175.76 85.93 14632.16
To allow for interop with older kernel initiators, we added
a config knob under which the success optimization can be
enabled or disabled.
Change-Id: Ia4c79f607f82c3563523ae3e07a67eac95b56dbb
Signed-off-by: Or Gerlitz <ogerlitz@mellanox.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457644
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Removing Fedora 26 and 27 which are long past their EOL date.
Adding Fedora 29 and 30.
Change-Id: I35bc8b7a5ac693b7a5af0e6abb35869b18ed5777
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457924
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: yidong0635 <dongx.yi@intel.com>
Reviewed-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
The next patch in this series introduces and RPC that makes the
selection more flexible.
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I55617d3c37b51cf9474c358cb92f5218397c0c58
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457157
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Report issue on ubuntu18 :
"spdk/autobuild.sh: line 55: hash: scan-build: not found"
Check that ubuntu18 should install this package:
apt-get install -y clang-tools, to fit version of ubuntu18
and later.
Move clang-analyzer which for fedora to vm_setup.sh as well.
Change-Id: Ia702c492f8b0f64705c7c15ee57a861ca14521f9
signed-off-by: yidong0635 <dongx.yi@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457554
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Missing try-except block around call_rpc_func().
If called function returned error there was an uncaught exception
which resulted in backtrace printed to screen.
Change-Id: Ifb37c29d70b93ef648a0503643a51cc1967ef8d0
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456956
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Gives the ability to change behavior of restore after dirty shutdown
without recompiling ftl library. User can define if partial recovery
or error should be returned after such a scenario.
Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: I6dda40df7b92d6a377957e4a70a3eab91a6ac4a9
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456185
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Make construct_ocf_bdev call accept 'wb'
as allowed cache mode.
Previusly user could only create OCF bdevs
in WriteThrough or PassThrough modes.
Change-Id: Ic2e1c8d2905cb51fc13c080aa7b7a4dfd7d9387f
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455628
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
This change will allow easier debugging of
what parameters are sent to spdk target.
It will also help to analyse which commands will fail
if multiple commands are sent to rpc.py.
Change-Id: I8e75b5edce791a1fc41e83664add8893f4c1bbfb
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455636
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
This script was getting a bit unwieldy. adding argparse will make it
easier to add more options in the future or to set default values and
validate arguments going forward.
Change-Id: I1ffdbdf2082287ceb8a88cd3eb6ecf9bbd6c9e11
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455724
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
This module simply sits on another virtual bdev
and adds a simulated average and p99 latency to that drive.
Change-Id: Ie9fc91e27585fd0636cb7dc845cb41744bf24625
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453594
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Fixed the following:
1) Missing python paramiko module in package dependecies.
2) Paramiko connection error: TypeError: missing_host_key() missing 1 required positional argument: 'key'
The set_missing_host_key_policy call should take a class instance AutoAddPolicy(), instead of the class name.
3) JSON config file defines "rdma_ips" instead of "nic_ips"
4) Added result file location in the documentation
5) Added env variable for PYTHONPATH to documentation
6) config.json changed default rw to randrw instead of read
Change-Id: I96624e2912131f62254c684a6c03a53a7f806cde
Signed-off-by: John Kariuki <John.K.Kariuki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455285
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Liang Yan <liang.z.yan@intel.com>
After iSCSI login sysfs is not updated immediately, but when session is
initialized. It can take a while and causes that sometimes scripts
cannot write to sysfs.
Change-Id: Ie4fa512704f022041bb3d9ab02dbb4210ddd04a0
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/454890
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
To bind VMD device it need to be specified
in PCI_WHITELIST.
Change-Id: If19c47941278e4df9c6157024bba8b76c90695c6
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Signed-off-by: Orden Smith <orden.e.smith@intel.com>
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455635
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Make the old name a deprecated alias.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ibbf50676e0d989b67121e465fc140f94faec46ed
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453033
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This is a bit more flexible in case script accidentally
has an extra newline at the end of the rpc.py input.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I1ebf08681e5900a4817712c7b62e4d5c21d43e54
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453956
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Adds support to spdkcli.py for raid devices via UIRaidBdev with
functionality to construct and destroy raid bdevs.
Information regarding the raid can be viewed via 'show_details' on a
given pooled_device child in the tree.
Signed-off-by: Mike Carlin <mikefcarlin@protonmail.com>
Change-Id: I2ca89e641b201c09b3d4db070eac6cc0c2cdec73
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452774
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
We marked the old set/clear/get_trace_flags as
deprecated in v19.01. These are now called
set/clear/get_log_flags (to reduce confusion with the
tracing framework). Use the new methods in this
patch series to mark these RPCs as deprecated.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I5ce8992ce8f6c1de5d5596b5f94a1587555e8546
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453563
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This will get used in upcoming patches that actually
create RPC aliases.
Thanks to Ben Walker for help on the deprecated_alias
helper.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ica4432888b1ecb9bd234690758f67ef82794c50a
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453035
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Ordinary python str.split() will by default split on each whitespace.
This includes arguments for rpc.py which expect a whitespace separated list,
eg. construct_raid_bdev -b "Bdev_1 Bdev_2".
shlex.split() prevents splitting on quoted arguments.
Change-Id: If37eb87cf82a161263f2b247baff5c8a77c43efc
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453652
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
If user calls rpc.py with no parameters, it currently
hangs, waiting for input. It should print help in this
case instead.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I5939999c57b8db0916267935d79ccc1a3201748b
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453431
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
There are cases where srq can be a detriment. Add a flag to allow users
to disable srq even if they have a piece of hardware that supports it.
Change-Id: Ia3be8e8c8e8463964e6ff1c02b07afbf4c3cc8f7
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452271
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Just a couple minor clarifications.
Change-Id: I6c368d263296f742d5bfb0df431d3bf40c800c6c
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452270
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
iSCSI is currently being adapted to SPDK's new threading model,
and this parameter doesn't make sense anymore. Remove it for
now until something functionally equivalent is added later.
Change-Id: Ia0e2f5aa81b72d99467c5a900619fbeeb42b2069
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452779
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
The product_name for raid bdevs was "Pooled Device" which was a legacy
naming convention that hadn't been cleaned up yet. This changes the
naming to be up to date with the current naming scheme.
Signed-off-by: Mike Carlin <mikefcarlin@protonmail.com>
Change-Id: I9092a2b793e48bb9ec0349087a31fdcde17ed9cd
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452269
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Users can now pipe a large number of requests to
rpc.py, separated by newlines - each line will be
executed as if it was passed to rpc.py individually.
There is significant savings using this feature when
executing multiple requests through rpc.py. On my
system, a loop of 30 RPCs related to setting up
10 NVMf subsystems takes 5 seconds when executed
one at a time. With this new feature, it takes
less than 1 second.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iec957ca67461af8e8c41aee47e1d113714b22d3d
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452770
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
This makes no functional change to the script, but
it prepares for an upcoming change allowing callers
to pipe a newline delimited list of rpc calls.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I695ca61510f9edd336713e88d134e42c1d78a1b4
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452769
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
We decided to name compress bdevs with COMP_ and then the name of
the underlying bdev so the RPC parm name was stale. Previously
hardcoded the path to PM for early dev so replaced name with pm_path.
Note that a sample create looks like this now:
rpc.py construct_compress_bdev -b NVMe0n1 -p ~/pm_files -d compress_isal
And that devs need to pay attention for the pathname they provide, its
easy to leave orphaned pm files around and they can get big.
Change-Id: Ifb5245c922461bbecec4bef266bdeb25e8b87f31
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452235
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This script is not general purpose like gen_nvme.sh,
so move it to the test directory at least for now.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I22df9375a8fcd585884523f22dae8ba9354768d3
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/451785
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
This will allow us to test a greater number of connections without
having to add more subsystems when doing NVMe-oF testing.
Change-Id: I33203d6db79b30abb065f098c16840096478c5de
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/451677
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Without this, spdkcli crashes when refreshing if any iSCSI
connections are established.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I82cd7a98e8c7d48e4730a7bd228ed483bdac28ef
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/451060
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
It seems that need to add help2man in pkgdep.sh , it was used in ISA-L module.
Change-Id: I9d69e2cba270b1802c333941160ca714b85c21d2
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/450562
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Add missing parameter numjobs. It is used in one of the cases.
Remove unnecessary arguments:
- cpumask: we do not want to restrict that
- percentile list: we do not want to restrict that, all
percentule buckets can be generated.
Change-Id: I6a4431af0defe92920f16d26091d67161e56dc5a
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/450519
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
For future NVMeOF benchmarking scripts must support TCP layer.
This is a recent feature so also added support of using custom
built nvme-cli.
Example config file was updated with appropiate fields.
Rename rdma_ips to nic_ips, as NICs no longer need to be
RDMA-enabled for this test.
Change-Id: I368b48eca08a30b4221f198cb76134b8836dbba9
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449879
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Enhance RPC method start_nbd_disk to take nbd_device as
one optional parameter. If it is not assigned, automaticly
choose an available nbd device path from /dev/nbd0 to
/dev/nbdN.
For github issue #324:
https://github.com/spdk/spdk/issues/324
Change-Id: I72c064d8bd476df342f5aa0af4d6120eb021c7ed
Signed-off-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/440453
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
User should know the request ID that was issued.
Change-Id: If1191f73b31ebe04f23a88a9ca31fa157474056b
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449315
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Client might wan't to handle error response on it's own. To keep
compatibility with legacy code we need to move exception rising to call
method.
Change-Id: I8a07e7556424e87ee10fa9332100a9a10abe05d8
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449314
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
The changes I made to the doc directory in spdk.github.io got
overwritten.
Change-Id: I468c7eeb8c36ccd969909e179c216d5def812602
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449360
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Paweł Niedźwiecki <pawelx.niedzwiecki@intel.com>
Defer the initialization when cache bdev mentioned in the config doesn't
exist when the FTL bdev is being created.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I084502ae38c9ae5e385e5566550ec3cb8b6f9d2e
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/448630
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Added means to configure libftl to use (optionally) another bdev as
persistent write buffer cache.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I97028a681be168d9386eac8a226631ff772f803b
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/448629
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Wrong indent level caused results to be duplicated in final
output file if configuration file was used to run more than one
workload combination.
Change-Id: I0fadc10eba38898df648ec2da5fbd0440a3b9693
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449150
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Just the starting point, base functions and structs.
Change-Id: I2ff2b672aa675a19b89e4449f1cc4aa664007f6f
Signed-off-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/435747
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
When testing on VM using a 10G .img file is problematic
due to size - sometimes there is not enough space.
Decreasing the size should improve this issue.
Change-Id: I7be91aaa56a4e7f60cfdffc67f85ae8b3f42b650
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449167
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Extend existing get_ocf_bdevs call to make it easier
to get information about attached cores.
Implementation is based adding additional optional argument "name"
to existing call. Based on "name" bdevs are filtered.
Backward compatability of RPC interface is preserved.
This patch also adds tests for the case when name is given.
Change-Id: I4300ebe37e936bc5cca8e066b5f09db462a87cf7
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/444841
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
scripts/fio.py and test/nvmf/fio/nvmf_fio.py are almost exact
same script. This commit unifies two files into one.
Change-Id: If753baaeb9f92dad2cda27bb4bed78ade4827d0b
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/448656
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Check if the UUID in the configuration is null (all zeroes) and treat it
as if no UUID has been supplied.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ic91c77591528a8aaa4cf5c0241e6bde51b3757f1
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/448628
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
On non NUMA architectures the numa_node sysfs entry doesn't exist.
Check if it exists before reading to prevent an error causing 'status'
to fail.
Reviewed-by: Qingmin Liu <qingmin.liu@broadcom.com>
Tested-by: Jonathan Richardson <jonathan.richardson@broadcom.com>
Reviewed-by: Jonathan Richardson <jonathan.richardson@broadcom.com>
Reviewed-by: Scott Branden <scott.branden@broadcom.com>
Signed-off-by: Jonathan Richardson <jonathan.richardson@broadcom.com>
Change-Id: Iacf5b062a4f5db5e049d2cfcff7c991d4ac5d3c3
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/448576
Reviewed-by: Scott Branden <sbranden@gmail.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Qingmin Liu <calmarrow@gmail.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
The python script for add_portal_group had required each portal is
an argument and portal list are made of remaining all arguments.
However this prevents adding new parameters to add_portal_group.
This patch changes portal list to be a single argument for python,
and each portal is separated by a whitespace in the argument.
Currently there will be very rare that a portal group has multiple
portals. Hence this change will not break any compatibility.
Updating CHANGELOG will be done together with other updates later.
Change-Id: I8c597a1194fff338afb371e6d24f9273122b753c
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/446379
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
For NVMe devices, in conjunction with the new batching options,
it can be advantageous to artificially delay between polling
for completions. Add an option to slow this rate down.
Change-Id: I0fc92709ff45ead0beb388dda60694bf1ed8b258
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/447716
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
The JSONRPCClient class now support the with statement. From now on the
__del__() method is changed to close() and user need to call it manually
at object disposal or use the 'with' Python statement.
Change-Id: I74afd93d411b9596ab8f9523c54a4a7523eec5e5
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/444686
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Pawel Kaminski <pawelx.kaminski@intel.com>
This will allow testing notifications code where user will most likely
issue many requests and socket buffereing might just squash them into
one packet.
Note that multiple requests are not send in batch mode.
Change-Id: Icfa4bfe21ff1268796bc41cf0443737f260b6040
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/436529
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Add extra long argument '--verbose <VERBOSE LEVEL>' to add more log
levels Default is error. Legacy'-v' parameter is translated into INFO
log level.
Some parts of code are logged using DEBUG level.
Change-Id: I2aaa6e2fbaf9b2101c6eeaec0cef6141636f135b
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/436528
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Switch to raw_decode in recv() function. This allow to decode JSON
objects one by one. Decoded part is removed from from receive bufer.
Change-Id: Id0d78a2ace85bcbb9cc8e30d72da6c1c2cad753c
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/435507
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Pawel Kaminski <pawelx.kaminski@intel.com>
OpenSUSE releases (OpenSUSE Leap and Tumbleweed) now use
/etc/SUSE-brand than /etc/SuSE-release as SUSE identification.
According to this change, This commit intends to update
scripts/pkgdep so that it could install packages for OpenSUSE.
Tested on OpenSUSE Leap 15.0 and latest Tumblweed.
Change-Id: I878b6671753084ef718e1f7630a42520a72ea151
Signed-off-by: Zhu Lingshan <lingshan.zhu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/446504
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This RPC was deprecated a couple of releases ago.
bdev modules now each have their own RPC for deleting
bdevs. Due to how bdevs are created differently on
different modules, it is simply not possible to
have one delete_bdev RPC that would work for all bdev
types.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ia46c95dce6e35f7557e8d41c41b8fea382924547
Reviewed-on: https://review.gerrithub.io/c/442615
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Default 'unmap' option stays as it was.
'Write_zeroes' comes useful when one wants to make sure
that data presented from lvol bdevs on initial creation presents 0's.
'None' will be used for performance tests,
when whole device is preconditioned before creating lvol store.
Instead of performing preconditioning on each lvol bdev after its creation.
Change-Id: Ic5a5985e42a84f038a882bbe6f881624ae96242c
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/442881
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This utility used to be called "bdevtest" so that's
what opts.name was set to for spdk_app_opts_init().
Change it to the new name "bdevio".
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Id8e6614c6fe82651952f46ec56de14ef0a8961f0
Reviewed-on: https://review.gerrithub.io/c/445800
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
When called from scripts, sometimes the arguments passed
will be empty.
Change-Id: I2f9fa1daa173eecc7b20928cd359284bb6f74ff3
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/444278
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
This function should be changed as well but was overlooked.
Fixing this here.
Change-Id: I2f7c8e475d89cfe177441e1843d3713397f6fe7b
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/444079
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
A new module switch which was missed at here.
Change-Id: If1784ace13657756d8034cd04e594af5b1799381
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Reviewed-on: https://review.gerrithub.io/c/444820
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
pycodestyle is the new style checker for python. We should install it as
available on all distributions.
Change-Id: I399f84a7027c0d7b8e9974fbc3be86c3c4beefae
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/444447
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
On some platforms, especially AWS, uio_pci_generic is not readily
available, and we don't have access to an IOMMU, so using the DPDK
igb_uio driver is a valid workaround.
Change-Id: Ic96776f925d9bbbcab625f5adb7642ca6bd4033d
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/444122
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
pycodestyle correctly detects that there isn't
two line breaks after the end of the class here.
The test pool pep8 does not catch this. We need
to move the test pool to the newer pycodestyle
tool.
Change-Id: I5fa8b66f8c3d76e41140e79fdcd264d7b38f86c7
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/444445
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
The current state for the setup script is that the vfio kernel driver
will be preferred against the uio_pci_generic driver only if an IOMMU is
present in the system. This is checked by looking for any IOMMU groups
in sysfs.
In case of vfio no-IOMMU driver, there are no IOMMU groups when loading
the vfio modules. The IOMMU groups are created when the PCI devices get
bound to the vfio-pci driver. Thus, even though the vfio driver is
loaded, the setup script will prefer to use the uio_pci_generic kernel
driver. This patch changes this behavior.
In order to support vfio no-IOMMU mode, the setup script will be
checking if the vfio module is loaded with parameter
"enable_unsafe_noiommu_mode". In other case, it will be falling back to
the uio_pci_generic driver.
Change-Id: I1e8317bc4e3d6af4ba8a9e0c51175c9f4190f47b
Signed-off-by: Nikos Dragazis <ndragazis@arrikto.com>
Reviewed-on: https://review.gerrithub.io/c/441062
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Python3 (3.6.7) gives W605 errors in regular expressions, causing the
SPDK style checks to fail on the Python scripts doing the style checking.
This fix allows these Python scripts to run without errors.
This is a known issue - see https://github.com/PyCQA/pycodestyle/issues/814
Change-Id: I71cdff5d6c89e19b200c989f3d9da35bb4f7189d
Signed-off-by: James Bergsten <jamesx.bergsten@intel.com>
Reviewed-on: https://review.gerrithub.io/c/443955
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Add prchk_reftag and prchk_guard to construct_nvme_bdev RPC.
In spdk_rpc_construct_nvme_bdev, create prchk_flags based on them
and pass it to spdk_bdev_nvme_create, and in spdk_bdev_nvme_create,
pass it to create_ctrlr.
A single option enable_prchk may be enough but add separate options
for reftag and guard to clarify that apptag is not supported yet.
The next patch will make per-controller PRCHK options configurable
by .INI config file.
Change-Id: I370ebbe984ee83d133b7f50bdc648ea746c8d42d
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/c/443833
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
strip_size as an rpc param is now deprecated and can be removed
in a future release. Either strip_size or strip_size_kb can be
used but only one of them or the rpc will fail.
Internally we maintain both fields because strip size always
comes in as KB but we convert it to blocks so having both elements
makes it clear for developers what they're looking at.
JSON output includes both strip_size and strip_size_kb.
Fixes#550
Change-Id: I5dc51e8af22eae3d56af8f8d37a564dbaae228fa
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/437873
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
iter_pci_dev_id abd iter_pci_dev_id functions should
not return BDF for devices that are not ment to be used
in tests.
Note that not all tests are ready for this change as they
discover functions on its own. Lets this changed in
separate patch.
Change-Id: I45a59ec121aa81e9f981acae7ec0379ff68e520a
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/443767
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Change-Id: I93f7e4e7d76f2fac4174050b249329993f28f71f
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/442095
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paweł Niedźwiecki <pawelx.niedzwiecki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Add PCI blacklist so we can skip only some devices.
Change-Id: I8600307dd53f32acb4dfeb3f57845e0b9d29fdb9
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/442977
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Bash interprets everything after command as additional
function arguments. To not confuse user just remove this part
and replace by '!'.
Change-Id: I44228003a1f96324271e726df4f5033f3258523c
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/442976
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Introduced a new variable to run functional tests.
It's enabled by default, and can be manually disabled
on systems where e.g. only unit tests are run.
SPDK_RUN_FUNCTIONAL_TEST is a supplement to SPDK_UNITTEST.
The two are completely independent - both can be enabled,
disabled, or run in any combination.
The new variable is prefixed SPDK_RUN_ as it aligns nicely
with SPDK_RUN_CHECK_FORMAT, SPDK_RUN_VALGRIND, and
SPDK_RUN_ASAN, all of which control how much is tested.
SPDK_UNITTEST should eventually follow the same pattern
as well.
This gives us 2 layers of configuration:
SPDK_TEST_* <- what is tested
SPDK_RUN_* <- how it is tested
The following would run UT+ASAN for FTL and BlobFS, without
running their functional tests:
```
SPDK_RUN_FUNCTIONAL_TEST=0
SPDK_RUN_ASAN=1
SPDK_TEST_UNITTEST=1
SPDK_TEST_FTL=1
SPDK_TEST_BLOBFS=1
```
Change-Id: I9e592fa41aa2df8e246eca2bb9161b6da6832130
Signed-off-by: Seth Howell <seth.howell@intel.com>
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/442327
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
I think this wording is slightly more clear.
Change-Id: I9fc500b8b87c0520bdf96e6c5db490bb98121da3
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/442435
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
This was marked deprecated in the v18.10 release, so
remove it now before v19.01 is tagged.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I57673a5ab475b97c812bebcefd77ff90d9305d1c
Reviewed-on: https://review.gerrithub.io/c/442412
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
- Add option to throttle iops in VMs using cgroups
- Add option to measure CPU utilization in VMs using SAR
- Add option to limit kernel vhost CPU cores (not NUMA optimized)
- Add option to do lvol preconditioning using fio bdev plugin
before running IO performance test
Change-Id: I7e0fcf977be96ecf837385c2abc9d5dabbe2f8c5
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/434229
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Paweł Niedźwiecki <pawelx.niedzwiecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
We can use spdk_strtol and spdk_stroll instead, which do a better job of
catching errors.
Change-Id: I09ff75356b932366b6c10cd6953610f609ce4b0e
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/441984
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
This patch introduces functional tests for FTL bdev.
The tests cover various I/O workflows and check data integrity. Several
scripts have been added to test the FTL library:
* generate_config.sh - prepares configuration scripts for specified
device
* restore.sh - tests restoring device's state from the SSD
* fio.sh - runs tests based on fio and fio_plugin
The tests are run from autotest.sh when the SPDK_TEST_BDEV_FTL flag is
set.
Change-Id: I561d99ed35fe91eadd3756789cc99afe2da8c1db
Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/431330
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
In SPDK, we will build isa-l with no shared option
and then integrate it into SPDK. And we do not need
to install isal in the system libaries.
Note: ocf build in autobuild.sh now needs to build
include/spdk/config.h before building the ocf library,
to ensure that header is available in a clean build
environment.
Change-Id: I3f0ce6932b386de17a77cf5bfdfd738b22417e2d
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Signed-off-by: paul luse <paul.e.luse@intel.com>
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Reviewed-on: https://review.gerrithub.io/c/441279
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Chunyang Hui <chunyang.hui@intel.com>
Marked as deprecated in 18.10.
Change-Id: I40d0e6103623aee6e6a0b9fa6e82f7b826ca1fe6
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/442420
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
This was marked deprecated in 18.10
Change-Id: Id47e770b0388c935fe684aeef7a9824f24cef47f
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/442416
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Add new RPC method for OCF bdev: get_ocf_bdevs
It is useful in respect to not registered OCF bdevs
which do not appear in standard get_bdevs call
Change-Id: I8a5fc86a880b04c47d5f139aa5fa4d07ca39c853
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.gerrithub.io/c/441655
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This patch adds FTL bdev. RPC scripts have been updated to allow for
creation and removal of FTL bdevs.
Change-Id: I82a5c5033b65bbeb67c238cae969a68cff767dcc
Signed-off-by: Jakub Radtke <jakub.radtke@intel.com>
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/431329
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This patch adds RPC calls for histograms in bdev layer.
Following calls are added:
- enable_bdev_histogram - enable/disable histogram structures for specified bdev and each of its channels.
- get_bdev_histogram - merges histograms from all channnels and encodes histogram as base64
Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Change-Id: Ib423a919dc1cde7dd7d92247db5482cfb9d66956
Reviewed-on: https://review.gerrithub.io/c/433573
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: wuzhouhui <wuzhouhui@kingsoft.com>
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Some users require to do write zeroes operation when
erasing data on lvol. Currently the default method is
unmap. This patch adds flag to spdk_rpc_construct_lvol_bdev
call that changes default erase method. This is also a base
implementation for possible future function for erasing
data on lvol bdev.
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I8964f170b13c2268fe3c18104f7956c32be96040
Reviewed-on: https://review.gerrithub.io/c/441527
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
This patch adds the RPC support for the Read/Write separate
bandwidth limit controls. The basic usage as following:
usage:
rpc.py set_bdev_qos_limit [-h] [--rw_ios_per_sec RW_IOS_PER_SEC]
[--rw_mbytes_per_sec RW_MBYTES_PER_SEC]
[--r_mbytes_per_sec R_MBYTES_PER_SEC]
[--w_mbytes_per_sec W_MBYTES_PER_SEC]
name
positional arguments:
name Blockdev name to set QoS. Example: Malloc0
optional arguments:
-h, --help show this help message and exit
--rw_ios_per_sec RW_IOS_PER_SEC
R/W IOs per second limit (>=10000, example: 20000).
0 means unlimited.
--rw_mbytes_per_sec RW_MBYTES_PER_SEC
R/W megabytes per second limit (>=10, example: 100).
0 means unlimited.
--r_mbytes_per_sec R_MBYTES_PER_SEC
Read megabytes per second limit (>=10, example: 100).
0 means unlimited.
--w_mbytes_per_sec W_MBYTES_PER_SEC
Write megabytes per second limit (>=10, example: 100).
0 means unlimited.
Change-Id: I822ec4814d21adff9826ce03a6af3783b1b98f44
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.gerrithub.io/c/417650
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Added set_read_only_lvol_bdev() RPC that marks
lvol bdev as read only.
This enables to create clones off of a base lvol,
without having to create additional lvol bdev with snapshot.
Change-Id: Ic20bbcd8fbebcef157acce44bfa1da035b0da459
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/440534
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
In a couple of cases, we do specify --ignore-submodules
for the actual check, but when dumping the results
to the console, we omit the --ignore-submodules. Fix that.
Same for check_format.sh - don't consider submodules for
files changes that suggest a CHANGELOG.md update.
This is in preparation for adding isa-l as an SPDK
submodule. isa-l doesn't use any .gitignore files, so I
want to make sure we're just ignoring anything related to
isa-l build artifacts. We could probably remove our
local DPDK submodule patch to its .gitignore after this
patch but will leave that for another time.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Id9be9ce87569004c426d02c6cd44d645f3ff859f
Reviewed-on: https://review.gerrithub.io/c/440808
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
This allows us to remove the requirement to install intel-ipsec-mb to
system directories.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I579655a98b515cf148b7cd17823a9bb541ea6ad7
Reviewed-on: https://review.gerrithub.io/c/440785
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Disks with NVMe driver were probably not whitelisted before.
Do not add them to NVMe subsystem configuration.
Change-Id: I9418590f5562a96750685d101323b60e56cc90cb
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/438426
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Modifed scripts/detect_cc.sh to take additional parameter specifing
the linker to use. Default to LLD on FreeBSD systems.
Change-Id: Idf97e9676a144028c0803d272ae6f0e903b0dd1f
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/438801
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Private network DHCP would put the wrong nameserver in the /etc/resolv.conf
file. This would break apt-get or anything that needed network access.
Removing that line from the Vagrantfile caused this to work.
Change-Id: I66dab9ddf72018a8e39e8e24d0b8175173ddb8f1
Signed-off-by: Lavar Askew <open.hyperion@gmail.com>
Reviewed-on: https://review.gerrithub.io/c/434392
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Previously, we allocate the buffer size according
to the MaxQueueDepth info, however this is not exactly
a good way for customers to configure, we should provided
a shared buffer number configuration for the transport.
Change-Id: Ic6ff83076a65e77ec7376688ffb3737fd899057c
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/437450
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
At least one is not included in the 16.04 distro that we use
in the TP.
Change-Id: Ia441dbe1ddeb140313da5e61009ab84ea8e80ffd
Signed-off-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/436781
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Add examples for installing guide.
Before running a vagrant virtual machine or script, we need this.
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Change-Id: I2f7ad8addf4300e75266d2b6fdf1b065eac0bb7c
Reviewed-on: https://review.gerrithub.io/436664
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Liang Yan <liang.z.yan@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
The original mechanism to identify files to cleanup
relied on glob matching from the output of 'echo'
piped to a grep. This yields a case where some
objects can appear and picked up as matching because
other items on the same line matched the grep
string. Changing this to use 'ls -1' which will
restrict the grep string matching to individual lines
thereby only picking up the entities the script
intended to do.
Change-Id: I020c80236fa68bcabeca0299fe7a27f3320de97b
Signed-off-by: Lance Hartmann <lance.hartmann@oracle.com>
Reviewed-on: https://review.gerrithub.io/437380
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Now trace facility can be enabled/disabled at runtime,
besides app starting parameters
Change-Id: I086564665ba4375b946a5339d5921d6584e9bfda
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/435470
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Skip pip and just use apt-get install for configshell
and pexpect. While here, don't try to install these
on older Ubuntu systems, since they aren't available
there. We know they're available on 16.04, so skip
it for anything older than that.
Fixes issue #532.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I40c1659333cd80b54fb3e01ed036294aa5bf6ca3
Reviewed-on: https://review.gerrithub.io/436231
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Use nvmf_tgt for nvmf tests.
Use iscsi_tgt for spdkcli iscsi tests.
Use vhost target for spdkcli vhost tests.
Fixes#511.
Change-Id: I5f084bccf87d32b0ede210c62ccc4a2f7926d716
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.gerrithub.io/435139
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Add SPDK_TEST_CRYPTO to switch crpto case which is
the same as other modules in spdk.
Change-Id: If1b99fe1d409437c538be9d09603b059276b7e74
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Reviewed-on: https://review.gerrithub.io/436276
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
For ubuntu18 and 16 logic added to only download sjohnsen boxes if
provider is libvirt.
Also changed PROVIDER to SPDK_VAGRANT_PROVIDER and initialized to
virtualbox. Now the directory of the default provider should
read unbuntu16-virtualbox as opposed to ubuntu16-.
Change-Id: I6cac03432b02cfd259759d504ec133d29d1f740e
Signed-off-by: Lavar Askew <open.hyperion@gmail.com>
Reviewed-on: https://review.gerrithub.io/434599
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
When the caller of the RPC API has all the necessary information about
how to access a Ceph cluster, then having to create configuration
files before calling the RPC API is problematic (has to touch files
owned by a local admin, changes must be removed again).
But having to encode support for certain configuration options in SPDK
is also problematic, because that might change depending on the
librados version.
The approach taken here is to merely pass through arbitrary key/value
config options. Existing config files are ignored when that happens.
The caller of the RPC then has full control over the connection setup
and can be sure that he does not inherit settings from a local file
accidentally.
In addition, user management is supported now, with or without a
config. This is useful for accessing a volume with a less privileged
user. Previously, passing NULL to rados_create implicitly chose the
"admin" user.
Change-Id: I5e7f36092df663a3d7ac503c04fc624a8fe1208e
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Reviewed-on: https://review.gerrithub.io/430460
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
check_format.sh isn't really function when astyle is too old - it
reports a bunch of style errors that don't really exist. So just
skip running astyle in these cases.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ifee769a296791d11b1ec693d398922e87d76bb69
Reviewed-on: https://review.gerrithub.io/435960
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
This allows sending multiple requests without waiting for response in
next patches.
Change-Id: I1a54313fbf6b18e1887febc7648e5b9cd8cf06ff
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/435475
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This decorator is not needed. This also removes one exit point in case
of exception.
Change-Id: I7e1492cf9cccdb891dd70494652e0bc86a8bbcca
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/435469
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
We will keep the RPCs for now but mark them deprecated.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I0407dcb392ea0c9e89c0f26cd5670aed2dbfadef
Reviewed-on: https://review.gerrithub.io/435345
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Subsystems should be started anyway when empty configuration is
provided.
Change-Id: Iabc85d319b11d19be7ec182d1f6fa4f40eacf7e2
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/432552
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Not all bdevs have uuid attribute.
Fixes#510.
Change-Id: Ie3055563c8b88c002b5af45eb528ffbef5ec2166
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.gerrithub.io/435466
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paweł Niedźwiecki <pawelx.niedzwiecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Grepping `mount` output for /dev/sdX does not guarantee
the device has no mountpoints. If /dev/sdX happens to
have an LVM built on top, then it could be
/dev/mapper/something that appears in `mount`.
Fix this by checking mountpoints of /dev/sdX and all its
children as reported by `lsblk`.
Change-Id: Id2fa6939645584d009bc87c7341a97f6948ebde9
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/434209
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Just return if the [AIO] section is empty.
Change-Id: I90c42afdca3af7969a1e9d500f92820892857127
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/433199
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
IOAT copy engine supports JSON RPC but doesn't support JSON config
dump yet. Hence config_converter fails conversion if the source
.INI config file includes the [Ioat] section.
Skip the [Ioat] section for now.
Change-Id: I7331db582917bf01e92f6251c75f2f5a36ab9382
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/433198
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
This is necessary to enable multiple VMs to run in the same physical
system since they each need a unique backing nvme drive.
Change-Id: Ie2d3aa9604533b4d39932e1f68cb977bafdb7ab4
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/432202
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
A simplification for machines that support multiple providers.
Change-Id: Iaa3c6eded84f8d1e750ccace49c3f85f54d12df7
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/432201
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Also make ipsec configuration and install optional
in pkgdep.sh.
Change-Id: I0c973398cebce0f50c704b21f6397e8d996ba9f9
Signed-off-by: Ed rodriguez <edwinr@netapp.com>
Signed-off-by: John Meneghini <johnm@netapp.com>
Reviewed-on: https://review.gerrithub.io/432424
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
The PCI devices used for SPDK are bound with vfio-pci or
uio_pci_generic kernel drivers. In setup.sh, if the path /sys/kernel
/iommu_groups is not empty, vfio-pci kernel driver is the only choice;
otherwise uio_pci_generic is selected.
In system, IOMMU can be enabled but set to pass through. It means
IOMMU will not affect the DMA transmission although IOMMU groups has
been configured. In this case both two kernel drivers are workable. The
script cannot deal with the case now.
The new option DRIVER_OVERRIDE is introduced in the patch and allow
user selects the kernel driver for PCI devices. With the patch the above
case can be handled correctly.
Change-Id: I540d8750bf837ce67b8bc8b516a1a3acb72c502c
Signed-off-by: tone.zhang <tone.zhang@arm.com>
Reviewed-on: https://review.gerrithub.io/427297
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This option is deprecated. Also, rename the rpc and configuration
options for setting the opts to reflect that they now only set the max
number of subsystems
Change-Id: Iaabcbf33dd0a0dc489d81233fda74e9e7f3e0d2e
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/430161
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
--with-reduce required to build reduce. This depends on
libpmem being installed.
We still need to work out details in pkgdep.sh and
vm_setup.sh. Some distributions like Ubuntu still
require configuring extra package repositories to
get libpmem packages.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I4e056ce1da9a1fecb4458f8f5e7ff5d61c422533
Reviewed-on: https://review.gerrithub.io/430646
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
And change names of parms to be consistent with others used in
RPC testing.
Change-Id: I8331c6a22866d89a2a4ffb5fc8d41d74b4b7b07d
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/428724
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Just considering the situation of multi-namespaces device.
Change-Id: I1b0d502874ab53891060c973a3ae75e81ec591f2
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/431698
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
With this change on 48 core system the checking format takes ~4.5s which
is acceptable execution time.
Change-Id: Icc97466c59a7a1072fcbd705203fc8ee0dba158e
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/431355
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
There are a couple of runtime errors in spdkcli that prevent it from
properly loading. One is the fact that it doesn't properly check to make
sure that specific subsystems are enabled before trying to load them.
The other is an apparently uninitialized variable in the iscsi object.
Change-Id: I458c6da71b60697b19c924308ac0e33a12f7a8a6
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/430638
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This patch introduces script that may be used for
sending RPC commands from remote machine. This script
requires remote user to authenticate with user name
and password set at launch time.
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I4ef7c870f3ea760c44da63ba4c16954aa54d4473
Reviewed-on: https://review.gerrithub.io/426531
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
"get_nvme_controllers" can be used to list NVMe controllers.
Names and transport IDs of the NVMe controllers will be
reported by this method.
Change-Id: Ie59b567afc09e70475f97939e86a872af39c5d8a
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/419094
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: GangCao <gang.cao@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Also, remove bdev_svc memory limit to prevent it from hanging while
handling discover_bdevs calls.
Change-Id: I2868839f40931b964d49a90a86dfb391e2247e30
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/429535
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
A recent patch changed bdevperf app name from 'bdevtest' to 'bdevperf'.
Change-Id: I6f89cb20560b819f8c451b34ee5afbf5fca68939
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/430343
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
This script should be called when testing the crypto module. It will
configure any qat cards with class code 37c8 for use with the spdk
crypto bdev module.
Change-Id: I233069fbe9ba3c7a6463f52443234f9fe2c9a1f7
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/428549
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
An example to send nvme_cmd rpc request:
scripts/rpc.py send_nvme_cmd -n Nvme0 -t admin -r c2h \
-c <base64_urlsafe encoded cmdbuf> -D 4096
nvme-rpc will be processed internally by bdev_nvme.
Change-Id: I6e731b76be0f503d48154a8b34a1e81b4b454396
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/417962
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
nvme-cli is updated to use mk/config.mk so we can now remove
CONFIG.local symlink.
Change-Id: I7d39c2559e43198b9e0bc4519edaa1cb3180de07
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/428373
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Everything we need is in CONFIG.local - so just use that
file. This allows us to remove the for loop as well.
CONFIG.local is currently a symlink to mk/config.mk - that
will be changed in the patch that removes the symlink.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ia7f84eac85d322f2873f089ab62c99a69c4d9ef9
Reviewed-on: https://review.gerrithub.io/429947
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: GangCao <gang.cao@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
This leaves files created by the root user in the directory and
makes future calls to make clean fail.
Change-Id: Ie33d0d33e8c01a2d17f6991284c5118b5bd545ff
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/429282
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Due to change of defauly Python interpreter to Python3
we need to decode byte object from check_output()
to utf-8, otherwise there is an error.
Change-Id: I83e2d79ec8c3934c5c6d00768288fbb4c5a50914
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/428172
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
If the ipsec submodule is registered to spdk, an empty intel-ipsec-mb
directory will be created. We could potentially try to run make inside
of this empty directory, so instead do a preemptive submodule update.
Change-Id: I367fdef468bf21ef91b8354155d199cea97c3daa
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/428404
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This patch added the support of RPC method to enable,
adjust, disable the bandwidth rate limit on the bdev.
And it can work together with the existing IOPS rate limit.
The RPC method has been consolidated to support both IOPS
and bandwidth rate limits as below:
usage:
rpc.py set_bdev_qos_limit [-h]
[--rw_ios_per_sec RW_IOS_PER_SEC]
[--rw_mbytes_per_sec RW_MBYTES_PER_SEC]
name
positional arguments:
name Blockdev name to set QoS. Example: Malloc0
optional arguments:
-h, --help show this help message and exit
--rw_ios_per_sec RW_IOS_PER_SEC
R/W IOs per second limit (>=10000, example: 20000).
0 means unlimited.
--rw_mbytes_per_sec RW_MBYTES_PER_SEC
R/W megabytes per second limit (>=10, example: 100).
0 means unlimited.
Change-Id: I9c03cd635280add01801a81c6a6c02f0cf85bee1
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.gerrithub.io/416511
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Some older versions of Fedora require putting the git grep contents
in quotes before feeding to the while loop.
Change-Id: I649d5520f4a8c0a903aed2e2dda337e91b5b596e
Signed-off-by: chenlo2x <longx.o.chen@intel.com>
Reviewed-on: https://review.gerrithub.io/428332
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Liang Yan <liang.z.yan@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Add create/delete methods from iscsi subsystem to spdkcli.
Change-Id: I950b3f29720fd5f09e4144d139e6d0e5885c69c8
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.gerrithub.io/424607
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Paweł Niedźwiecki <pawelx.niedzwiecki@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Patch 427539: Check file permissions in the check_format script
introduces at least two errors.
- exit too early from check_format.sh
- wrong file permisions on file vbdev_crypto.c
This patch fixes them.
Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Change-Id: I346523bce7448e749471bd78a06e9233e0d35ef2
Reviewed-on: https://review.gerrithub.io/428188
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
User can use this optional parameter to get the information of specific
vhost controller.
Change-Id: I3911c6c7d4e7b75e82277d1e4690d5e40019aa06
Signed-off-by: wuzhouhui <wuzhouhui@kingsoft.com>
Reviewed-on: https://review.gerrithub.io/425451
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Keeping consistent with the distributions we support in the build pool.
Change-Id: Iec13686507f890e9e18ecd5a1f4f238b2370f1d7
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/427161
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Allow the option to run vm_setup.sh during provisioning on a fedora
system. This is one step closer to fully automating the setup of a build
pool test machine.
Change-Id: Ia3965b31e0a9217d176ffe3c165b8eb6b3ccad13
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/426946
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
For some use cases (e.g. creating a test pool vm), we won't want to
rsync spdk from the host, and instead will want to clone a new one from
scratch.
Change-Id: I4c27f8ffc6c04aa0901dfe5b536b7e6ba94f7693
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/426945
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Override configshell constructor to allow command paramaters to use
whitespaces and quotes.
"This patch will be obsolete once
https://github.com/open-iscsi/configshell-fb/issues/46 is fixed."
Change-Id: Ia5f883991b031901be7dadf61cbe7ebdcf5a9cb7
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.gerrithub.io/426037
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Allow specifying size for created NVMe image file.
Default is 1024M but some tests suites (vhost_initator
for example) need more.
Change-Id: I4bf7a5241a0e9c2320cd473b1787c0be25d9a8e4
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/426068
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Enable users to automatically create VM image dependency
needed to run vhost tests.
Current state of Vagrantfile_vhost_vm is only valid
for Ubuntu 16.04 and 18.04.
Change-Id: I9cdf46c1db7fcd55cfda0dd7db7ff5570610c5ee
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/422948
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Most vbdevs have complete test suites that include rpc calls
to set them up and teare them down. With crypto there's not
a lot to do w/test other than make sure it's configured and
run IO so right now that amounts to tools that use a config
(ini) file. Adding create/delete calls here to get basic rpc
coverage in CI.
The fixes/udpates in the vbdev module and the .py files were
fixes found in enabling testing of the rpc calls.
Change-Id: I5d7d25e1c96dcf76e0055f23d27781b858e290b0
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/425875
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
The vagrant proxy plugin doesn't appear to support FreeBSD or pkg in the
same way that it supports linux distros and their associated package
managers. This change supplies the needed proxy configuration to get
around this issue.
Change-Id: I7dfbd963c5fd01d598c1456c5d8139be81777440
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/426422
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
If the user does not specify a serial number, just set it
to all 0.
Change-Id: Ie1aaa4c1bf5f9eaf76fcc9ed6f6bd6ded2aeb200
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/425324
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ziye Yang <optimistyzy@gmail.com>
rpmbuild pointed out that this is wrong to have executable files without
shebang. Taking the opportunity and fix this for other files too.
Change-Id: Ib21f436672150edc0aff511bff2eb6839870cf79
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/425382
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Add a new script to enable converting
old INI config files to the new JSON-RPC
config file format.
This prepares for deprecating the INI
config file file format in the future.
Change-Id: I0f2bfa9a585ce3537772b662d8e4028ab08a554d
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.gerrithub.io/420457
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Initial support for softare AESNI_MB DPDK driver only.
Have tested (both aesni and QAT seprately and concurrrently) on underlying NVMe devices
with bdevio and a bdevperf script that runs IOs from 512B to 128K each with Q depths from
1 to 512 in powers of 2 for 30 seconds each run.
QAT can be included in the code (but not makefile) and marked as experimental
until we are ready to test in CI. It works well on 2 systems but is a big PITA to get
the hardware setup and configured for use with DPDK (IMHO).
Change-Id: If518c3df8e74e00efa18afdf194824c5e69778fc
Signed-off-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/403107
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
- Add independent functions to create transport with specific opts
and add to target while maintaining backward compatibility with
current apps and rpc configuration that still use the add listener
method to create a transport.
- Add new rpc function to create transport and add to target.
+ Update json reporting to include new rpc function.
+ Update python scripts to support new rpc function.
+ New nvmf test script (cr_trprt.sh) to test new rpc function.
Change-Id: I12d0a42e34c9edff757755f18a78b722d5e1523e
Signed-off-by: John Barnard <john.barnard@broadcom.com>
Reviewed-on: https://review.gerrithub.io/423590
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This it to prepare for RPM package. Also lower number of dependencies
needed by SPDK tools.
Update changelog to deprecate Python 2 and explicit interpeter invoking
in scripts.
Change-Id: I2497cca721cbcbadc1c99c675f8b8b7f682d5efa
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/425233
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Add a new nvmf_subsystem_create RPC. The current RPC,
construct_nvmf_subsystem, is old and jumps through a lot
of hoops to maintain backward compatibility. Leave it in,
but create a new RPC as the main mechanism going forward.
Change-Id: Ia99cb5994a90579956136dac1800ed1fdce482dd
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/425169
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
This patch adds support for IP based load balancing. All the rdma connections
coming from a particular IP will be assigned the same core on target. The load
balancing policy - roundrobin/hostip can be configured through rpc during startup only.
Roundrobin is the default policy.
Change-Id: I7bb216cb4cb2639fa401e23090143c4cf1d7bb5a
Signed-off-by: Avinash M N <avinash.m.n@wdc.com>
Reviewed-on: https://review.gerrithub.io/422190
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
User can pipe or redirect data as needed. As features like
bdev crypto are added, storing config data in files needs to
be carefully scrutinized - don't make it easy by providing
ways to write data to files that may not be sufficiently
protected.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I24dd32f81b965e275e1bac875f6d008d1b68007a
Reviewed-on: https://review.gerrithub.io/425165
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Changed "/bin/sh" to "/usr/bin/env bash" in scripts/pkgdep.sh
Previously script failed on my Ubuntu server, with error:
trap: ERR: bad trap
Change-Id: I054c0388c462e2d0340b6e3d4e581a8e6cdc7097
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.gerrithub.io/424998
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
This is an artifact from before SPDK had a configure
script or a DPDK submodule. Make configure the
only supported way for specifying the location of the
DPDK installation to use with SPDK.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I5c197c46220928bb18b97c8807755967d76ea42c
Reviewed-on: https://review.gerrithub.io/424893
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
This RPC does not work for a lot of bdev types. For
example, NVMe namespaces and virtio scsi LUNs are not
explicitly constructed by an RPC - they are indirectly
constructed by an RPC associated with an NVMe controller
or virtio-scsi controller.
While here, remove spdk_bdev_config_json. It was
only created to facilitate this get_bdevs_config RPC.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I537166d8f91ab458bd2000859d74f7254bfc9c0a
Reviewed-on: https://review.gerrithub.io/424584
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
It is likely that bizarre errors occur if any of these commands fail
so don't continue but just exit with an error.
Change-Id: I007192f22c89f3d7f92b1f4a0cc31cb50b605dc3
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/424412
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This ensures the bdev.conf file is found.
Reported-by: Amir More (in GitHub issue #420)
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I309d8f211e0a3e0e4530c5d3c225a52b6ea68c2c
Reviewed-on: https://review.gerrithub.io/423931
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
CHange double brackets to single brackets for compatibility with
freebsd, also added -y to a pkgdep call to avoid the script hanging when
called programmatically.
Change-Id: I9a851005efae77b9146f14cac8079e2d448143d4
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/423936
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Add an new RPC to get current authentication group configuration.
This patch is utilized in the next patch to support JSON config dump
for authentication group configuration.
Change-Id: I34be9e196f8e7a484bcd316da54f05d0f6ee0300
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/421468
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Previous patches enabled users to configure CHAP secrets dynamically
by RPCs. Subsequent patches will enable users to load CHAP secrets
from JSON config file.
Loading CHAP secret file is done by default and this will conflict to
JSON config file.
Hence the path to CHAP secret file is required to specify in the config
file or JSON RPC set_iscsi_options explicitly after this patch.
Users who have used CHAP secret file are expected to specify it explicitly
and this will be no harm for them.
Besides, CHAP secret file is not oly for discovery sessions but also for
login to iSCSI targets. However there were wrong description to make user
misunderstand. Hence remove these wrong description in this patch too.
Change-Id: Ic4093cabc0c14b87e26baef4bba6b0d292e40c06
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/421467
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Add RPCs to add/delete a secret to/from an existing authentication group
dynamically.
Use mutex to ensure exclusive access to CHAP secrets.
Tries to use descriptive message in RPCs when error occurs.
Change-Id: I59650ae11a2fe675d03b90bbd4d2dc5b9c0160ed
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/421465
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This patch adds new RPCs add/delete_iscsi_auth_group to add and
delete authentication group dynamically.
Mutex is added to ensure exclusive access to CHAP secrets.
Additionally provide descriptive message in RPC when error occurs.
Change-Id: Iaddfbdd5688ca7907d2c7d859835faa056deecd1
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/421464
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
According to https://tools.ietf.org/html/rfc3720, the
default value for the FirstBurstLength is 65536 bytes
while SPDK iSCSI target picks the smaller 8192 as the
default setting. This value is the communication for
the iSCSI initiator to send the unsolicited data and
instead of having a fixed setting here, expose it as a
user configurable parameter to fit the real use case,
especially for the data out iSCSI write.
Example of usage as following in the iSCSI.conf:
FirstBurstLength 8192
Change-Id: I71690c7c48aa0875f1f975c0ea935389de6d1e6d
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.gerrithub.io/421142
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Although a pre-req to crypto only, the latest ipsec library now
requires root to install. This patch takes 2 approaches to making
sure the ipsec module is installed properly. First, in configure,
if crypto is enabled it will check to see if ipsec is already installed
and if so do nothing. If it is not it will exit and instruct the user
to manually install and re-run the configure script.
Secondly, to be proactive, this patch adds the installation, if not
already installed, to the pkg dependency script because it is already
executed as root.
It also does the same for nasm, checking the required versions in both
places.
Change-Id: I037629f21be474b88c12451733ac12ff0b1651f5
Signed-off-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/422746
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
--skip-alias is not supported in all versions of which.
While here, explicitly import CalledProcessError - some
versions of Python didn't seem to mind, but it's not
true for all versions. Also print related error
messages to stderr instead of stdout - this helps
with debugging since in some tests the stdout is
redirected to a file.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I7637adc36da6290ece28c77b3398185191347eef
Reviewed-on: https://review.gerrithub.io/423414
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Get the path of fio binary dynamically.
Change-Id: Ieb29bdfec33716828339a60e67dfe786e8ad278a
Signed-off-by: Chen Wang <chenx.wang@intel.com>
Reviewed-on: https://review.gerrithub.io/423057
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
An new RPC set_iscsi_target_node_auth enables users to configure CHAP
authentication for the target node dynamically.
New setting is used for new iSCSI sessions. For existing iSCSI sessions,
new setting is not used until user logout and login again.
Try to use descriptive message in the RPC when error occurs.
Change-Id: I0bd40d92262d708c1f7de0effb208078bdf8cc41
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/420974
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
An new RPC set_iscsi_discovery_auth enables users to configure
CHAP authentication for discovery sessions dynamically.
Try to use descriptive message in the RPC when error occurs.
Change-Id: I8883d5e054539cf6db091a5b79d0479e62119811
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/420976
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Current parameter names of `set_iscsi_options` and `get_iscsi_global_params`
RPC method for CHAP in discovery session may not be clear and long
compared with CHAP in login to target nodes.
This patch changes parameter names of `set_iscsi_options` and
`get_iscsi_global_params` RPC method for CHAP in discovery session
from no_discovery_auth, req_discovery_auth, req_discovery_auth_mutual,
and discovery_auth_group to disable_chap, require_chap, mutual_chap,
and chap_group, respectively.
Old parameters are still usable because decoder of them are not removed.
Change-Id: If3102f1233d57ee3cadfb733a6fc1fca14a0e972
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/423050
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Apparently not all distributions include it by default.
We use it a lot in setup.sh and test scripts, so add it
as a dependency.
Change-Id: I5dc5e7f01ff8b07a01487b61725a928e92f583d2
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/423052
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
SPDK deletes those files automatically for single-process
applications and the multi-process' ones can now be freed
with `./setup.sh cleanup`.
The script clears so called DPDK runtime directories, which
are defined as follows (for DPDK 18.05+):
* If DPDK is running as root, /var/run/dpdk/<per-pid-prefix>/
* If DPDK is not running as root:
* If $XDG_RUNTIME_DIR is set, ${XDG_RUNTIME_DIR}/dpdk/<per-pid-prefix>/
* Otherwise, /tmp/dpdk/<per-pid-prefix>/
Since DPDK 18.08, even the shared config file is placed in
the runtime dir, making it the only dir we have to clear.
Change-Id: I13f58a98f6a07e609b888b4e816dd45eac10f49c
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/422485
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Use libvirt provider to provision the VM by syncing vhost dir
which holds all dependencies needed to run vhost tests
(VM image, fio binary and SSH keys).
Change-Id: If8c65ef683cae37301e980fdaecc753c7371aeeb
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/421919
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
We have to sync spdk sources regardless of used provider.
Can also sync tools if available.
Change-Id: Ia149a68d632abadf69d20d7c34c6d7f158e2d7cb
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/422285
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This was most likely overlooked in the original implementation.
Change-Id: Ib3efbcf4a3abcbd6ad6bf9ced7280879b73f86bf
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/422081
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
pushd and popd not in default path for /bin/bash
Change-Id: I83e0bd1f87005e1c8542ac3db44b26f83eedf96c
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/421903
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Use tabs in all vagrant script files for indentation.
Change-Id: Iaf238b742b4958ea31f31ebe2d0f352944cb3d57
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/421511
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
At start of spdkcli and on each configuration call
check if spdk subsystems are initialized.
Disallow using get/create/delete commands if system is
not initialized.
Change-Id: I8b9e0362c31a5007e6460d4f79cf3f9e91f71054
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/419510
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Delete_bdev is depracated in favor of specialized delete_* calls.
Removing it's use from "/bdevs" node so that it's no longer
available.
Moving split related methods to split class while at it, because
that's where they belong.
Change-Id: I1ff8d8b74cd8c27a734bca3065a98679b8676428
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/421208
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Since delete_bdev should be used only for debug purpose,
this patch adds delete call specific for virtio bdev.
This is generic call for both - blk and scsi.
Scsi specific call has been removed.
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I3ed629942e9ef6067b488d94ca508b40e9e74f42
Reviewed-on: https://review.gerrithub.io/417008
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Since delete_bdev should be used only for debug purpose,
this patch adds delete call specific for NVMe bdev.
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ib9a0475d735af2616a3005d04530ca825ece8a52
Reviewed-on: https://review.gerrithub.io/416546
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
In python script:
- timeout should be nop_timeout
- min_connections_per_core was min_connections_per_session by mistake.
- max_queue_depth was not included.
- call_cmd was missing.
- A space should be added after semicolon ":"
In SPDK library:
- type of options was not int32_t but int. Hence decode failed.
- type of options in dump were not correct.
Change-Id: Iaa79236c03d368cdf286a72c2386509ffa81530f
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/419553
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Paweł Niedźwiecki <pawelx.niedzwiecki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
It is not an error if bdev is smaller than cluster size so convert this
error to infolog. This fixes false error message dring examine process.
To return proper error message when creating blobstore using RPC the
_spdk_bs_alloc was adjusted to return errocode that is propagated up and
converted to "No space left on device".
Fixes#316
Change-Id: Ic9803720a55125fcfa34263346f2d9e1aae03a53
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/420054
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
