Commit Graph

1114 Commits

Author SHA1 Message Date
Tomasz Zawadzki
99cff8350b test: remove SPDK_TEST_EVENT flag
Test in event.sh take 5 seconds on CI at this time.
For the tests to run both SPDK_RUN_FUNCTIONAL_TEST
and SPDK_TEST_EVENT had to be enabled.

This served as very minor time saver, so removing
this flag shouldn't affter test times more that the
5 seconds.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ie3eaa587daf28b2efbc3cfdff5ad648b7529e7f1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1493
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-03-31 07:57:40 +00:00
Michal Berger
6b8f9a0f81 test/bdev: Convert setup_rbd_conf() to RPC calls
Change-Id: Icfbb571d1cd2a2539cd9026f1437c75450b77512
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1022
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-03-31 07:55:58 +00:00
Karol Latecki
051a6902d8 rpc/client.py: replace use process_time() instead of time()
time.clock() was depreceted since Python 3.3 and was finally
removed in Python 3.8.
https://docs.python.org/3.8/whatsnew/3.8.html#api-and-feature-removals

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I3ffe372b61c8afc0ddbbee0064b95059fa184616
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1369
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-03-23 09:37:44 +00:00
Karol Latecki
4f24f2ca4a scripts/pkgdep: change archlinux pmdk installation dir
By default "make install" installs pmdk header files in
/usr/local/include while autotest scripts expect them
to be in /usr/include.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I906ffd8a7858a37525758c3fc8129cb22286a8f5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1368
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-03-23 09:37:17 +00:00
Maciej Szwed
3524dd0153 pkgdep: Add ncurses to pkgdep
ncurses library will be required by upcoming spdk_top app.

Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I16268c2ac772a45abdbd12d8a761d46041ce6e20

Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1247
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
2020-03-20 08:41:11 +00:00
Karol Latecki
2da5c24d8e scripts/pkgdep: change keyserver for archlinux rdma-core installation
Previously used keyserver keeps rejecting the gpg key.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: Ifed1d5c4db61f1b6d1e985fa048cc17801046f35
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1329
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-03-19 08:51:13 +00:00
Karol Latecki
9bf3be9f95 scripts/pkgdep: add archlinux dependency for pmdk
pkg-config was missing for pmdk installation.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I795c4b3a71d611292beaa650960afb639e827e78
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1328
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
2020-03-19 08:51:13 +00:00
Maciej Wawryk
b5700d4d7c scripts/vagrant: Fix unsupported machine type error
Vagrant fails during creation virtual machine using custom qemu emulator.
Setting machine type to "pc" resolved this problem.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I1822439bb2235b5881d3c331c378017928953ac4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1248
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-03-19 08:50:53 +00:00
Karol Latecki
db43b387ba test: do not use iter_pci_class_code in tests
Do not use iter_pci_class_code function in tests to
iterate over NVMe drives. This function can return
drives which at the moment of execution can not be
whitelisted for use.

This can result in test errors (such as simply
bdev_nvme_attach_controller RPC command failing) or
even using and deleting data from NVMe drive which
was not meant to be used in tests.

Fixes #1235

Change-Id: I82b9935fc88605b636c2096be6c71d4880a567c8
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1309
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-03-18 08:03:21 +00:00
Maciej Wawryk
0172f27974 perf/nvmf: Add more latency measurements
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I1ff5e6440ebef50d0488f020edfd8d5b7e274162
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1281
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-03-18 08:03:02 +00:00
Maciej Wawryk
3509fc12df Add cpus_allowed_policy param for NVMeOF benchmark tests
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I82a1632d4bb8742bd93f33a5a094a9cb4aa2e927
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1259
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2020-03-17 16:38:16 +00:00
Maciej Wawryk
0ebd0dce19 Add subsystem division in Kernel initiator
This will allow us to use num_cores and cpus_allowed option also
in Kernel initiator side.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Iad4e8bf7cd8a614ada8f443a8b1036dc2c4803e4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1173
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2020-03-16 08:45:50 +00:00
Maciej Wawryk
0eab351528 Add cpu_frequency param for NVMeOF benchmark tests
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Id594da091d7bcd5d23a752ccfab9bfa5b7726b49
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1113
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-03-16 08:45:50 +00:00
Maciej Wawryk
e068db83f1 Add cpus_allowed param for NVMeOF benchmark tests
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I0d113e39c1ae9ff284f408998f5bba82352a2891
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/993
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-03-16 08:45:50 +00:00
Liu Xiaodong
f2bfad5908 scripts/common.sh: print error msg to stderr
Error msg in common.sh should be printed into stderr
in order to avoid improper usage.

For example:
  If env has no lspci and pciconf, then function
iter_all_pci_class_code will return the error msg back,
every word in error msg will be used by caller wrongly.

Change-Id: I6a875e49527539ba82bd331c8878e972e26cbc39
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1227
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-03-12 09:03:33 +00:00
Darek Stojaczyk
8b98cdb64a scripts/rpc.py: add daemon mode
Add rpc_cmd() bash command that sends rpc command to an
rpc.py instance permanently running in background.
This makes sending RPC commands even 17 times faster.

We make use of bash coprocesses - a builtin bash feature
that allow starting background processes with stdin and
stdout connected to pipes. rpc.py will block trying to
read stdin, effectively being always "ready" to read
an RPC command.

The background rpc.py is started with a new --server flag
that's described as:

> Start listening on stdin, parse each line as a regular
> rpc.py execution and create a separate connection for each command.
> Each command's output ends with either **STATUS=0 if the
> command succeeded or **STATUS=1 if it failed.
> --server is meant to be used in conjunction with bash
> coproc, where stdin and stdout are named pipes and can be
> used as a faster way to send RPC commands.

As a part of this patch I'm attaching a sample test
that runs the following rpc commands first with the regular
rpc.py, then the new rpc_cmd() function.

```
time {
        bdevs=$($rpc bdev_get_bdevs)
        [ "$(jq length <<< "$bdevs")" == "0" ]

        malloc=$($rpc bdev_malloc_create 8 512)
        bdevs=$($rpc bdev_get_bdevs)
        [ "$(jq length <<< "$bdevs")" == "1" ]

        $rpc bdev_passthru_create -b "$malloc" -p Passthru0
        bdevs=$($rpc bdev_get_bdevs)
        [ "$(jq length <<< "$bdevs")" == "2" ]

        $rpc bdev_passthru_delete Passthru0
        $rpc bdev_malloc_delete $malloc
        bdevs=$($rpc bdev_get_bdevs)
        [ "$(jq length <<< "$bdevs")" == "0" ]
}
```

Regular rpc.py:
```
real    0m1.477s
user    0m1.289s
sys     0m0.139s
```

rpc_cmd():
```
real    0m0.085s
user    0m0.025s
sys     0m0.006s
```

autotest_common.sh will now spawn an rpc.py daemon if
it's not running yet, and it will offer rpc_cmd() function
to quickly send RPC commands. If the command is invalid or
SPDK returns with error, the bash function will return
a non-zero code and may trigger ERR trap just like a regular
rpc.py instance.

Pipes have major advantage over e.g. unix domain sockets - the pipes
will be automatically closed once the owner process exits.
This means we can create a named pipe in autotest_common.sh,
open it, then start rpc.py in background and never worry
about it again - it will be closed automatically once the
test exits. It doesn't even matter if the test is executed
manually in isolation, or as a part of the entire autotest.

(check_so_deps.sh needs to be modified not to wait for *all*
background processes to finish, but just the ones it started)

Change-Id: If0ded961b7fef3af3837b44532300dee8b5b4663
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/621
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-03-11 11:35:03 +00:00
Shuhei Matsumoto
49cffc1e68 rpc: Add thread_get_io_channels RPC
Add an new JSON RPC thread_get_io_channels to retrieve IO channels
of the threads. IO channel don't have its own name and so get the
name of the corresponding IO device instead.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Id8fbb328ffba09e1eace35994cdbf36b24832b82
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/887
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-03-10 09:13:02 +00:00
Shuhei Matsumoto
0b0f0c5415 rpc: Add thread_get_pollers RPC
Add an new JSON RPC thread_get_pollers to retrieve pollers of all
the threads. By adding a helper function spdk_poller_state_str(),
output poller state as string to improve readability. Most of the
code of thread_get_stats and thread_get_pollers are common and so
unify these two RPCs as possible as we can.

Sample output of thread_get_stats RPC in doc/jsonrpc.md was wrong
because thread_get_stats doesn't output ticks. Fix this together.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I003ffe569d3c0651ae65c5858eff8287f7e9031d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/604
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-03-06 10:29:56 +00:00
Maciej Szczepaniak
12642633bf bdev/ftl: Add ftl_path parameter to rpc parsing
Signed-off-by: Maciej Szczepaniak <maciej.szczepaniak@intel.com>
Change-Id: I51cc68536f162b0d920d135ba491b1657e699618
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/880
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-03-06 10:28:21 +00:00
Shuhei Matsumoto
95b35daa5f rpc: Add thread_set_cpumask RPC
Add an new JSON RPC thread_set_cpumask to control CPU affinity of
the thread dynamically from outside of SPDK application.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I6b6b794405f612200a2c3cb27dc4fc1ad2e88bec
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/501
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-03-04 10:03:30 +00:00
Maciej Wawryk
23f78f22cf scripts/perf: extract SKT section to new file
PCM cpu measure give as result 1306 columns, we want check only ones which contains
SKTdata and SKTtraffic values.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I52b6781a7ff99e0d894f7f1c2b25b655fa274eb1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/946
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-03-02 10:33:48 +00:00
Artur Paszkiewicz
339fb7cd15 genconfig.py: allow numbers in config option names
To make it possible to add CONFIG_RAID5.

Change-Id: I1712bde5bae552811343fb1865f1224d3adb1c7e
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/854
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-02-20 14:40:19 +00:00
Maciej Wawryk
f7d520e864 scripts/vagrant: Fix missing line break
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Iaace103027e4261e01b5e13233afc4094abf2386
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/939
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-02-19 09:47:56 +00:00
Maciej Wawryk
af694c3de8 scripts/pkgdep.sh: Add dependencies for Centos8
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I5921d5f7f70f2df634ccd008eb27a02496a47bee
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/603
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-02-18 14:42:16 +00:00
Maciej Wawryk
e5a5eaa660 scripts/pkgdep.sh: Add flag for building docs dependencies
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: If2c64eaa688fdc067479a357cbfd407570937419
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/602
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2020-02-18 14:42:16 +00:00
Maciej Wawryk
fe5954c67f scripts/pkgdep.sh: Add flag for RDMA dependencies
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Id38a3c5066c2c71d681d61f7ea8be748ddadaaf0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/601
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2020-02-18 14:42:16 +00:00
Maciej Wawryk
2fc1ed76af scripts/pkgdep.sh: Add flag for FUSE and NVMe-CUSE dependencies
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I2cfdd33b7d0e655d80e28958aa0f12c465592cb7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/600
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-02-18 14:42:16 +00:00
Maciej Wawryk
f035937dd5 scripts/pkgdep.sh: Add flag for pmem dependencies
Add flag for pmem dependencies needed by reduce and pmdk components.
Also add missing libpmem in Ubuntu section.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I596a1e6660f8b6f92bdab81046fa3533ae78fca3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/599
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-02-18 14:42:16 +00:00
Maciej Wawryk
c79a950fe4 scripts/pkgdep.sh: Fix NASM dependencies Ubuntu
Add information about missing dependencies for Ubuntu versions smaller than 19.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I776d93e156c07c618df9cb5eeb99c0ca8f78e388
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/598
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-02-18 14:42:16 +00:00
Maciej Wawryk
0c50f8311e scripts/pkgdep.sh: Fix libfuse3-dev dependencies for Ubuntu
Add information about missing dependencies for Ubuntu versions smaller than 19.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Id48b462be35a761fd9fad4bdc48c1757cf5a7580
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/597
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-02-18 14:42:16 +00:00
Maciej Wawryk
9ebbf3acac scripts/pkgdep.sh: Add flag for developer tools installation
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I8876fd7c60198c597d931c5f498644c415c2a868
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/596
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-02-18 14:42:16 +00:00
paul luse
f070b78cd1 module/crypto: add RPC and plumbing to support QAT AES_XTS cipher
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Iadd03f8da305e97e3f1d57ef4aeaece400e8d959
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/441
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-02-18 08:05:42 +00:00
paul luse
e753402aa1 lib/accel: add RPC to select which accel module should be used
Startup RPC only to avoid issues trying to switch while in use.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I7f1801905de0927f3610c4065182dd8fa88cdfdd
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/660
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-02-18 08:05:34 +00:00
paul luse
aa7a13afc7 global: rename copy to accel
The copy engine library, modules and public APIs have been renamed.
Use of the word `copy` has been replaced with the word `accel`
short for accelerator in preparation for adding new capabilities
in the future. Additionally, APIs for what was previously called
the `memcpy` engine have been renamed to identify the engine as a
software accelerator.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Ia607aa718416146fbba1e6792b8de0f66bd8a5de
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/576
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-02-18 08:05:34 +00:00
Karol Latecki
93be26a51d Fix Markdown MD022 linter warnings - headers blank lines
MD022 Headers should be surrounded by blank lines

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I768324b00fc684c254aff6a85b93d9aed7a0cee5
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/656
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-02-17 10:07:21 +00:00
Maciej Wawryk
344c318464 scripts/vagrant: Create prebuild box and allow using spdk boxes
Build local vagrant box and add it to local vagrant box list.
Check if spdk box exist and use it instead of generic boxes.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Ib3b87c9783354680657f96b5b5adea1706712c5e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/848
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-02-17 10:06:09 +00:00
Maciej Wawryk
dd9a88cc7f scripts/vagrant: Unified distribution names in Vagrantfile and create_vbox.sh
Remove unnecessary case statement in Vagrantfile

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I6aa5d0650d268851ee5af73ffc2aa54d9396fec5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/834
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-02-17 10:06:09 +00:00
Maciej Wawryk
22310f6176 scripts/vagrant: Make the Box as Small as possible
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I23b92fb6ef7d1169712f81d6d842db3981b8f764
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/677
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-02-17 10:06:09 +00:00
Maciej Wawryk
58e6ef44f8 scripts/vagrant: add option to build box for spdk
New option --package-box will install all dependencies needed to build SPDK

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Iedc21812b754692a5eba87127186e11ec44d805d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/538
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-02-17 10:06:09 +00:00
Seth Howell
b4aa3074bc setup.sh: make it easier to override with igb_uio
When overriding with igb_uio you used to have to manually probe uio
before calling setup.sh. I feel this is a common enough use case that we
should check it in setup.sh directly.

Change-Id: I8a31f90bab6960eade468816ffe960820f83d2fe
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/818
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-02-13 09:51:54 +00:00
Seth Howell
615b6849b4 setup.sh: fix DRIVER_OVERRIDE behavior.
This wasn't done properly for passing the whole path to a .ko file
previously.

Change-Id: Iabfcda95537003a68f952e4e8e6bcd141d2936f7
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/793
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-02-13 09:51:54 +00:00
Maciej Wawryk
db65d58359 Add pcm measurement to nvmf tests
PCM - Processor Counter Monitor is an application to monitor performance
and energy metrics of Intel® Core™, Xeon®, Atom™ and Xeon Phi™ processors.

Using this measure tool will help us to capture memory bandwidth consumed
on Intel processors.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Ie580d2aad6ddd98c5256df65f65e478dd78317e5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/541
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-02-11 08:37:42 +00:00
Tomasz Kulasek
8e3b0c159b rpc: fix allow to use bdev_nvme_apply_firmware in rpc.py
Change-Id: I53c5c69b7b954a491e29a72305bfc1b6c57aabb1
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/532
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-02-06 09:46:17 +00:00
Karol Latecki
3d8a0b19b0 doc: Fix Markdown MD032 linter warnings
"MD032 Lists should be surrounded by blank lines"
Fix this markdown linter error by inserting newlines or
adjusting text to list points using spaces.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I09e1f021b8e95e0c6c58c393d7ecc11ce61c3132
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/434
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
2020-02-06 09:46:06 +00:00
Maciej Wawryk
4c67fe81eb scripts/pkgdep.sh: Separate minimal required dependencies
We want to split pkgdep.sh to install packeges that are required to run SPDK
and that are needed by developers.
This patch is only split one install command to divide the minimal
dependencies from the developer tools.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I332cbc430bd04e6ef2872ab762da99e837bf0fc2
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/481595
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-02-04 19:50:26 +00:00
Maciej Wawryk
3d2ee8d1ce scripts/vagrant: allow password authentication
Add password authentication to vagrant for future
boxes building functionality

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Iad9c3c4000ac43243900ee6b5396863b9ab820ff
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/477062
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-02-04 18:19:43 +00:00
Wojciech Malikowski
dcd3fc1fd1 bdev/ftl: "use_append" parameter for bdev_ftl_create RPC
Allow for using appends instead of writes.

Change-Id: I2f0d3bcdbb0eee034f7b0b6349de854ddbf7273d
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/481839
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2020-02-04 18:10:01 +00:00
paul luse
6ea187fcbb scripts/vagrant: update readme
Fix a bunch of typos, remove a few things and add some disclaimers
for legal cleanliness :)

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I366380f7b822c5b7a5ca92ab5a9bd3fd68296bee
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/483752
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
2020-02-04 17:04:12 +00:00
paul luse
e914958951 module/compress: rename set_compress_pmd RPC to match convention
set_compress_pmd -> compress_set_pmd

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Icb1ecc7adfe10485c44f98ab9e31eaa6857596e7
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482597
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-31 12:37:31 +00:00
Seth Howell
f257be1313 doc: replace all cloudfront refs with ci.spdk.io
It's good to have a human readable domain again.

Change-Id: If773e28a267b635a94a45b07fba9a32d62d0c248
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/483146
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-31 12:36:19 +00:00
Seth Howell
0e9727db73 scripts: add a parser for the dpdk memory stats.
This script takes an optional dpdk stats file argument and tries to
parse the information from that file into a comprehensible structure. It
then provides a basic UI for printing that memory information at various
levels of granularity.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: Ibbb09f41122be18b7e640781f3e140618e52c0ab
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/477635
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-01-24 05:46:43 +00:00
jiaqizho
f1687a1bd2 script: add iostat.py to check bdev iostat
this tool likes sysstat/iostat
added to qos test

Signed-off-by: jiaqizho <jiaqi.zhou@intel.com>
Change-Id: I7ed6538095d34fa8c5835f28b25d618f1b3d81b7
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476537
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-01-22 13:51:41 +00:00
Michael Haeuptle
c9874b91a7 lib/nvmf: custom admin handler config
This commit disables the custom identify handler by default.
The user has to explictly enable this handler via the set_nvmf_config
RPC or conf file.

Change-Id: I767816ba7639ebe78683993408ce6db02c7620fe
Signed-off-by: Michael Haeuptle <michael.haeuptle@hpe.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/479603
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-20 09:56:35 +00:00
Pawel Kaminski
50e5d88317 test: shellcheck - ignore SC2010
SC2010: Don't use ls | grep. Use a glob or a for loop
        with a condition to allow non-alphanumeric filenames.

Replacing e.g. "ls /dev/nbd* | grep -v p" would be complex
and problematic as flag -I for ls is not working when
looking for pattern.

Change-Id: I9e88359d247fe5dcf952a3388b0e7f35164768c1
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/481602
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-17 14:07:39 +00:00
Pawel Kaminski
13c6c8db6e test: shellcheck - apply rule SC2002
SC2002: Useless cat. Consider 'cmd < file | ..' or 'cmd file | ..' instead.

Minor fix in test_converter: don't use python3 as
    config_converter.py is executable.

Change-Id: I250e893b48fa95e94778dcc31deb91e6ac0b39c0
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/481600
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-17 14:07:39 +00:00
Maciej Wawryk
6c2747491d Shellcheck - ignore SC2034 warnings
Add to pernamently excluded in shellcheck:
    SC2034: foo appears unused. Verify it or export it.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Ib63edb8cdf8ec718981d28d5acec75772b825a48
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478102
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-01-15 14:30:06 +00:00
Maciej Wawryk
19e48a66a4 test: Shellcheck - correct rule: Iterating over ls
Correct shellcheck rule SC2045: Iterating over ls output is fragile. Use globs.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I436878774d6f86c23b1c5cf5220da297053f03e2
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/477397
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-01-15 14:30:06 +00:00
Wojciech Malikowski
f2c8083ef3 lib/ftl: Functional tests
This patch adjust functional tests to
new FTL stack.

Change-Id: I0b8ff08c61793d7f1685942031aecf9cc88d1f24
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/468738
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2020-01-15 12:24:44 +00:00
Seth Howell
2f476a638f autobuild.sh: simplify and unify test suite.
We have added a lot of flags to this script over time to try and shorten
parts of it that take a long time or modify the way we make and
whatnot.

In the test framework today we really have two settings, we either want
to run the entire autobuild package with all the bells and whistles, or
we just want to make the code and get on with the rest of the tests. I
believe this change can save between 1 and 3 minutes on each of the
functional test suites.

Change-Id: I7519e8320aa16b57f09f633f866dc36cb494aa80
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478483
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-14 07:49:08 +00:00
Pawel Kaminski
91dc22bae6 test: shellcheck - move rule SC1083 to permanent exclude list
SC1083: This {/} is literal. Check expression (missing ;/\n?) or quote it.
This rule points to places that are part of command
and cannot be fixed:
1. git archive HEAD^{tree}
2. vpp (e.g. dpdk { no-pci })

Change-Id: I30117913f964415cdaed94d29b1209d119e4d5a4
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/479986
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-01-14 07:48:03 +00:00
Pawel Kaminski
7200ad8199 test: shellcheck - move rule SC2046 to permanent exclude list
SC2046: Quote this to prevent word splitting.

Change-Id: Iaafcf823c72a7dcf1b6eab893e14ea85e2c67ccf
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/477321
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-01-14 07:48:03 +00:00
Pawel Kaminski
9c47dd8e09 test: shellcheck - move rule SC2086 to permanent exclude list
SC2086: Double quote to prevent globbing and word splitting.

Change-Id: Ia427dbb3b5d5ad20bcc09a3a455b80d0f69fb583
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/477266
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-01-14 07:48:03 +00:00
Pawel Kaminski
6df3f23f22 test: Shellcheck - apply rule SC2097 and SC2098
SC2097: This assignment is only seen by the forked process.
SC2098: This expansion will not see the mentioned assignment.

Apply these two rules together as they point to the same code.

Change-Id: I61020e8fd4b6db2b06ae8134d1004290a9bef9e7
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476777
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-01-13 12:45:04 +00:00
Wojciech Malikowski
52a9661d94 bdev/ftl: Remove punit parameter from ftl bdev configuration
FTL library is consuming whole OCSSD device
so punit parameter is not needed for bdev ftl
configuration.

Change-Id: I56f62ea6d09b3157b70c02ccfffcd3cb07ba4597
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/467950
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
2020-01-10 08:42:51 +00:00
Seth Howell
fd73cb78d8 check_format: add SC2015 to SHCK_EXCLUDE list.
This shellcheck case warns users when they use the pattern A && B || C
so that they don't accidentally try to use it as an if-then-else
statement. However, there is a valid way of using this pattern to
denote, do A, if it succeeds, do B, if either A or B fails, do C.

Change-Id: I283b433c15d07873befd182e8327f2e0602490aa
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476960
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2020-01-08 15:46:18 +00:00
Shuhei Matsumoto
876fdf2a59 event/rpc: Add framework_get_reactors RPC
Add an new RPC `framework_get_reactors` to retrieve list of all
reactors.  Running threads on each reactor are included
in the output.  Update jsonrpc.md and CHANGELOG accordingly.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I985d087dd41afe3033b7678af141ec8e5a2a822e
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478027
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
2020-01-03 08:33:25 +00:00
Karol Latecki
f1d0fc75cf scripts/perf: use additional kernel NVMe-oF initiator params
Add configuration option "extra params" to run kernel initiator
with nvme connect options like -i, -I, -D, etc.

Change-Id: I01d9fd0ed15dc5607f0c0f58d97581c5e0b821ba
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/477767
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-12-24 08:13:14 +00:00
Karol Latecki
f610df8d52 scripts/perf: remove hardcoded NVMeOF initiator parameter
Removing hardcoded "-i 8" value for kernel NVMe-oF initiator
parameter "nr-io-queues".

This was previously merged by mistake and had to be removed
manually in test runs.

Change-Id: I454ad468863f93ac2ce93a1f0bd00f678ed4995c
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/477766
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-by: Seth Howell <seth.howell@intel.com>
2019-12-24 08:13:14 +00:00
Karol Latecki
9dfe2a4bce scripts/perf: clean up constructor parameters
Move code around so that lines containing arguments
for constructors are better aligned with how they're
inherited.

Change nvmet_dir and fio_dir to nvmet_bin and fio_bin to
point directly to executable files. There is no advantage
in using directories.
Additionally fix how fio_bin (previously fio_dir) is passed
to parent classes - it was broken and caused exceptions.

Update README.md accordingly.

Change-Id: I10524593d543e236bf318a8cd51ea5375c334e8b
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/477015
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-12-24 08:13:14 +00:00
Konrad Sztyber
64fe514efa bdev/ocssd: parallel unit ranges
This patch adds the ability to create bdevs on specified parallel unit
ranges on one OCSSD controller.  It allows the user to create multiple
isolated bdevs, each operating on a separate set of parallel units.

To create a bdev on a specified range, a new parameter -r|--range was
added. For example:
 rpc.py bdev_ocssd_create -c nvme0 -b nvme0n1 -r 0-4

will create a bdev on parallel units 0 through 4 (inclusive).  If the
parameter is ommited, the bdev will span across all available units.

Change-Id: Icd94cf1e22fcc72806ef1ce1efd2d7031010009f
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/469898
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-12-20 10:02:15 +00:00
Karol Latecki
e0a8a21dab scripts/vagrant: Adjust Vagrantfile for FreeBSD boxes
- Use "scsi" libvirt disk bus
  generic/freebsd boxes do not boot with default
  "virtio" bus
- Manually provision proxy settings
  Proxyconf plugin seems to not work with generic/freebsd boxes.
  Adjust shell and pkg proxy settings manually via scripts.
- Disable synced_folders provisioning
  FreeBSD boxes don't have rsync and NFS installed by default.
  Vagrantfile "synced_folders" step executes before "priovision"
  steps responsible for setting proxiy settings, therefore we
  are unable to install it.

Change-Id: I1386ec6e89cff6b8412e97e4df95a6149da2a02c
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/477805
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
2019-12-16 12:30:47 +00:00
Karol Latecki
08b23462f5 scripts/vagrant: exit on first script error
Use bash -e option to exit from script if any error
occurs.

Change-Id: Ieaad74df4598e193f619947ad34d5a1fc572d1a9
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/477453
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-12-13 11:06:54 +00:00
Seth Howell
de467ec91d env_dpdk/rpc: add rpc to get memory stats.
This will enable us to create a helper applciation to print memory
statistics for a given SPDK application.

Change-Id: I0ad5a8163166a5d67d9a5533303ec0858297ba6b
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/477511
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-12-13 11:05:57 +00:00
Maciej Wawryk
7561d1b5c2 scripts/vagrant: fix scripts for default emulator
Default emulator does not support OCSSD and Multi-namespace NVMe's.
Add if statement to check if user change emulator.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I30858bfed0c2e571c1aefa6ff7ccbf3bc436a294
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476454
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
2019-12-10 17:33:32 +00:00
Evgeniy Kochetov
542be8790b nvme/rpc: Make 'delay_cmd_submit' configurable via RPC
Optional parameter 'delay_cmd_submit' has been added to
'bdev_nvme_set_options' RPC method.

Signed-off-by: Evgeniy Kochetov <evgeniik@mellanox.com>
Signed-off-by: Sasha Kotchubievsky <sashakot@mellanox.com>
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Change-Id: Ia09c57bbac8078ee6ab1e8f0bd279ffeb9e08cb8
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462584
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
2019-12-10 17:32:10 +00:00
Tomasz Kulasek
b7b45bc7bc lib/nvme: remove device name parameter from nvme cuse
This patch removes posibility to set cuse device path. Instead
"/dev/spdk/nvme*" path is used.


Change-Id: I7c3087772a3661eebe03fce21356c35cc8204b49
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/474598
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2019-12-09 17:40:35 +00:00
Konrad Sztyber
6bda87b324 bdev/ocssd: create / delete calls
Added a way to create and delete OCSSD bdevs on top of OC NVMe
controller.  The controller can be created using the regular NVMe bdev
RPC call.  For instance, the following (assuming 0000:00:04.0 is an
OC device):

 rpc.py bdev_nvme_attach_controller -b nvme0 -a 0000:00:04.0 -t pcie
 rpc.py bdev_ocssd_create -c nvme0 -b nvme0n1

creates Open Channel controller nvme0 and OCSSD bdev nvme0n1 on top of
it.  The bdevs can be deleted either by the bdev_ocssd_delete call or by
deleting whole NVMe controller, in which case all bdevs are destroyed.

Change-Id: I9f2f02103fc5570a53bd26479c8690be206829c3
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/468984
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-12-06 14:45:05 +00:00
Jim Harris
5bd76a1098 rpc.py: expliclty print the exception's message
This worked fine as-is with Python 3.  But when
trying to run rpc.py with Python 2, it was noted
that simply doing print(ex) didn't print the message,
it needs to be print(ex.message).

Note that the similar code for printing the exception
message when running an RPC script was already doing
print(ex.message), so this change also makes the
code more consistent.

We aren't supporting or testing rpc.py with Python 2,
and there are some fairly significant mods required
to get it to work, but this small change at least
reduces that effort for those needing to implement
it and maintain it themselves.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I0a6c94889dc631ef33835433f20f35ddc738aafb

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476633
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
2019-12-05 12:23:30 +00:00
Jim Harris
8bc9798dc7 rpc.py: add new kv to existing params dict
When we get an error when calling an RPC, client.py
makes extensive use of the ** operator to add
the method and req_id to the params dict variable.

This doens't work with Python 2.  We don't really
support Python 2, but there are at least some folks
out there who make their own mods to get rpc.py
to work with it.  We can very easily implement this in
a way that is Python 2 friendly, so let's do that
to make it a tiny bit easier for those folks.

Incidentally, I do think the changes here also make
this part of the code a bit easier to read and
understand.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I1846c80e21032ffba67128bee946b041a61d0621

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476632
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2019-12-05 12:23:30 +00:00
Maciej Wawryk
1f08daeb81 test: Shellcheck - move rule SC2120 to permanent exception list.
SC2120: foo references arguments, but none are ever passed.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Ic8d515ca5d0b81b77e22e1e726e403dc6678af5f
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476469
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-12-04 15:22:07 +00:00
Maciej Wawryk
4aa03903c7 test: Shellcheck - move rule SC2119 to permanent exception list.
SC2119: Use foo "$@" if function's $1 should mean script's $1.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I8366debbf83dfc8f3c1033e1e9fe3df17e45ab49
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476468
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-12-04 15:22:07 +00:00
Maciej Wawryk
d55a5efffb test: Shellcheck - include check SC2090
Remove from shellcheck excluded SC2090: Quotes/backslashes
in this variable will not be respected.
After fix SC2089 this warning is not actually present in
any of spdk/spdk scripts.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Ibe0015b569202eb7d3f3197f5c5630a67e701332
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476467
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-12-04 15:22:07 +00:00
Maciej Wawryk
855dba2d97 test: Shellcheck - correct rule: Quotes/backslashes...
Correct shellcheck rule SC2089: Quotes/backslashes will be treated literally. Use an array.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I40d4eef334cd030294669f8e8494f9e7a563d495
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476466
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-12-04 15:22:07 +00:00
Karol Latecki
4342c0f0d5 scripts/vagrant: unify VM boxes source
Unify all distro selections to use the same generic/* source
for downloading boxes.
"generic/" is provided by Roboxes.

Change-Id: Id5d704e35313c382a564da2a50177766293a88c1
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476160
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2019-12-04 15:16:04 +00:00
Karol Latecki
672c73fb12 scripts/vagrant: fix tsocks.conf provisioning for Ubuntu
There is no "wheel" group on Ubuntu distros. Chown command
caused Vagrantfile to fail.
While at it - put the whole bash command flow into a variable
and run it as part of single "shell" provision step instead of
doing it in three steps.

Change-Id: I2853d4176f66f07f1762b339b20ced3386486f87
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476159
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2019-12-04 15:16:04 +00:00
Karol Latecki
1502112764 scripts/vagrant: update boxes list
Update VM boxes list.

- Remove Fedora 28 as it is obsolete since May 2018
- Add Fedora 31
- Add CentOS 8
- Add FreeBSD 12
- Make Fedora 30 a default box selection

Change-Id: I006f49e1e596ed61926777e6ae26d9bedea1fac3
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476158
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: yidong0635 <dongx.yi@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2019-12-04 15:16:04 +00:00
Karol Latecki
4033c66071 scripts/vagrant: do not treat SPDK_VAGRANT_DISTRO as array
Dash and double dash ("-" and "--") getopts params are safely
shifted using "shift" and "OPTIND". After that it's safe to
concatenate param array to string using "$*", especially that
there is only 1 positional parameter, which is distro version.

Previous approach ("$@" as array) made "export" impossible and
as a result VM was created with Fedora 28 distro on each run.

Change-Id: I48fa588220f8b7f46c561d480ee32920847c7903
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476128
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Pawel Kaminski <pawelx.kaminski@intel.com>
2019-12-04 15:16:04 +00:00
Pawel Kaminski
b98f5c2429 scripts/setup.sh: Fix. Variable virtio_names used as array contains empty first element.
It fixes change 474989.

"for blkname in $blknames; do"
was changed to
"for blkname in "${blknames[@]}"; do"
if blknames=" vda" then first blkname="" and
linux_bind_driver is called for vda although it has active mountpoint

Change-Id: I16d4379db274faa5703230072cfcf8d18a8ef5e6
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476061
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-12-04 15:12:46 +00:00
Maciej Wawryk
0951fc4fea test: Shellcheck - move rule SC2016 to permanent exception list.
SC2016: Expressions don't expand in single quotes, use double quotes for that.
We need to exclude this shellcheck because in some cases, for example in
test/vhost/common.sh we echo line to new script file.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I98d322d917cbda034787cb29a267b86a8ad7e7ec
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/475713
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2019-11-27 07:08:57 +00:00
Maciej Wawryk
a69df7ea94 test: Shellcheck - correct rule: Use find...
Correct shellcheck rule SC2012: Use find instead of ls to better handle non-alphanumeric filenames

Change-Id: Ibfc54ac06ae4cecadd22954159d259295f99912f
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/475712
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-11-27 07:08:57 +00:00
Maciej Wawryk
fdc82d3bc8 test: Shellcheck - correct rule: Double quote array
Correct shellcheck rule SC2068: Double quote array expansions to avoid re-splitting elements.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Iefc4f0104249f4d437a66c3d9c7a195f4f6f6da3
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/475690
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-11-27 07:08:57 +00:00
Maciej Wawryk
25f601cfb8 test: Shellcheck - correct rule: Expanding an array
Correct shellcheck rule SC2128: Expanding an array without an index only gives the first element.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I0e7c335af678114dc78dfb12a02369a69158e435
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/474989
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Seth Howell <seth.howell@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2019-11-27 07:08:57 +00:00
Maciej Wawryk
1d95f22d62 scripts/vagrant: Add automatic emulated disk create
Current scripts/vagrant allow us to manual create every emulated device.
This patch allow to automatic create whole emulated disk while we
create virtual machine.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: If772adc282c4f6d94dab5eb7337f097a08f708e9
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/475070
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-11-22 13:52:11 +00:00
Maciej Wawryk
029251878d test: Shellcheck - correct rule: Consider using { cmd1; cmd2; }
Correct shellcheck rule SC2129: Consider using { cmd1; cmd2; } >> file instead of individual redirects.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Ic692e9f78f2d3d8a3b27d702884cbe97c880727c
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/474959
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2019-11-22 13:51:15 +00:00
Maciej Wawryk
f924c94b4c test: Shellcheck - correct rule: Use grep -q...
Correct shellcheck rule SC2143: Use grep -q instead
of comparing output with [ -n .. ].

Change-Id: Ibfce9633eda332c64b27914b2fde51c9a26ff7f3
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/474957
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-11-20 10:03:15 +00:00
Maciej Wawryk
809a6f82de scripts/vagrant: Allow running multiple VMs in parallel
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I7047c3152558911a782be86e5ccf8214f2418c02
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/472692
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Seth Howell <seth.howell@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-11-18 13:15:17 +00:00
Maciej Wawryk
ff231506d3 scripts/vagrant: Add more NVMe emulation options
Current scripts/vagrant allow us to create only a single file for
emulating NVMe with single namespace.
This patch allow to create multiple emulated drives with multiple namespaces.
This patch also allows to emulate OCSSD disks which are used in OCF tests.

Change-Id: Ia7a4667bd405ba18235e72bcdf608270002b5242
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/466555
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Seth Howell <seth.howell@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-11-18 13:15:17 +00:00
Ziye Yang
2df770535a test/ceph: fix the dev name in start and stop.sh
I am confused with the dev_backend variable, I think that
we may not need this variable.

Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I26fa5e8a829e574bb1b97add753b21ad1b0fc23e
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/473910
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: yidong0635 <dongx.yi@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-11-18 13:01:58 +00:00
Mateusz Kozlowski
195be29e9b bdev/zone: Rpc functions for zoned block bdev
Added rpc handling for creation and deletion of zoned block bdev

Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: Ib7b6f3928511aa840cbe098472682c951e92e5ed
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/468031
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-11-15 20:27:14 +00:00
dongx.yi
65cb0a108d scripts/ceph: Fix some compatible issues about ceph-14.*.*.
This is fix the issue on fedora30 about ceph-14.*.*.
Our ceph setup is a simple or demo case for rbd creation.
Make this script can run well on all versions.

Unify the filesystems about filestore and bluestore. That
all versions' logs seem similar.

We can trace the log through file:/var/log/ceph/ceph-mon.a.log.
It can record the whole processes.

osd.0 [v2:127.0.0.1:6800/2770008,v1:127.0.0.1:6801/2770008] boot
osdmap e5: 1 total, 1 up, 1 in

fix issue: #980.

Signed-off-by: dongx.yi <dongx.yi@intel.com>
Change-Id: I594fdab791fdc6af6df739dacef4648e396e3d35
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/473831
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-11-14 16:29:58 +00:00