Commit Graph

4082 Commits

Author SHA1 Message Date
Ziye Yang
34bff15a19 ftl ut: Fix the compilation issue.
If we did not configure with "--enable-debug". it will
trigger the compliation failure since this function is
not defined.

Change-Id: Ia1795da736be55d4dd1418a2e74edefff7797f11
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1097
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Michael Haeuptle <michaelhaeuptle@gmail.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-03-04 05:02:35 +00:00
Michal Berger
692fce3222 test/common: Introduce timeout in discover_bdevs()
In case something goes awfully wrong with the test which calls this
particular function, e.g:

  https://ci.spdk.io/results/autotest-per-patch/builds/4787

the CI's autotest runs may be blocked for max of 15 minutes before
failing the build. Instead, introduce internal timeout for the
routine itself (default of 30s) and fail the test sooner when it
expires.

Change-Id: I2c792c1781987e80e34c1fa19ec3eef43c25e38a
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1026
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-03-03 07:55:12 +00:00
Karol Latecki
ba5329d3e4 test/nvme: do not skip tests if nvme-cli is unavailable
We do not want to skip tests, especially that the
'if' statement is inside the test script.
So even though the test wasn't actually ran it still
is reported as executed because "spdk_nvme_cli.sh" returned
with rc=0.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I89c553aebcc1346f8f584dd86eeae1ca146e7340
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/980
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
2020-03-03 07:55:07 +00:00
Richael Zhuang
ceb601188b test: fix error about unittest_thread uninitialised parameters
When running unittest_thread in fedora29 with memcheck, there is
error "Conditional jump or move depends on uninitialised value(s)"
in for_each_channel_unreg test.

Signed-off-by: Richael Zhuang <richael.zhuang@arm.com>
Change-Id: I38abf4c70ff513e39232b83357d571dded8f02c3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1072
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-03-03 07:55:01 +00:00
Michal Berger
70cc99bccf test/nvmf: Sanitize random string
In case the random string beggins with `-` (still a valid ascii char
that nvme supports) the rpc.py arg parser will treat it as an
additional argument instead of actual subsystem value.

To mitigate, escape the `-` in case it's the char that starts the
string.

Fixes: 1240

Change-Id: I07bd8134fb9fc16234e72e9ebd27e2b6b90d6b62
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1054
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-03-02 10:34:03 +00:00
Konrad Sztyber
336505abbf lib/ftl: use per-io_channel write buffers
Replaced single global write buffer with the per-io_channel write
buffers.  This means that the "rwb" module and all of its references
were removed and replaced with the recently added interfaces.

Change-Id: Idc899d3a4d63a8a2bede1ac26549ed06e9a2e784
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/909
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
2020-03-02 10:27:06 +00:00
Konrad Sztyber
925cc3b8a9 lib/ftl: pending batches queue
Added queue responsible for keeping track of full, ready to be written
batches.  A batch might be put on this queue in case it'd already been
filled, but could not be written out due to lack of resources or it was
written out, but failed and needs to be resent.

Change-Id: Iba49cd359425300d21b8100f13f17189e99d4c7c
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/908
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Maciej Szczepaniak <maciej.szczepaniak@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
2020-03-02 10:27:06 +00:00
Konrad Sztyber
87731e1105 lib/ftl: write buffer entry address calculation
Two functions were added:
 - ftl_get_addr_from_entry() returning an address pointing at a given
   entry
 - and it's inverse, ftl_get_entry_from_addr(), returning a write buffer
   entry for a given address

Change-Id: Iea1e2e59f240f50183fdb549280a0f0b722efbef
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/906
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-03-02 10:27:06 +00:00
Konrad Sztyber
35fdf2dd9e lib/ftl: write buffer batches
This patch adds the ftl_batch strucutre, which describes a single batch
of data to be written to the disk.  It's comprised of multiple write
buffer entries (the actual number depends on the write unit size of the
underyling device).

Additionally, a function responsible for filling out a batch was added.
It iterates over available IO channels and dequeues submitted write
buffer entries until a batch is completed.  The IO channel queue is
shifted, so that each time the function is called a subsequent IO
channel is used first, which guarantees that all channels are treated
fairly.

Change-Id: Ie61d8c6cb51d5c5175540c975447bc27590c5cb4
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/905
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-03-02 10:27:06 +00:00
Konrad Sztyber
1e8c9a4de2 lib/ftl: write buffer acquire/release
Added functions responsible for acquiring/releasing write buffer entries
from the per-io_channel caches.  They're also in charge of limiting user
writes by reducing the number of available entries to the user.

Change-Id: I23adfe36065139036f9b72a016e34fbf6eb29230
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/904
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-03-02 10:27:06 +00:00
Konrad Sztyber
bfafd4e472 lib/ftl: IO channel array
The IO channel pointers are now stored inside an array of the device
they belong to.  Once write buffer entries are tied to IO channels,
it'll provide a method for dereferencing an entry from its address.

Change-Id: Iaf401525eb0f5af8dc6047a1dc8bae11b56761d7
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/901
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-03-02 10:27:06 +00:00
Konrad Sztyber
085bfa0d13 lib/ftl: allocate separate memory for IO channels
This is the first patch of a series that replaces single global write
buffer with per-io_channel buffers.  This change is intended to improve
performance for multithreaded workloads.

This patch changes the way the ftl_io_channels are allocated by only
keeping an ftl_io_channel pointer inside spdk_io_channel's context.  It
allows for delaying IO channel destruction, which in turn allows the FTL
to iterate over all exisiting IO channels without locking.

Change-Id: I5e0cab8043a2b5f747e971dd3d65ed2546c8cf26
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/900
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-03-02 10:27:06 +00:00
Shuhei Matsumoto
595173aaba bdevperf: Make bdevperf_construct_targets() asynchronous by sentinel value
Follow the idea of bdev_get_iostat in module/bdev/rpc/bdev_rpc.c.

Add a global variable g_bdev_count and then set it to 1 as sentinel
value in bdev_construct_targets().

Subsequent patches will make _bdevperf_construct_targets() asynchronous.
Then bdevperf_construct_targets() will be truely asynchronous.

This patch is too small to be an independent patch but keep to
hold previous review result.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I49d0b6e6d61df65d4eb31c402f98cc4300482e20
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/639
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-02-28 14:38:15 +00:00
Shuhei Matsumoto
a4335feb6d lib/event: Support SPDK_THREAD_OP_RESCHED operation in reactor
Add a flag resched to check if reschedule operation is requested
to struct spdk_lw_thread. Add _reactor_resquest_thread_reschedule()
to set the resched flag, and add it to the case SPDK_THREAD_OP_RESCHED
in spdk_reactor_thread_op(), and return true in the case
SPDK_THREAD_OP_RESCHED in spdk_reactor_thread_op_supported().

Then _spdk_reactor_run() checks if the resched flag is true for each
thread. If true, set the resched flag to false, and remove the
thread and call _reactor_schedule_thread(). Add continue to avoid
use-after-free issue for both reschedule and terminate cases.

This idea follows voluntary thread termination and will remove our
worries for all complicated rare cases.

Add unit test case to verify this update.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I656872d32dbb469ae70f771cd0419a77236bfe18
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/500
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-02-28 14:37:53 +00:00
Seth Howell
bad25cad1b test/nvmf: don't return before calling nvmftestfini
This is necessary to avoid leaving the machine in a bad state between
test runs.

Change-Id: I2c68d2dc88f0f9f4cab099564d25a3e21eb5671c
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1027
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-02-28 08:55:07 +00:00
Wojciech Malikowski
f93ac08d26 lib/ftl: Merge ftl_thread structure with ftl_dev
Since ftl supports only one core thread ftl_thread
structure is not needed anymore.

Change-Id: I66eda9249d74b3eff0c4473dcd0aba4a6135e296
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/548
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Maciej Szczepaniak <maciej.szczepaniak@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2020-02-28 08:54:07 +00:00
Tomasz Zawadzki
49de02ec4d Revert "test/hotremove: Select test cases to be run for scci and blk hotremove."
This reverts commit 29e9fdc857.

Nightly tests after merging the reverted patch show that it does not
pass in current state. It affected the per-patch, since the new
tests weren't added to skipped_tests list.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: If8a58cbe9ecd0e58f20c0a9ee844bc9a8ee046a3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1045
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-02-27 18:11:36 +00:00
Maciej Wawryk
dad9d17406 test/common: Add smartmontools dependencies for FUSE and NVMe-CUSE in vm_setup.sh
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Ia8f12aca0a70e93bddd0ad7aeb658f85044ffdfa
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/956
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2020-02-27 10:16:19 +00:00
Pawel Kaminski
29e9fdc857 test/hotremove: Select test cases to be run for scci and blk hotremove.
With this change user can defined test cases that he wants to run.
It could enable to run hotremove test cases per-patch and nightly.

Change-Id: I6876f39886a347ada4dd548a2f45b99a37207c3e
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/768
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-02-27 10:16:10 +00:00
Darek Stojaczyk
ebe4bfc2bb test/common: add one more fio leak to ASAN suppresions list
It comes up on failed fio runs and bloats the log.

Change-Id: I8dd0196bfc98b3ee888aca0def83f0961fbc1197
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1010
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-02-27 10:15:52 +00:00
Michal Berger
2695854b90 test/nvmf: Add skel for nvmf error testing
The goal of this test is to make sure that proper parts of the lib
fail as they should. These particular tests focus on:

  - spdk_rpc_nvmf_create_subsystem(): model and serial number checks

Change-Id: I504279d08b192d6dfe37a8a601eda44084b596d4
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/693
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-02-27 10:15:40 +00:00
Michal Berger
c0fb1b93d0 test/unit: [nvmf/ctrlr.c] Increase feature get|set coverage
Add tests for the following features:

  - SPDK_NVME_FEAT_TEMPERATURE_THRESHOLD (4),
  - SPDK_NVME_FEAT_ERROR_RECOVERY (5),

For Temperature Threshold feature the validity of the THSEL and TMPSEL
is covered as described in the NVMe spec (1.4):
	"Figure 279: Temperature Threshold – Command Dword 11"

For the Error Recovery feature the validity of the DULBE is covered
as decribed in the NVME spec (1.4):
	"Figure 280: Error Recovery – Command Dword 11"

Random value is selected for the lsb in the cdw11 (0x42).

Change-Id: Ia57ab4d79439ec315ddc9bbfdad8400aa926062a
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/692
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-02-27 10:15:40 +00:00
Michal Berger
c843612041 test/common: Introduce new test flag for enabling debug app runs
SPDK_AUTOTEST_DEBUG_APPS - if set to 1, all applications will be
			   executed with --log-flags=all

This should increase, to some extent, coverage of each application.

Change-Id: I1fc82a837abb7e94bda55f47056a120bd9be40d7
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/691
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2020-02-27 10:15:40 +00:00
Michal Berger
ee7cdc2c76 test/fuzz: Rename FUZZ_APP to VHOST_FUZZ_UP
Since there's more than one fuzz app covering different areas, rename
FUZZ_UP to VHOST_FUZZ_UP to make the type of the app clear right off
the bat.

Change-Id: I7730f1e14895d9221ec76dea0f53b5054a97be1c
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/978
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2020-02-27 10:15:40 +00:00
Michal Berger
b4c5509c6e test/common: Source *_APP from a common place
Instead of declaring each app in the actual test, source their default
declarations from a single location. Additionally, use them as arrays
now instead for easier cmdline management.

Change-Id: I0ea39403f070dc57987065ebb800e91c35de643a
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/690
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2020-02-27 10:15:40 +00:00
Pawel Kaminski
6f7cb2ad58 test/vhost: Move negative tests from nightly to per-patch.
Change-Id: Ia679f176f84b1d461f6bcc69b3fd509b6f7a66d1
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/556
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-02-27 10:15:28 +00:00
Pawel Kaminski
50e44f841a test/vhost: Add negative test for vhost_scsi_controller_add_target
Change-Id: Ie8305b8128ca2a6b16020f72ac5bfa9dec2bb7c0
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/536
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2020-02-27 10:15:28 +00:00
Pawel Kaminski
6668f0d9f2 test/vhost: Add negative test for vhost_controller_set_coalescing rpc
Change-Id: I883d20d9ff2d1da1de89543cf81c9f3d3712e44f
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/535
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-02-27 10:15:28 +00:00
Pawel Kaminski
765ae06958 test/vhost: Negative test - get controller that does not exist
Change-Id: I9f241c2dea10891bed62e7dcd26cb6d7ffc061d5
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/831
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2020-02-27 10:15:28 +00:00
Michal Berger
e338a8dd37 test/common: Don't attempt to install tsocks package on fedora 31
Currently, this particular package is not available in fedora 31's
repos, thus don't exit the setup if its installation fails.

Change-Id: I56852ed172c95b23c92e25b1cec391b4e909f2bd
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/936
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-02-27 10:15:11 +00:00
Michal Berger
e4c26938a4 test/common: Drop single quotes in --extra-cflags arg
Since '' are placed within "" they are treated as literal part of the
argument passed to configure. Due to that, configure fails to run cc
since all -W flags are treated as one.

Change-Id: I69e182f308628cc5b63bd363d3a47cf87336a59a
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1024
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-02-27 10:14:51 +00:00
Shuhei Matsumoto
1368aec870 lib/event: Use spdk_thread_lib_init_ext() to initialize thread library
Remove the prefix "spdk" from spdk_reactor_schedule_thread(), add
spdk_reactor_thread_op() and spdk_reactor_thread_op_supported().

For SPDK_THREAD_OP_NEW, spdk_reactor_thread_op() calls
_reactor_schedule_thread() and spdk_reactor_thread_op_supported()
returns true.

Then replace spdk_thread_lib_init() by spdk_thread_lib_init_ext()
with spdk_reactor_thread_op() and spdk_reactor_thread_op_supported().

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I232a3b2c6bcaf4d86b0dd3cefacd3e47eadda6d8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/968
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-02-27 10:14:23 +00:00
Shuhei Matsumoto
d82d69017f lib/thread: Add spdk_thread_lib_init_ext() and generic thread operation function
Add enum spdk_thread_op and two function typedefs spdk_thread_op_fn
and spdk_thread_op_supported_fn.

The first operation type of enum spdk_thread_op is SPDK_THREAD_OP_NEW,
and it is used as an alternative to spdk_new_thread_fn.

Add global variables, g_thread_op_fn and g_thread_op_supported_fn, and
then add spdk_thread_lib_init_ext() to initialize these.
spdk_thread_lib_init() requires both of thread_op_fn and
thread_op_supported_fn are specified or not specified.

spdk_thread_create() calls g_thread_op_fn() with SPDK_THREAD_OP_NEW
if g_new_thread_fn is NULL, g_thread_op_supported_fn is not NULL,
and g_thread_op_supported_fn(SPDK_THREAD_OP_NEW) returns true.

Update unit test to test these addition.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I56db903f62437f6ff3198248ffc5dede396c22bc
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/967
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-02-27 10:14:23 +00:00
Shuhei Matsumoto
515733ca4a lib/thread: Add unique ID for each created SPDK thread
Add an unique ID for each created SPDK thread. Use a single 64 bits
variable, g_thread_id, and guard its update by the global mutex
g_devlist_mutex. For our safety, further thread creation is not
allowed if g_thread_id rolls over, and request user to restart SPDK
application.

Besides, as a minor update, move the debug log down and add ID to it
in spdk_thread_create(), and ID is added to thread_get_stats RPC and
framework_get_reactors RPC.

The thread ID will be used to set the cpumask of the running thread
to the specified value in the subsequent patches.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ic09f11d4c7175c3b89acba6a42e76063acd0d1a0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/498
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-02-27 10:14:23 +00:00
Jacek Kalwas
fb7129881a nvmf: add const to nqn getters
There is a warning triggered when holding ref to const obj and passing
to these getters.

Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: I2c7b4ea0d325d84d66923fc524273ea44a3a311b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/997
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-02-26 09:27:11 +00:00
GangCao
c65d64a6b8 vbdev/raid: close the base bdev on its opened thread
With the JSON configure, the base device will be opened on the
same thread (RPC thread) to handle the JSON operation. Later the
virtual device upon the base device can be opened on another
thread. At the time of virtual device destruction, the base
device is also closed at the thread where opening the virtual
device, it is actually different than the original thread where
this base device is opened through the JSON configure.

Add a thread here to record the exact thread where the base
device is opened and then later route it back to handle the base
device close operation.

Change-Id: Ib95e8d190bd87158ae1ecc6698da95ccc4ba9579
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/992
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-02-26 09:26:49 +00:00
Changpeng Liu
bad2c8e86c nvme: detach the controller in STUB and flush the admin active requests at last
In the autotest, when calling kill_stub() function, there is error log
like this: "Device 0000:83:00.0 is still attached at shutdown!", so it's
better to detach the controller when exit the stub process.

But after call spdk_nvme_detach() in the stub process, there is another issue:
1. NVMe stub running as the primary process, and it will send 4 AERs.
2. Using NVMe reset tool as the secondary process.

When doing NVMe reset from the secondary process, it will abort all the
outstanding requests, so for the 4 AERs from the primary process, the 4
requests will be added to the active_proc->active_reqs list.

When calling spdk_nvme_detach() to detach a controller, there is a
assertion in the nvme_ctrlr_free_processes() at last to check the
active requests list of this active process data structure.

We can add a check before destructing the controller to poll the
completion queue, so that the active requests list can be flushed.

Change-Id: I0c473e935333a28d16f4c9fb443341fc47c5c24f
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/977
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-02-26 09:26:09 +00:00
Changpeng Liu
4f7fa18d69 nvme: export internal security send/receive APIs as public APIs
There are synchronous security send/receive APIs defined in nvme.h,
however, we still need the asynchronous APIs so that we can make the
OPAL library can be used in asynchronous way. As the asynchronous APIs
are already defined in nvme_ctrlr_cmd.c, so just export them to public
APIs.

Change-Id: I5646f342a4bf70faad37daa956476f05a1327bcc
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/675
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-02-26 09:22:15 +00:00
Jim Harris
39dce00bb5 test: add FreeBSD-specific ps options to get process name
Fixes issue #1145.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ifee3d8e041ca2f7acfb3e00c001e77027bf5cdb8

Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/970
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-02-26 09:16:45 +00:00
Wojciech Malikowski
8175aa1fea lib/ftl: Use fio_plugin for ftl tests
Use fio with fio_plugin instead nbd.

Change-Id: Ibd94f9de65296814c4bbe45c67f9b47a717c79ac
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/958
Reviewed-by: Maciej Szczepaniak <maciej.szczepaniak@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-02-25 10:32:17 +00:00
Pawel Kaminski
33981a6d30 test/vhost: Replace ini config with json in shared test
Change-Id: Iad98a16f3a01c68c34e23bcd880035c8c6550da4
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/765
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-02-25 10:31:49 +00:00
Michal Berger
64a55791cc test/blobfs: Use Bash's time builtin instead of the separate binary
Some distro installations don't provide the time package by default.
Since Bash is capable, use its time instead.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I8ca31e233c23d9c89be6afba29affe45f3f9882c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/961
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-02-24 08:02:48 +00:00
Michal Berger
6d230de495 test/blobfs: Don't allow run_bsdump() to fail from within the trap call
If this call fails, for whatever reason, $ROCKSDB_CONF won't be
removed.

Change-Id: I6fff996b04415fc760d3593c0f628d2631ce96ba
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/960
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2020-02-24 08:02:48 +00:00
Michal Berger
75df5fec18 test/blobfs: Dump *_db_bench.txt to stderr in case db_bench fails
Since both stdout and stderr of the db_bench run is redirected to a
file, xtrace won't write out anything of value in case said run fails.

Example:
  https://ci.spdk.io/public_build/autotest-per-patch_4071.html

To avoid looking into the file directly, try to locate last .txt that
was written out and dump its contents to stderr to increase the
visibility of what might have caused the failure.

Change-Id: Ic4873626e979130f2722a4f2d5266d93bc2bde76
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/959
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-02-24 08:02:48 +00:00
Tomasz Kulasek
03842fd950 test/nvme: fix correct controllers name in nvme-cli cuse test
Fixes issue #1223

Change-Id: I16bf739d9be54249600e135a07fdeb554c77f4cf
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/981
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-02-24 08:02:41 +00:00
Richael Zhuang
5e1b30bd28 test: fix error about unittest_nvmf uninitialised parameters
When running unittest_nvmf in fedora29 with memcheck, there are
many errors about "Conditional jump or move depends on uninitialised
value(s)". The failed tests are:
in ctrlr_ut:
  test_reservation_notification_log_page
  fused_compare_and_write
in ctrlr_bdev_ut:
  spdk_nvmf_bdev_ctrlr_compare_and_write_cmd

Signed-off-by: Richael Zhuang <richael.zhuang@arm.com>
Change-Id: Ib1e6a744e86876c15ee53206909364e853574dd1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/965
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-02-21 09:39:52 +00:00
Allen Zhu
52a0feb404 bdev/nvme: handle namespace resize event
while the size of namespace is changed,
the resize event will be notified.

Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Signed-off-by: Evgeniy Kochetov <evgeniik@mellanox.com>
Signed-off-by: Allen Zhu <allenz@mellanox.com>

Change-Id: I5d85f17df898dc21c0ae1eb9f529dcb624a457ac
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/849
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-02-21 09:38:42 +00:00
Tomasz Zawadzki
83e8cc3b90 ut/blob: always use local blobstore pointer
Always call API on pointer that was received from blobstore
initialization or load, instead of relying on global one.

Where necessary added declaration, assignment and asserts.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ic4198ac984c13a8b77428deaa32ce9864e6750c1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/942
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-02-21 09:35:27 +00:00
Tomasz Zawadzki
56cec80813 ut/blob: make use of simpler CUnit test addition
CUnit provides a helper macro CU_ADD_TEST() that
simplifies using CU_add_test() function.
Test name no longer needs to be provided,
instead test is named after test function name.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I9f5d9e0d8e72ab36b2a5c04e33c336e6c0a1fa6a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/931
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-02-21 09:35:27 +00:00
Tomasz Zawadzki
b88f0c4f20 ut/blob: simplify error handling for UT
Failure to prepare CUnit for tests is always a fatal error,
and application exits.
Most test managment functions in CUnit set some CU_ErrorCode,
that can be retrived by return code from a function or CU_get_error().

Thus usual structure of UT in SPDK checked every output from every
test managment function.
There is a helper function CU_set_error_action() that when set to
CUEA_ABORT, will abort the application run with proper error return code.
Along with nice error message.

For example if one were to add same UT twice:
"Aborting due to error #32: Test having this name already in suite."
Application return code set to 32.

Using CU_set_error_action(CUEA_ABORT), removes the need for error checking
on test managment functions in CUnit.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ida7de5e040b714509e79957d47a006ee518a42b8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/930
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-02-21 09:35:27 +00:00