If re-enqueue of pending crypto ops failed in crypto_dev_poller()
and DPDK reports errors then stop re-enqueue, remove the ops from
the re-submit queue and fail the IO.
Signed-off-by: Yuriy Umanets <yumanets@nvidia.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11625 (master)
(cherry picked from commit ddb443751b)
Change-Id: I258f7b8986f35fa70e4af25bc8ad2b3b26aa206b
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12667
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Kamil Godzwon <kamilx.godzwon@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
- Switched to using rte_mempool for mbufs instead of spdk_mempool. This
allows using rte pkt_mbuf API that properly handles mbuf fields we need
for mlx5 and we don't have to do it manually when sending crypto ops.
- Using rte_mempool *g_mbuf_mp in vbdev crypto ut and added the mocking
API code.
- crypto_ut update to follow pkt_mbuf API rules.
Signed-off-by: Yuriy Umanets <yumanets@nvidia.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11623 (master)
(cherry picked from commit a837ea37da
Change-Id: Ia5576c672ac2eebb260bfdbb528ddb9edcd8f036
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12666
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Kamil Godzwon <kamilx.godzwon@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
In vbdev_crypto_init_crypto_drivers() when g_session_mp init failed it
was possible to jump to cleanup label but return 0 instead of -ENOMEM.
Signed-off-by: Yuriy Umanets <yumanets@nvidia.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11659
(cherry picked from c58d5161e9)
Change-Id: I5f494ceeeca0a963e172ef3f7e55959aa407123a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11859
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
_crypto_operation_complete(bdev_io) should not be called in
_crypto_operation() because it is done by caller function
on read or write.
Signed-off-by: Yuriy Umanets <yumanets@nvidia.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11629
(cherry picked from bfb676e93d)
Change-Id: I3d754d5aba7b222e99b7035cfe1d52461fefb9a2
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11857
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
- Fixed missed spdk_bdev_module_release_bdev() during error handling.
- Fill the keys with zeros before releasing memory.
- Fixed issue with g_number_of_claimed_volumes that can become negative
because of invalid error handling.
Signed-off-by: Yuriy Umanets <yumanets@nvidia.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11628
(cherry picked from commit 3d0bae35c4)
Change-Id: I8afd6027abb4c8efc332994c5a2997b79583c174
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11856
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
ACWU and NACWU are 0-based values. But spdk_bdev_get_acwu()
specifies the compare-and-write-unit in terms of blocks
(i.e. 1-based). So the bdev/nvme module needs to add 1
to this value before registering the bdev.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12384 (master)
(cherry picked from commit ab9c7a6a81)
Change-Id: I7c19975a2bd8c09bb65374838fe20aad690d1ecf
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12491
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
When sending a fused compare and write command, we pass a callback
bdev_nvme_comparev_and_writev_done that we expect to be called twice
before marking the io as completed. In order to detect if a call to
bdev_nvme_comparev_and_writev_done is the first or the second one, we
currently rely on the opcode in cdw0. However, cdw0 may be set to 0,
especially when aborting the command. This may cause use-after-free
issues and this may call the user callbacks twice instead of once.
Use a bit in the nvme_bdev_io instead to keep track of the number of
calls to bdev_nvme_comparev_and_writev_done.
BACKPORTING NOTE:
I had to change the nvme_qpair1 name to ctrlr_ch1, because some
of the changes to multipath are not introduced to LTS version.
-Krzysztof Karas
Signed-off-by: Alex Michon <amichon@kalrayinc.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12180 (master)
(cherry picked from commit 2bc134eb4b)
Change-Id: I0474329e87648e44b08998d0552b2a9dd5d34ac2
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12489
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
We try avoiding write a closed socket by checking if the return value
of recv() is zero. However it is not possible to completely avoid writing
a socket which is already closed by the target.
Repeatedly adding/removing listener in the NVMe-oF TCP target caused
SIGPIPE to the NVMe-oF initiator.
Fix the issue by adding MSG_NOSIGINAL to the flag of sendmsg().
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12119 (master)
(cherry picked from commit 5fd1f68b63)
Change-Id: I273679c91c4b867792e966b1dc2121f6d2188f16
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12480
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
If the req's cb_fn will close the socket, there is heap-use-after-free
error if continuing to access sock.
Signed-off-by: Richael Zhuang <richael.zhuang@arm.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11855 (master)
(cherry picked from commit 3ee923eff1)
Change-Id: I88c6adb9d25e52d94b08f53e8ccac611c4d29fff
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12479
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
in bdev subsystem, if any of the bdev module fails to initialize in
bdev_modules_init(), this function just stops immediately. in general,
the non-zero rc is returned to the callback func passed to spdk_subsystem_init().
if spdk app is used for building the spdk application, it's very
likely that app_start_rpc() is used as this very callback func.
in this case, app_start_rpc() would just pass the `rc` to spdk_app_stop()
which tears down all subsystems one after another.
bdev tears itself down by calling all its modules' module_fini(),
including those whose .module_init never gets called. the problem is,
if a bdev module marks its `.async_fini` true, and it calls
spdk_bdev_module_fini_done() only if spdk_io_device_unregister(),
then a bdev module which fails to initialize would leave us an spdk
application hanging in the air.
a typical logging message sequence looks like:
[2022-02-27 20:47:13.766578] bdev.c:1438:spdk_bdev_initialize: *ERROR*: bdev modules init failed
[2022-02-27 20:47:13.766622] subsystem.c: 169:spdk_subsystem_init_next: *ERROR*: Init subsystem bdev failed
[2022-02-27 20:47:13.766638] app.c: 691:spdk_app_stop: *WARNING*: spdk_app_stop'd on non-zero
[2022-02-27 20:47:13.766658] thread.c:2050:spdk_io_device_unregister: *ERROR*: io_device 0x10d3c30 not found
this is exactly the case we could run into if a bdev module fails to
initialize and bdev_null is unable to call spdk_bdev_module_fini_done()
when being teared down, because spdk_io_device_unregister() just refuses
to call the callback if the I/O device is never registered.
since `g_null_read_buf` is set in bdev_null_initialize(), in this change,
this pointer is checked for zero before calling spdk_io_device_unregister(),
if it is NULL, spdk_bdev_module_fini_done() is called directly instead
of calling spdk_io_device_unregister(). this helps to address the
hanging issue.
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11750 (master)
(cherry picked from commit a7d174e2ef)
Change-Id: I3a41fcd2f1c986e416dacecd5ca352dfd1e379b7
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12475
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
- Reduce the size of initial memory needed by OCF.
Number of allocator buffers equal to 16383 is tested to work
on 24 caches running IO of io_size=512 and io_depth=512, which
should be more than enough for any real life scenario.
This reduces initial OCF memory usage from 726 MiB to 392 MiB.
- Fix string handling for the name of the mempool.
Signed-off-by: Rafal Stefanowski <rafal.stefanowski@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10843 (master)
(cherry picked from commit c39647df83)
Change-Id: I40063ab1897c479c25904ae4096c5dae3351f73b
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12474
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
In a test case, test/nvmf/host/failover.sh, we got ANA error even if
the target did not enable ANA reporting.
We marked the corresponding namespace as ANA state updating but we had
no way to clear it.
Check if we can read ANA log page before setting the flag.
If read ANA log page failed, disable ANA feature until the nvme_ctrlr
is created again. In this operation, all ana_state_updating flags are
cleared.
Fixes#2335
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11399 (master)
(cherry picked from commit 79829ae40b)
Change-Id: I4e2608a35d9dfa0395ad74fceebae9faf8cd973c
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12466
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
bdev_nvme_qpair() should use qpair instead of ctrlr_ch->qpair for DTRACE.
Otherwise, we get segmentation fault when enabling DTRACE.
Fixes issue #2440
The master branch fixed this bug in the commit d7f0a18 but it was in a
large patch series. So extract only the fix and apply it to the 22.01.x
branch. The 21.10.x branch also has this bug but 22.01.x is a long term
support release that will be supported for 1 year.
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reported-by: Or Gerlitz <ogerlitz@nvidia.com>
Change-Id: I458ad62c410af6ce7c228857fa9de28b2cea6282
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12086
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
This patch aligns namespace comparison with Linux kernel
implementation:
- UUID is optional and may be NULL
- command set (CSI) should be the same
Signed-off-by: Evgeniy Kochetov <evgeniik@nvidia.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11312 (master)
(cherry picked from commit 08f9b40113)
Change-Id: I8f889989f24cd51b104057217f87eb303b30fa68
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11322
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Add more dtrace probes to help with identifying issues
in production.
Change-Id: I8fb621a15c5e33ae94d75b4fc31135e2635dcfce
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10561
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This is a workaround for #2338.
Ideally the fix should remove this define and use number of cores
from the application.
With large number of QAT devices following error can be obsered:
compdev_isal_create():
ISA-L library version used: 2.30.0
vbdev_compress.c: 358:vbdev_init_compress_drivers: *NOTICE*: created virtual PMD compress_isal
EAL: memzone_reserve_aligned_thread_unsafe(): Number of requested memzone segments exceeds RTE_MAX_MEMZONE
RING: Cannot reserve memory
isal_comp_pmd_qp_setup(): Failed to create unique name for isal compression device
vbdev_compress.c: 268:create_compress_dev: *NOTICE*: FYI failed to setup a queue pair on compressdev 48 with error 4294967295 so limiting to 84 qpairs
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I689ab6bda991e3864da9f4135f57849e3c0c3986
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11179
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Historically only QAT_SYM devices for crypto
were supported. The DPDK submodule explicitly
disabled its compilation.
For details please see:
https://review.spdk.io/gerrit/c/spdk/dpdk/+/9217
Starting with DPDK 21.11 QAT_SYM and QAT_ASYM were
merged together, so it is no longer possible to
disable it QAT_ASYM as it was before.
As vbdev_crypto didn't make use of it,
this driver is now skipped in preparation for
update to DPDK 21.11.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ib606a4b450cd224d96bc21a64384297b2182967c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11178
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
When nvmf_tgt application shuts down, it stops all
subsystems, than destroyes poll groups and than
destroyes nvmf_tgt. Part of nvmf_tgt destruction is
destruction of subsystems and this process may require
cross thread communication but since poll groups and
threads are already destroyed, we may get segfaults.
One possible solution is to change the order and destroy
nvmf_tgt before destroying poll groups but it doesn't
work since nvmf_tgt is registered as io_device and
poll groups have its channel, so it can't be destroyed
while poll groups exist.
This patch adds a new state to nvmf_tgt state machine
which destroys all subsystems before destroying poll
groups and nvmf_tgt. It guarantees that all threads
exist when subsystems are destroyed.
Also rename state NVMF_TGT_FINI_FREE_RESOURCES to
NVMF_TGT_FINI_DESTROY_TARGET, the new name better
reflects the purpose of this state.
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Change-Id: I08971d78cc9ad70d43cd43c346fd74d35c8bda60
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9668
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
From SAM-4, section 5.13 (Sense Data);
“When a command terminates with a CHECK CONDITION status, sense data shall be returned
in the same I_T_L_Q nexus transaction (see 3.1.50) as the CHECK CONDITION status. After
the sense data is returned, it shall be cleared except when it is associated with a unit
attention condition and the UA_INTLCK_CTRL field in the Control mode page (see SPC-4)
contains 10b or 11b.”
SPDK does not set UA_INTLCK_CTRL to 10b or 11b, so we set the unit attention condition
immediately against a single IO or Admin IO after reporting it via a CHECK CONDITION.
Once the failed IO received at iSCSI initiator side, it will be retried. In the case of
resize operation, if there is no IO from iSCSI initiator side, the unit attention
condition will be delayed to report until the first IO is received at the iSCSI target
side.
Meanwhile, we clear the resizing (newly added) flag on our SCSI LUN structure after
first time we report the resize unit attention condition.
The kernel initiator won’t actually resize the corresponding block device automatically.
It will report a uevent, and then you can set up udev rules to trigger a rescan. SPDK
iSCSI initiator will automatically report the LUN size change.
Change-Id: Ifc85b8d4d3fbea13e76fb5d1faf1ac6c8f662e6c
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11086
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
We can just queue things up until we get -EBUSY and not track the queue
depth.
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: I49d3bcae0e6705a322de54fa91c9e1c6dfaea0c2
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11028
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Now that we have a much more robust retry framework,
set the default bdev_retry_count to 3. Users can
still override this default with the bdev_nvme_set_options
RPC as before. This ensures that by default, we will
retry I/O when possible.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I045bf4969d02be32b951e72a148ce6b6e251dec1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11107
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
We can do all of the configuration in spdk_idxd_get_channel, and the
configuration step was always done immediately after getting the channel
anyway.
Change-Id: I9fef342e393261f0db6308cd5be4f49720420aa0
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10349
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
The state of a nvme_ctrlr can be more fine grained than a boolean
and such state gives more information to end users for debug or
root cause analysis.
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I3e2459f449e2dac73f04b155e38b696495f1a335
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10183
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
If ctrlr_loss_timeout_sec is set to -1, reconnect is tried repeatedly
indefinitely, and I/Os continue to be queued.
This patch adds another option fast_io_fail_timeout_sec, a flag
fast_io_fail_timedout to nvme_ctrlr.
If the time fast_io_fail_timeout_sec passed after starting reset,
set fast_io_fail_timedout to true not to use the path for I/O submission.
fast_io_fail_timeout_sec is initialized to zero as same as
ctrlr_loss_timeout_sec and reconnect_delay_sec.
The name of the parameter follows the famous DM-multipath, its fast_io_fail_tmo.
Change-Id: Ib870cf8e2fd29300c47f1df69617776f4e67bd8c
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10301
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Previously reconnect retry was not controlled and was repeated indefinitely.
This patch adds two options, ctrlr_loss_timeout_sec and reconnect_delay_sec,
to nvme_ctrlr and add reset_start_tsc, reconnect_is_delayed, and
reconnect_delay_timer to nvme_ctrlr to control reconnect retry.
Both of ctrlr_loss_timeout_sec and reconnect_delay_sec are initialized to
zero. This means reconnect is not throttled as we did before this patch.
A few more changes are added.
Change nvme_io_path_is_failed() to return false if reset is throttled
even if nvme_ctrlr is reseting or is to be reconnected.
spdk_nvme_ctrlr_reconnect_poll_async() may continue returning -EAGAIN
infinitely. To check out such exceptional case, use ctrlr_loss_timeout_sec.
Not only ctrlr reset but also non-multipath ctrlr failover is controlled.
So we need to include path failover into ctrlr reconnect.
When the active path is removed and switched to one of the alternative paths,
if ctrlr reconnect is scheduled, connecting to the alternative path is left
to the scheduled reconnect.
If reset or reconnect ctrlr is failed and the retry is scheduled,
switch the active path to one of alternative paths.
Restore unit test cases removed in the previous patches.
Change-Id: Idec636c4eced39eb47ff4ef6fde72d6fd9fe4f85
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10128
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
This is a clean up as a preparation to the following patches.
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: Ib8bc90e17f52086d4e887463e04f65273bb1079b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11068
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
We noticed the difference between the SPDK 21.10 and the latest master
in a test.
The simplified scenario is as follows:
1. Start SPDK NVMe-oF target
2. Run bdevperf for the target with -f parameter to suppress exit
on failure.
3. Kill the target after I/O started.
With the SPDK 21.10, bdevperf retries failed I/Os and exits after
the test time is over.
With the latest SPDK master, bdevperf hungs and does not exit even
after the test time is over.
The cause was as follows:
reset ctrlr is repeated very quickly (once per 10ms by default) and hence
I/Os were queued infinitely because nvme_io_path_is_failed() returned
false if nvme_ctrlr is resetting.
We should queue I/O when nvme_ctrlr is resetting only if reset is throttoled
and fail-fast for the repeated failures is supported.
Hence in this patch, fix the degradation and remove the related unit
test cases.
Reported-by: Evgeniy Kochetov <evgeniik@nvidia.com>
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I4047d42dc44488a05264c6a841d101a7c371358b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11062
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
config_param and config_file are not conflict to specify rados configurations,
support specify both of them is more reasonable. Therefore, After this patch,
users can choose the one from the three ways: config_param, config_file + key_file
or config_param + config_file + key_file.
Signed-off-by: Tan Long <tanl12@chinatelecom.cn>
Change-Id: Ide17af72c4965df1e6541f4f50d4fa5309865486
Signed-off-by: Tan Long <tanl12@chinatelecom.cn>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10679
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: GangCao <gang.cao@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
In project practice, config_file and key_file are often used to connect
to a rados cluster, config_file includes "mon_host" and other rados
configurations like "rbd_cache", and key_file includes the secret key
and the access authority to each pool for current user. This patch adds
key_file option, user can specify config_file and key_file or only config_param
to connect rados cluster. This will make it much more flexible for users with
his/her convenience.
Signed-off-by: Tan Long <tanl12@chinatelecom.cn>
Change-Id: I6b49aad70b578bdeb3ac8ea9ca0fcbd931582025
Signed-off-by: Tan Long <tanl12@chinatelecom.cn>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10485
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: GangCao <gang.cao@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
If acceleration tasks are exhausted, then we can exit
the submission loop earlier, also print number of IOVs
for each R/W request.
Change-Id: Ia98ed43b0bb2be229b7c0054f3ade0ad39337b09
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10836
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
This uses a batch with the fence flag for now. There are several other
implementation options that will be explored in the future.
Change-Id: I4f344d671400508de05f80b026d42f775c5b9588
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10289
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Compare two scattered memory regions
Change-Id: I6ce5c9e7bc1ee1ef0e9173c00e86628d43a1e41f
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10287
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
This RPC will stop the specified discovery service,
including detaching from any controllers that were
attached as part of that discovery service.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I9222876457fc45e1acde680a7bd1925917c22308
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10832
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Requests that are completed immediately (i.e. those not using the accel
engine) are now queued and their completion is delayed to the completion
poller. It ensures that they're not completed from the context of a
submission, which gets rid of an spdk_thread_send_msg() call.
It significantly improves performance on some workloads. For instance,
4k zcopy reads (queue depth 128) on an malloc bdev exposed through
NVMe/TCP went from 204k IOPS to 485k IOPS.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I196f55fc07d167f1ed117d2430e9c37f9d05f70d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10805
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
The only thing these functions were doing was completing the IO, so it
could just be inlined.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I5fbd9df763dd68953b1bda9c7752c57ef9ee5dd6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10804
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This poller is registered on each IO channel and can be used to schedule
asynchronous completion of a request. This can be especially useful for
requests that can be completed immediately. For now, nothing enqueues
the requests to be completed through this poller - this will be changed
in the following patch.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: If6b26541907bb46402fc0904216bff74dad57b88
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10803
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
It'll allow the malloc bdev to store per-thread data. For now, it's
only used to keep the pointer to the accel library's IO channel, more
fields will be added in subsequent patches.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I604a38877ae8d6075b911f5a484d1793d4bc2ddb
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10802
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This patch adds support for zero-copy operations in the delay bdev.
They use the same delay values as regular IO operations:
- (avg|p99)_read_latency for zcopy_start with populate=true,
- (avg|p99)_write_latency for zcopy_end with commit=true.
All other zcopy operations (e.g. zcopy_start with populate=false) are
not delayed.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I8b32c1d99f9f2b36b16617122881ea95d02ecc87
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10798
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
If a path whose namespace is optimized is restored, the corresponding
I/O path cache should be cleared and the path should be chosen as the
optimal path.
This bug was found by a system test.
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: Ibc3983dbff3418adb090a09df32c2a92a8910d05
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11004
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Rename a few functions for a full ctrlr reset sequence to
clarify what we do and make the following patches easier.
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I051e3ab68c3cd77fd6040a2d069d50a700123ae6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10920
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
We sort secondary trids to avoid using disconnected trids for failover.
However the sort had a bug.
This bug was found by running test/nvmf/host/multipath.sh in a loop.
Verify the fix by adding unit test.
Fixes#2300
Signed-off-by: Shuhei Matsumoto <shuheimatsumoto@gmail.com>
Change-Id: I22b0ede4d2ef98b786c3e0d1f5337a2d568ba56d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10921
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
This patch adds the framework for a discovery
service in the bdev/nvme module.
Users can specify an IP/port of a discovery service.
The bdev/nvme module will connect to a discovery
controller, get the discovery log page, and then
register for AERs. It will connect to each
subsystem specified in the initial log page.
AER completions will trigger fetching the log
page again, at which point new subsystems will
be connected to, or removed subsystems will be
detached.
This patch does the following:
* Adds the new start_discovery RPC
* Connects to the discovery controller
* Gets the discovery log page
* Registers for AERs
* Detach from discovery controllers at shutdown
Subsequent patches in this series will:
* Connect to subsystems listed in discovery log page
* Detach from subsystems that were listed in earlier
discovery log pages but subsequently removed
* Add a stop_discovery RPC
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I54bfa896a48c5619676f156b5ea9f2d1f886c72f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10694
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Replace total_busy/idle_tsc by current ones for
decision making when tuning frequency.
Change-Id: I89524a9febfa963b14c3120433e5aff9de2a28dd
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10342
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
