lvs_load() function verifies if options passed to it
are valid, but doesn't return, if they are not (only error
is logged and callback is called with -EINVAL code). Now
it is corrected and the function ends after the error
is reported.
Change-Id: I19b0b22466b6980345477f62084d27ef13414752
Signed-off-by: Marcin Spiewak <marcin.spiewak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17582
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Mike Gerdts <mgerdts@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
When changing src/dst buffers, we copied memory domain pointers, but we
didn't copy memory domain context, which is obviously incorrect. It was
probably missed, because we never append a copy with non-NULL memory
domain. Added a unit test case to verify this behavior.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ic174e0e72c33d3f437f0faddd3405638049f0c74
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17425
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This file should be external to enable out-of-tree accel modules.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I2e973d0e88d7145d0fc9714f56db48486b00f3b7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17419
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
It always returns 0 and any errors are reported in the callback. Making
it void simplifies error handling.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I0d4299a2789a688eae38d76de46d1baf27cbbd8f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17194
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Deprecation notice for pmem was removed, as libreduce will
still use it until pmem is supported.
Change-Id: I7555dbf20a408a67fac8a6e7b2eaa23edf985eec
Signed-off-by: Marcin Spiewak <marcin.spiewak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17538
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
To make sure we don't lose statistics of destroyed channels, they're now
added to a global stats structure when a channel is destroyed.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ic3b4d285b83267ac06fad1e83721c1b15cc8ec8a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17567
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
The RPC allows the user to retrieve accel framework's statistics.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I5cd1b45686504c08eda50513ad1dae2f8d65013b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17191
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This patch adds support for collecting statistics in accel framework.
Currently, it counts the following events:
1. The number and the type of executed/failed operations.
2. The number of executed/failed accel sequences.
For now, these statistics are only collected and there's no way of
retrieving (or resetting) them - that will be added in the following
patches.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Id211067eb810e7b7d30c756a01b35eb5019c57e7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17190
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
It'll allow for setting accel-specific options. For now, it makes the
size of iobuf caches configurable.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Iaf505cc5e98dc6411453d9964250a4ba22267d79
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17188
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
In case SPDK is build with shared libraries and there's no
LD_LIBRARY_PATH around, linker will complain about missing .sos
similar to:
/usr/bin/ld.bfd: warning: librte_meter.so.23, needed by
/root/spdk/dpdk/build/lib/librte_ethdev.so, not found (try using -rpath
or -rpath-link)
We can't see that under CI since autotest_common.sh always makes sure
the LD_LIBRARY_PATH is properly set.
Add the -rpath to make the build less spammy.
Change-Id: I1d9d1775b2aa24e65cc4b776c2549457b0d7aac3
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17492
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
In commit 23baa67, we will start virtio device only once,
and update the VQ's information in SET_VRING_KICK message
context, so when multi-queues are enabled, SPDK doesn't
register VQ's interrupt handler, here we add it when enable
VQ.
Fix issue #2940.
Change-Id: I29dbd7bf0b81b23c2e47e37c467952cc5887b5bf
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17354
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
bdev modules should have call spdk_bdev_io_complete twice
for the same IO. We can help find cases where this happens
by adding an assert in spdk_bdev_io_complete - confirming
that the current status is still PENDING, before changing
it to the status passed by the caller.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Id8a044a94113f1ac5e3c8d86e426654bfa8d5c5a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17330
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reset the status for a bdev_io that fails with NOMEM status
back to PENDING immediately when it is put on the nomem_io
list, instead of waiting until it gets submitted again.
This helps keep the bdev_io states consistent, so that if
we need to complete these IO for abort reasons later, that
we aren't completing IO that already have a non-PENDING
state.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I9532095141209ed6f7af362b52c689da62e755ce
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17335
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
Handling this callback is quite complex and may lead to
various problems. In most of places, the actual event
when qpair is dosconnected is not importnat for the
app logic. Only in shutdown path we need to be sure
that all qpairs are disconnected, it can be achieved
by checking poll_group::qpairs list
Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: I453961299f67342c1193dc622685aefb46bfceb6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17165
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Michael Haeuptle <michaelhaeuptle@gmail.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
If the qpair is already in the process of disconnect,
the spdk_nvmf_qpair_disconnect API now return -EINPROGRESS
and doesn't call the callback passed by the user.
Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: If996b0496bf15729654d18771756b736e41812ae
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17164
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michael Haeuptle <michaelhaeuptle@gmail.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Current implementation of spdk_nvmf_qpair_disconnect
saves and calls user's callback correctly only on
the first call. If this function is called when
qpair is already in the process of disconnect, the
cb_fn is called immediately, that may lead to stack
overflow.
In most of places this function is called with
cb_fn = NULL, that means that the real qpair disconnect
is not important for the app logic. Only in several
places (nvmf tgt shutdown flow) that is important to
wait for all qpairs to be disconnected.
Taking into account complexity related to possible stack
overflow, do not pass the cb_fn to spdk_nvmf_qpair_disconnect.
Instead, wait until a list of qpairs is empty in shutdown path.
Next patches will change spdk_nvmf_qpair_disconnect behaviour
when disconnect is in progress and deprecate cb_fn and ctx
parameters.
Fixes issue #2765
Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: Ie8d49c88cc009b774b45adab3e37c4dde4395549
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17163
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Michael Haeuptle <michaelhaeuptle@gmail.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This patch removes usage of cb_fn argument of
spdk_nvmf_qpair_disconnect API. Instead of relying
on the callback, post a completion on delete SQ
command when transport qpair_fini is called.
Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: I68dec97ea94e89f48a8667da82f88b5e24fc0d88
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17168
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Make sure UUID is present for every bdev, even ephemeral ones.
Furthermore, this change removes assumption that bdev UUID
may remain empty.
Change-Id: I924c1ba9dedfe88a05044bb1073f28085735b1c1
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17106
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Commit aaba5d introduced a build warning with some
compilers. While fixing it, I realized the function was
difficult to immediately understand. So in addition to fixing
the build warning, I also made the following changes:
* Improved names for local variables
* Use TAILQ_INIT for local TAILQ instead of TAILQ_HEAD_INITIALIZER.
* Add comments explaining more clearly what the nested loops are
doing.
* Use TAILQ_SWAP instead of a FOREACH + REMOVE + INSERT.
Fixes: aaba5d ("subsystem: Gather list changed conditions.")
Fixes issue #2978.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ic8740b5706537938d62a0acfac62625b2424b85f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17496
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Mike Gerdts <mgerdts@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
rte_rcu is available on all versions of DPDK supported by SPDK. It is
also required by quite a few DPDK libraries. So just include
it always, it's a small library so let's not try to over-complicate by
trying to figure out exactly when it's needed.
This change fixes linking issue when crypto enabled (and vhost not).
Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: Ibdd6acb5a25c401b462022bbd94bd380690640d0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17514
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Fixed function is used to determine if it is possible to get iobuf
from the pool. To make sure that buf size alignment requirement is
satisifed value returned shall include alignment value but subtracted
by one.
e.g.
transaction size length = 64k
buffer alignment = 1 byte (no alignment requirement)
metadata length = 0
Without the fix the function returned 64k + 1, now it returns 64k
which is correct behavior and allows to proceed with further command
processing (if max buffer size limit is set to 64k only).
Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: I09104ad21b3652ba1aa5c3805a04b1c6549d04ac
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17513
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
In the presence of hardware offload (for data digest) we may not be
able to immediately release all PDUs to free a connection. Add a
state to wait for them to finish.
Fixes#2862
Change-Id: I5ecbdad394c0296af6f5c2310d7867dd9de154cb
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16637
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Fixes#2812
This patch adds support for '--lcores <map_list>'
parameter in spdk.
This parameter allow mapping of the lcores
to CPU IDs, if the system contains CPUs with IDs
greater or equal to 128 (RTE_MAX_LCORE). Such CPUs
can not be directly included in core mask
specified in '-m <mask>' parameter, as the dpdk
rejects cores if IDs are greater than 127.
The only way to use them in spdk is to map lcore
to CPU using --lcores parameters specified
in command line.
--lcores and -m parameters are mutually
exclusive. Please use only one of them.
Examples:
build/bin/nvmf_tgt --lcores 0@130
build/bin/nvmf_tgt --lcores 0@150,1@151
build/bin/nvmf_tgt --lcores "(5-7)@(10-12)"
build/bin/nvmf_tgt --lcores "(5-7)@(136,138,140)"
Change-Id: Ia92be4499c8daaa936b1a4357d52ae303d6f3eb1
Signed-off-by: Marcin Spiewak <marcin.spiewak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17403
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
This patch adds support for lcore mapping list, which
is needed by spdk if someone wants to use CPUs with IDs
greater than RTE_MAX_LCORE (128). For such CPUs it
is impossible to include them in the core mask (passed
to dpdk as '-c <mask>') as the dpdk doesn't allow
IDs greater than RTE_MAX_LCORE. Therefore they
must be mapped to lower lcore values using
'--lcores <maping_list>' passed to dpdk
Change-Id: If68f15cef2bca9e42a3457bf35477793b58ec53d
Signed-off-by: Marcin Spiewak <marcin.spiewak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17399
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
A host will consider a QID as reusable once it disconnects
from the target. But our target does not immediately
free the QID's bit from the ctrlr->qpair_mask - it waits
until after a message is sent to the ctrlr's thread.
So this opens up a small window where the host makes
a valid connection with a recently free QID, but the
target rejects it.
When this happens, we will now start a 100us poller, and
recheck again. This will give those messages time to
execute in this case, and avoid unnecessarily rejecting
the CONNECT command.
Tested with local patch that injects 10us delay before
clearing bit in qpair_mask, along with fused_ordering
test that allocates and frees qpair in quick succession.
Also tested with unit tests added in this patch.
Fixes issue #2955.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I850b895c29d86be9c5070a0e6126657e7a0578fe
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17362
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Just remove the duplicated Code and make the
Conditions for g_subsystems list to subsystems_list
together.
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Change-Id: I011b550b83d32580bfd25130dab9e44bcbdc1daf
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13753
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
If user passes --no-huge as part of env_context, do
not add other huge-related options to the EAL command
line. Instead emit an error message and return failure, if
any of them were specified explicitly.
Fixes c833f6aa ("env_dpdk: unlink hugepages if shm_id is not specified")
Fixes issue #2973.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I7aa49e4af5f3c333fa1e7dec4e3f5b4b92e7d414
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17483
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
iSCSI connection closes its socket when it is terminated. After the
socket is closed, the connection cannot access to it. However, the iSCSI
fuzz test terminated a connection while processing a text command. The
connection aborted the text command and the corresponding completion
callback accessed the closed socket. This unexpected access caused a
NULL pointer access.
Add a check if conn->sock is not NULL to iscsi_conn_params_update()
to avoid such NULL pointer access. The return type of the most iSCSI
library functions are void. Here, it is enough not to return 0. Hence,
use -ENXIO simply to indicate there is no available socket.
Fixes the issue #2958
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I2c1f58a63ee0a40561a17f81d4b4264061f411f6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17353
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Michal Berger <michal.berger@intel.com>
When bs_create_blob() is creating the internal xattr for the esnap ID,
it errors out if the ID is too long. This error path neglected to set
the return value. It now returns -EINVAL in this case.
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: I6d756da47f41fb554cd6782add63378e81735118
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17292
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Move cq_is_full() closer to its caller post_completion() and along with
fixing comments.
Signed-off-by: Swapnil Ingle <swapnil.ingle@nutanix.com>
Change-Id: I93262d1805f0f9f075c6946ed97cd3006ffba130
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16415
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
From the issue report in #2507
that comparing blob maybe be NULL.
So add assert, that in CI may catch this issue.
And other funtions add this also.
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Change-Id: I98179ec76f2b6785b6921c37373204021c0669b6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12737
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This will make it much easier to mock this library for use in unit
tests.
Change-Id: I7dc835865f75f9e29e8b709a634d30053ada2055
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16296
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This makes it much easier to mock this code in unit tests without having
to mock up the entire thread library.
Change-Id: Ic3d9cb826ae71af780a06f88669c37cef2c9a4ae
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16173
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Now that DPDK v23.03.0 has been released, the version on the main branch
points to the next release, v23.07.0-rc0, so we need to adjust the
version check to enable testing against the main branch.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I37d165111c446612d573c19199e4ace6aa24d191
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17480
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Since there were no ABI changes in the interfaces used by SPDK, the
v22.11 functions are reused.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Iff73405eec197f7ed1752366b6b38c28710a73ec
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17479
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
This allows the pipe to fill up entirely, instead of reserving 1 byte.
This tends to keep copies from the pipe aligned over time.
Change-Id: I4801d62fa839165efef61ea1c83f602931ee7018
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16990
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
If an lvol is opened by way of spdk_bs_blob_open() or spontaneously due
to blobstore activity (e.g. the blobstore is loading), the
esnap_bs_dev_create callback is called with blob_ctx equal to NULL.
Under ideal circumstances, blob_ctx refers to the lvol that is being
opened.
With this change, a NULL blob_ctx triggers lvs_esnap_bs_dev_create() to
look through the lvstore's volumes to find the one that uses a blob id
that matches the id of the blob that is passed in. Now, lvol library
consumers that need to support snapshots can count on both bs_ctx (lvs)
and blob_ctx (lvol) to be non-NULL.
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: Iaaa9cc27664e28e54f0fbd75afe1d6ffbad92580
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16588
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Once an lvol's blob is closed, the lvol should not retain a reference.
Dereferencing lvol->blob could result in a use after free.
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: Ia96a5a488fc96d642aa4cde83c6efa5cff88b068
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16426
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
As an lvstore is opening it calls spdk_bs_load(), which briefly opens
each blob and has no use for external snapshots. Since there is no point
in opening them at this time, don't open them. Once the blobstore has
been loaded, update lvs->load_esnaps so that external snapshots are
opened as the lvols open their blobs.
Change-Id: Ib16c8474300ff4b106aad0baa5b8b38332c23b01
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16424
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This provides the lib/lvol wrapper around blobstore's external
snapshots. Later commits make this work with vbdev_lvol.
The blobstore external snapshot implementation stores an opaque
identifier in an internal xattr. Lvstore uses this to store the
stringified UUID of the bdev that will act as the external snapshot.
This is used by the newly introduced spdk_lvol_create_esnap_clone() to
store the bdev UUID in the blob's metadata.
Change-Id: I58c7b32b656ad1d21a446e3b91e59e655efac7e4
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14977
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
This updates spdk_lvs_opts to be consistent with opther options
structures in that it can now be extended with additional fields.
The fields in spdk_lvs_opts are now documented as well.
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: Ibd93c3a4aa1d2a33ac550d7056a69afece4dc592
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16172
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This prepares for upcoming patch to enable retrying
CONNECT commands when a duplicate QID is detected.
Most important part is moving the spdk_nvmf_request_complete
call into the ctrlr_add_qpair function. This facilitates
deferring the spdk_nvmf_request_complete call when we want
to do a retry.
Note: for adding admin qpair, we now call
spdk_nvmf_request_complete instead of _nvmf_request_complete,
but this is actually no functional change.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I1282be00441851ee2ed3c2dd281e68b4475d3d28
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17361
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
The max qid check is only needed in the add_io_qpair
function, since an admin qpair could never fail this
check.
Upcoming patches will retry a duplicate QID check
after a short period of time. This patch helps
make the upcoming code path more clear, since there
is no need to do this max qid check more than once.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I95f69fc39ae7989d51d36b0d04b5d9a4087a7c4a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17360
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Instead of adding the spdk_fd_group's fd to the reactor fd group, use
the new nesting functionality.
Change-Id: I00727e836da6ba191d5bf778613f31956c9baacf
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15477
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Community-CI: Mellanox Build Bot
These provide a way to nest one fd_group into another in a more
efficient manner than just adding the fd_group's fd to the parent. It
also keeps track of which events belong to which group, so the unnest
operation can be implemented.
Change-Id: I63d63365f1160cce8b4b6388a0ea2003ef424b9e
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15473
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
It is fine to prefetch an invalid address, but ASAN
doesn't like it. So move the prefetch slightly to
make ASAN happy.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ib51ab8890e5fe91d30057f65e1399cfc9dd1dd49
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17432
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This function was intended to be deleted as unused, however it can be
useful for debug and test capabilities.
Its declaration was left in header file, so just adding implementation
for PCIE and VFIO USER transports.
Signed-off-by: Szulik, Maciej <maciej.szulik@intel.com>
Change-Id: I670acb53c2f88a844525a0ecea27143b055f117b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17400
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
This patch makes functions related to Asynchronous Event and error
handling public, so that they can be used in custom nvmf transport
compiled out of SPDK tree.
Signed-off-by: Szulik, Maciej <maciej.szulik@intel.com>
Change-Id: I253bb7cfc98ea3012c179a709a3337c36b36cb0e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17237
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Fixed issue indicated by Klocwork scan. 'name', which potentially
might be NULL, is passed as function parameter. Now the function
name will not point to NULL, and will be the same in interrupt
structure and in event handler.
Change-Id: I5588821139d11288a96f5041703d5b7b71890ad6
Signed-off-by: Marcin Spiewak <marcin.spiewak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17356
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
The nvmf generic transport code creates a mempool of
I/O buffers, as well as its own per-thread cache
of those buffers. The mempool was being created
with a non-zero mempool cache, effectively duplicating
work - we had a cache in the mempool and then another
in the transport layer.
So patch 019cbb9 removed the mempool cache, but the
tcp transport was significantly affected by it. It
uses a default 32 buffers per thread cache which is
very small, it was actually mostly relying on the
mempool cache (which was 512). Performance regression
tests caught this problem, and Karol verified that
specifying a higher buf_cache_size fixed the problem.
So change both the tcp and rdma transports to specify
UINT32_MAX as the default buf_cache_size. If the
user does not override this when creating the transport,
it will be dynamically sized based on the size of
the buffer pool and the number of poll groups.
Fixes: 019cbb9 ("nvmf: disable data buf mempool cache")
Fixes issue #2934.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Idd43e99312d59940ca68402299e264cc187bfccd
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17203
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
Allow transports to specify a default UINT32_MAX
as the buf_cache_size. If user does not override this
when creating the transport, calculate the buf_cache_size
dynamically using the number of poll groups and the
size of the buffer pool (num_shared_buffers). We will
allocate 75% of the buffers for the caches, meaning
the buf_cache_size will be calculated as:
(num_shared_buffers * 3 / 4) / num_poll_groups
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I97768aea701060bbe0ff1925e5322229fa8d051c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17334
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
When buf_cache_size is 0, just return early. This
allows us to un-indent a large section of code.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I167da677fdcd0504c6f2bfdb8b1a818155642f66
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17333
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This will be useful in upcoming patch, where we
use the number of poll groups to dynamically pick
the buf_cache_size for each transport poll group.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Id166098244287c56f12cdd88ba27a17fa34a4348
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17331
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Kernel `ublk_drv` driver will do memcpy to this data buffer in
unit of page size, for a simple 4KiB I/O, it may call memcpy
twice if the data buffer isn't page aligned. Moreover, SPDK
may also has double buffers with this case, so here, we use
page aligned data buffer at initialization.
Change-Id: Ica86a9702283327a2bb491e38990b8c00bc77f57
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17283
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
If CONFIG_ISAL and CONFIG_ISAL_CRYPTO are both defined, the build was
only including the LOCAL_SYS_LIBS for libisal_crypto. This fixes that
bug using the same technique used in other Makeifles.
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: I4c0869d60742cd6bdb0812d67db3abbfa7e69122
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17345
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Fixed issue indicated by Klocwork scan. 'desc->bdev'
is assigned to 'bdev' ptr, before verification that
'desc' is not NULL
Change-Id: I36e63c27b4d3220e85524133a0ec0e3521770875
Signed-off-by: Marcin Spiewak <marcin.spiewak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17350
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This refactors the code paths that call lvs_load() to allocate the
spdk_lvol_store structure before calling lvs_load(). Previously this
allocation was done in lvs_load_cb(). This is being done because a later
patch requires a pointer to the structure to be passed to lvs_load via
the spdk_bs_opts structure.
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: I2e942d1f7525fa5a16cd34b1b4b3a0a821e13006
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17220
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
The blob's parent_id and allocate_all examined and/or modified in a
two places bs_inflate_blob_open_cpl(). This transforms the two if
statements scattered around the function into a switch statement to make
it easier to understand how these two values are related.
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: I2cff2d07a0089b52678035b2ece60db6a5f67a8e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17178
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
There are multiple locations where a struct lvol_store is allocated.
This invites inconsistency in initialization, which will become more of
a problem as esnap clones have additional initialization.
Now all struct lvol_store allocations should be done with lvs_alloc().
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: I07a2f274475375072f80c25ed67cb1fb802cc4e1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16231
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
There are several places where new lvols are created and each reproduces
much of the same code. Esnap clones will add yet another in lvol.c and
more in unit tests. This introduces lvol_alloc() to minimize the chance
of unintended skew over time.
A side effect of this is that snapshots and clones now inherit clear
method from their parent. Previously they would fall back to the
default. The old behavior seems to be accidental, hence the change.
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: Ibf6f79c567e92354ea73e6589c736b1b946731a0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14976
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
The thin_provision member of struct spdk_lvol is set but never used.
When needed, an lvol's thin provision state is obtained by looking at
the lvol's blob. This removes the unused thin_provision member.
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: I5a2048b5334a26772a25a0bd238e42d3aeb63b49
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17173
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
When an esnap clone blob's external snapshot arrives after the blob is
opened, it can now be hot-added to the blob. Presumably the new device
replaces a place-holder device that did not really atteempt IO.
Change-Id: I622feb84efa66628debf44f7e7cb88b6a012db6d
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16232
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This adds the ability to abort IOs as esnap bs_dev channels are being
destroyed.
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: Ia63d4cbef5cd4c84dc8d5e2e9e407bacd961385f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16423
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
As the blobstore is being unlaoded, async esnap channel destructions may
be in flight. In such a case, spdk_bs_unload() needs to defer the unload
of the blobstore until channel destructions are complete.
The following commands lead to the illustrated states.
bdev_malloc_create -b malloc0
bdev_lvol_clone_bdev lvs1 malloc0 eclone
.---------. .--------.
| malloc0 |<--| eclone |
`---------' `--------'
bdev_lvol_snapshot lvs1/eclone snap
.---------. .------. .--------.
| malloc0 |<--| snap |<--| eclone |
`---------' `------' `--------'
bdev_lvol_clone lvs1/snap eclone
.--------.
,-| eclone |
.---------. .------.<-' `--------'
| malloc0 |<--| snap |
`---------' `------'<-. .-------.
`-| clone |
`-------'
As the blobstore is preparing to be unloaded spdk_blob_unload(snap) is
called once for eclone, once for clone, and once for snap. The last of
these calls happens just before spdk_bs_unload() is called.
spdk_blob_unload() needs to destroy channels on each thread. During this
thread iteration, spdk_bs_unload() starts. The work performed in the
iteration maintains a reference to the blob, and as such it
spdk_bs_unload() cannot do its work until the iteration is complete.
Change-Id: Id9b92ad73341fb3437441146110055c84ee6dc52
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14975
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This adds support for inflate and decouple for esnap clones. Since there
are no immediate consumers that will provide back_bs_dev->is_zeroes()
that can return true, a shortcut is taken in that inflate and decouple
of esnap clones are the same.
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: I4d2e6565126991acd650f073ce876466334e986d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11574
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
An esnap clone needs special handling as snapshots are created and
removed. In particular: the following must exist on the blob that
directly references the external snapshot and must be removed from
others:
- Ensure SPDK_BLOB_EXTERNAL_SNAPSHOT invalid flag exists only on the
esnap clone.
- Ensure BLOB_EXTERNAL_SNAPSHOT_ID internal xattr exists only on the
esnap clone.
- Clean up any esnap IO channels on a blob that is no longer an esnap
clone due to snapshot creation or removal.
See the diagrams and description in blob_esnap_clone_snapshot() in
blob_ut.c for details.
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: Ie4125d64d5bac9cfa7d6c7cc9a543d72a169f6ee
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11573
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
The channel passed to blob IO operations is useful for tracking
operations within the blobstore and the bs_dev that the blobstore
resides on. Esnap clone blobs perform reads from other bs_devs and
require per-thread, per-bs_dev channels.
This commit augments struct spdk_bs_channel with a tree containing
channels for the external snapshot bs_devs. The tree is indexed by blob
ID. These "esnap channels" are lazily created on the first read from an
external snapshot via each bs_channel. They are removed as bs_channels
are destroyed and blobs are closed.
Change-Id: I97aebe5a2f3584bfbf3a10ede8f3128448d30d6e
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14974
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
As per https://github.com/DPDK/dpdk/commit/71998eb61ff
Change-Id: Ie4e5a38976145e1037ef45593b4dc4265091482d
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17322
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Let us consider the following process:
1. one fabric connect request A comes but the subsystem is paused
due to adding/removing ns or other operations, so this request A
will be put into sgroup->queued until the subsystem becomes active;
2. the subsystem is paused for a long time until the connect timeout,
related qpair is destroyed, the sgroup->queued will not be cleaned
because qpair's ctrlr is NULL;
3. if a new request B comes, it is more likely to be allocated to the
same memory as the previous fabric command request. And it will be
put into sgroup->queued again, where has already exists the exactly
same pointer with request B.
This leads to the pointer hanging problem and it will cause infinitely
loop when traversing sgroup->queued!
So this patch avoids the ptr-hanging problem by checking and cleaning
all sgroups queued req whose qpair is the being destroyed qpair in
_nvmf_qpair_destroy when ctrlr is NULL.
This problem is already described in issue #2133.
Signed-off-by: Peng Lian<peng.lian@smartx.com>
Change-Id: I909d673b5050f21fa193914cc4ffe6634232fa7d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17147
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Add an API to easily determine if a blob is an esnap clone, similar to
what already exists for snapshot, clone, and thin_provisioned.
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: Ie07cd09b30513893e82f1c85e94a24a93c79d71e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16862
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
When a sequence is used to perform IO on an esnap clone, differenent
channels will be needed for the blobstore device and the esnap device.
No special esnap handling is required when a sequence is used to perform
IO directly on the blobstore device.
This commit splits bs_sequence_start() into bs_sequence_start_bs() and
bs_sequence_start_blob() to handle these two scenarios. A later commit
introduces special handling of ensap clone blobs.
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: I3a6f46640cdb7fdc380bf557736638f1b39f05e3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17172
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
For the various forms for read_bs_dev() and readv_bs_dev() to perform
reads from esnap devices, the spdk_bs_request_set used for the IO needs
to keep track of the back_bs_dev IO channel as well as the blobstore's
IO channel.
This commit has no change in functionality: it is preparation for a
change in a later commit.
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: I8edd9c4bf29bc074194331b42c5ef9d27590ce88
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14973
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
External snapshots have a slightly more complicated cleanup of
back_bs_dev. This moves all calls to back_bs_dev->destroy() into a
function so that this more complicated cleanup can have a single
implementation.
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: I78460aa3877481788118e2b0b76931dcf5c56338
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14972
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
When consumers open a blob with spdk_bs_open_blob_ext(), they can set
esnap_ctx in struct spdk_blob_open_opts to have that context passed
to bs->external_bs_dev_create().
Change-Id: I0c1a9cec0e5aed5ef2a7143103e822cbe400aabb
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14971
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
- fix precision
when one convert to seconds and then multiply
we can have precision errors
for example if one have 77ms, it will go to 0 when converted to seconds
and then multiply that 0 by 1000 will return 0 instead of 77ms.
- fix mismatch nsec/usec
nsec was multiplied by 1000*1000 while usec by 1000*1000*1000
it should be the opposite.
anyway the implementation had changed.
- implementation description
* env_ticks_to_msec: j / (tick_hz / 1000)
this is exactly the same as (j * 1000) / tick_hz (eq #2).
but this implementation (eq #2) can only handle 54b in j (before overflowing)
because of the multiplication by 1000 (10b).
with the correct implementation we use all 64b in j.
we assume that tick_hz will be prefectly divisible by 1000 so we are ok.
* env_ticks_to_usec: j / (tick_hz / (1000 * 1000))
same as in msec case, we use all 64b in j.
here we assume that tick_hz is perfectly divisible by (1000 * 1000)
i.e. we assume that CPU frequency is some multiple of 1MHz.
* env_ticks_to_nsec: (j * 1000) / (tick_hz / (1000 * 1000))
in this case we can't assume that tick_hz is divisible by 10^9
because there are many CPUs with 2.8GHz or 3.3GHz for example.
so we multiply j by 1000
this means that we can only handle correctly j up to 54b.
(64b - 10b, 10b for the *1000 operation)
Signed-off-by: Amir Haroush <amir.haroush@huawei.com>
Signed-off-by: Shai Fultheim <shai.fultheim@huawei.com>
Change-Id: Ia8ea7f88b718df206fa0731e3f39f419ee922aa7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17078
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
atomic64 functions should operate with atomic64 and long types.
Signed-off-by: Amir Haroush <amir.haroush@huawei.com>
Signed-off-by: Shai Fultheim <shai.fultheim@huawei.com>
Change-Id: I2ea8f1cc06d6df0f7dd5b9d628839138b78bc412
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17077
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
During resize, we correctly determine if we have enough
md_pages for new extent pages, before proceeding with
actually allocating clusters and associated extent
pages.
But during actual allocation, we were incrementing
the lfmd output parameter, which was incorrect.
Technically we should increment it any time
bs_allocate_cluster() allocated an md_page. But
it's also fine to just not increment it at the
call site at all - worst case, we just check that
bit index again which isn't going to cause a
performance problem.
Also add a unit test that demonstrated the original
problem, and works fine with this patch.
Fixes issue #2932.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iba177a66e880fb99363944ee44d3d060a44a03a4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17150
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: 阿克曼 <lilei.777@bytedance.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Community-CI: Mellanox Build Bot
If blob_freeze_io() is called twice in a row,
and the second time occurs before the for_each_channel
for the first completes, the second caller will
receive its callback too soon.
Instead just simplify the whole process, always do
the for_each_channel and don't try to optimize it
at all. These are infrequent operations - correctness
and simplicity are in order.
A few additional changes:
1) Make same changes for unfreeze path.
2) Add blob_verify_md_op() calls, just to be sure
these are only called from md_thread. This was
already checked in calling functions, but as these
functions get called from new code paths (i.e.
esnap clones) it can't hurt to add additional
checks.
3) Add unit test that failed with original code, but
passes with this patch.
Fixes issue #2935.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ibefba554547ddf3e26aaabfa4288c8073d3c04ff
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17148
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Mike Gerdts <mgerdts@nvidia.com>
Community-CI: Mellanox Build Bot
It is quite common for a user to use the exact same iovec (in memory) to
describe buffers for two different operations. If that iovec was
describing accel buffer, accel would modify it replacing it with an
actual buffer. This is broken if that iovec was used by some other task
in a sequence, as accel wouldn't be aware that it has been changed too.
To address this, accel will use a new iovec from the aux_iovs array. It
means that accel buffers always *must* be passed using a single iovec.
Theoretically, users could chunk that buffer into several iovecs, but
spdk_accel_get_buf() always returns a single buffer, so, in practice,
this should never happen, and therefore is unsupported.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I25271bc032987dd6028fb7b3adde061657759b4b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17039
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Requests that have their data pushed/pulled from a memory domain or have
an accel sequence executed aren't handled by a bdev module, so we
shouldn't submit an abort request. Those operations cannot be aborted
either, so the abort request is failed in this case.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Icd185c4a2951a555d321cd037de0af1ab157f37a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17020
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
These operations are handled internally by the bdev layer, so it should
first wait until they're completed before issuing reset to a bdev
module.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I74f0d42dcb9a289aa7c3115ca309cb92870548e2
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17019
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Similarly to requests executed by accel, we need to track bdev_ios that
have their data pushed/pulled.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ie6b0d2c058e9f13916a065acf8e05d1484eae535
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16978
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
It will make it possible to check if a request is being processed by
accel when doing resets/aborts.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ice07211df316e1eee9640e750ff8e176c8a3ca6f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16977
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
This patch enables passing accel sequence for read requests. The
handling is pretty similar to writes, but the sequence is executed after
a request is completed by a bdev module.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I79fd7d4873265c81a9f4a66362634a1c4901d0c9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16975
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
It is now possible to submit a write request with a sequence of accel
operations that need to be executed before actually writing the data.
Such requests will be directly passed to a bdev module (so that it can
append subsequent operations to an accel sequence) if that bdev supports
accel sequences and the request doesn't need to be split. If either of
these conditions are not met, bdev layer will execute all the
accumulated accel operations before passing the request to a bdev
module.
The reason for not submitting split IOs with an accel sequence is that
we would need to split that accel sequence too. Currently, there's no
such functionality in accel, so we treat this case in the same way as if
the underlying bdev module didn't support accel sequences (it's executed
before bdev_io is split).
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I66c53b3a1a87a35ea2687292206c899f80aaed4a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16974
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
bdev_io_should_split() adds some non-zero overhead, so checking it
multiple times in an IO path is inefficient. So, to avoid that, call
bdev_io_should_split() once during IO initialization and cache the
result in bdev_io.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I1da6514d409f8a4e4bbb14722dd53b2c88988cac
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17058
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
This channel will be used to execute accel operation sequences.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ied4bb57d14a50a923908ffb13ef4ba34ca65175c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16972
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Modules can now report that they support accel chaining for specific
operations through the accel_sequnce_supported() callback.
The support is reported per IO type. This allows modules to support
accel sequences for some operations, while relying on the bdev layer to
handle them for other IO types.
Only bdevs without separate metadata buffers are allowed to support this
new mode. That's because metadata in separate buffer is expected to use
the same memory domain as data buffers. With an accel sequence, those
data memory domains can change, while metadata's memory domain always
stays the same. To support bdevs with separate metadata buffers, we'd
need to add separate pointers for metadata's memory domain. For now,
simply disallow registering bdevs with separate metadata supporting
accel sequences.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I0c49cc00096837d70681a69b2633c2cb3dfd4e39
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16971
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
If an IO is completed, before submitting it to a module, it isn't put on
the io_submitted list, so we can't use bdev_io_complete() to complete
it, as it'll break that list. To avoid that, a new function was added,
bdev_io_complete_unsubmitted(), that will safely complete the IOs in
such case. For now, it's equivalent to executing user's completion
callback, but it'll serve as a good place to release any resources that
should be freed before an IO is completed.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I1442ead9d272d9210553803bed1d1c989a2bf761
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16970
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
This function can be useful in places other than accel modules (e.g. to
check if a buffer belongs to accel), so it needs to be declared in
accel.h.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I8fdd58b2ed40dc4a4acce2a8d3e1c5f76944c929
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16969
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
They were disabled before the v23.01 release, because none of the other
libraries were using the new spdk_accel_append_* API. But now, they
will be used in the bdev layer and bdev modules, so they need to be
re-enabled. We're using the same values as we do in the bdev layer.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ibda86ca5619e4104e107048ce0965171501fdc5a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16968
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
