When accel task is processed is processed in
several iterations (submit part of cryops, wait
for completion and submit next part of cryops),
sgl is initialized with offset to exclude previously
processed blocks. However there is a bug since
spdk_iov_sgl_init doesn't advance iovs, as result
when we do sgl->iov->iov_len - sgl->iov_offset,
we may get unsigned int underflow.
Fix is init sgl with 0 offset and then advance it
with offset.
Modified unit test and added an assert in code to
verify this fix.
Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: Ib53ff30f0c90d521f2cf6b3ec847b0d06869c2b5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17456
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
When RPC server is used for configuration, rpc_cmd
function waits 15 seconds to read all replies. If
mlx5 dpdk driver is used on slow machines or in
container, RPC framework_start_init may take more
than 15 seconds to execute. As result, rpc_cmd
exits earlier and output of some comamnds
remains in the pipe. Next call of rpc_cmd may
read wrong data, that leads to malformed json
config. To avoid this problem, redirect RPCs to
a regualr rpc socket.
Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: Ibfcf56bb0a7f84f69394846d83746c91a4024b9e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16389
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
There's no point in keeping these separate as dnf-aware distros
also support yum and there are no plans to drop it anytime soon.
In fact, since the actual list of packages between dnf and yum
was different, the centos7 was not provisioned to the full extent.
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: Ieec6796bf457d37b2618a1c2756d281f4af0c5b4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16931
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
The new name puts more emphasis on what is the main designation of
the script as it does not really touch anything VM-related.
The vm_setup.sh is preserved as a symlink available for a transition
period.
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I968a52cc069706f4c5e1b8a871988809e701a3fe
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16928
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Most of the information gathered there is outdated and generic and
falls out of scope of what vm_setup.sh/pkgdep is actually doing.
Simply mention the main purpose of the script, leaving actual
configuration to the user.
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I126515dea019e7f1cd76c8be1339aea080d2a2b0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16927
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
We no longer support ubuntu's Xenial and Bionic flavors so they can
be removed.
swupd, Clearlinux's package manager, is also no longer supported.
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I526a89f4d3b3078949f235e46f8bb3a39b2a24b6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16926
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Some minor code shuffle + removal of the autorun-spdk.conf creation.
Creating this config has little sense as some of these flags cannot
be used together anyway - it basically serves as a dump of all
supported flags which we usually are having a hard time to keep up
to date. That said, autotest_common.sh (and get_config_params())
gives a better view as to what flags are actually supported and
how they are used in practice.
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: Ib223ec90be58e68ecab69176d213c353df530498
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16925
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Signed-off-by: Kuo Kan <kuox.kan@intel.com>
Change-Id: Ib3d33cefc78f543e157ea552ee88f0514e305054
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15795
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I313d65b01c9214a6bde5775488fb32c70cefa4d6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15357
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This is the location freebsd_update_contigmem_mod() looks up to copy
the modules into right /boot directories.
Signed-off-by: Michal Berger <michallinuxstuff@gmail.com>
Change-Id: Ic5919cc6382433c641c4c7a8b1100a50abfc246a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12925
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Use it in all of the places that were previously hooking
spdk_mempool_get.
Change-Id: I311f75fb9601b4f987b106160eb0a0014d3327cd
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16329
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This makes it much easier to mock this code in unit tests without having
to mock up the entire thread library.
Change-Id: Ic3d9cb826ae71af780a06f88669c37cef2c9a4ae
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16173
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
This OCF update is mainly focused on:
- New volume API
- New cache attach config instead of old cache device config
- UUID moved to different struct
- Persistent metadata is not supported due to metadata flapping (see 689c44c76ba87f80a9538c17220bb9ca6bffdda0 in OCF)
Signed-off-by: Amir Haroush <amir.haroush@huawei.com>
Signed-off-by: Shai Fultheim <shai.fultheim@huawei.com>
Change-Id: Ic3bc0f1b58550dc3b03b0afc9bcb43b2b9b988c6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17066
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Robert Baldyga
This allows the pipe to fill up entirely, instead of reserving 1 byte.
This tends to keep copies from the pipe aligned over time.
Change-Id: I4801d62fa839165efef61ea1c83f602931ee7018
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16990
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
As an lvstore is opening it calls spdk_bs_load(), which briefly opens
each blob and has no use for external snapshots. Since there is no point
in opening them at this time, don't open them. Once the blobstore has
been loaded, update lvs->load_esnaps so that external snapshots are
opened as the lvols open their blobs.
Change-Id: Ib16c8474300ff4b106aad0baa5b8b38332c23b01
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16424
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This provides the lib/lvol wrapper around blobstore's external
snapshots. Later commits make this work with vbdev_lvol.
The blobstore external snapshot implementation stores an opaque
identifier in an internal xattr. Lvstore uses this to store the
stringified UUID of the bdev that will act as the external snapshot.
This is used by the newly introduced spdk_lvol_create_esnap_clone() to
store the bdev UUID in the blob's metadata.
Change-Id: I58c7b32b656ad1d21a446e3b91e59e655efac7e4
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14977
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
This may have been needed in some previous iteration
of this duplicate QID test, but it's no longer required.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I5c788a50d9c87cec7268ebb6f64d91fdc7b00873
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17369
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
We need to be checking whether the specified qid
is too large or not - the current test says it is
doing that in the comments but is really just
checking the duplicate QID path (which following
test already covers).
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I21dac3c7ec83a5c8e31fac77481708d54cb40e9e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17368
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
The 2 MiB minimum may be not reasonable for some users, so it is changed
to 4 KiB to allow wider range of possible values.
The new default is introduced to keep backward compatibility (2 MiB).
Signed-off-by: Szulik, Maciej <maciej.szulik@intel.com>
Change-Id: I450ff555f73ddd9be727ecc49209eb5af90fa88e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17406
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Community-CI: Mellanox Build Bot
Needed for the recent changes in the 6.2 kernel.
Also, add an extra patch to address changes in the net/core not
available in the out-of-tree driver.
Change-Id: I2e279a7728bbb462da99b79fad6a91ebd97ecae7
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17337
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Kamil Godzwon <kamilx.godzwon@intel.com>
Needed for the recent changes in the 6.2 kernel.
Change-Id: I1788b11712dc581520ddcc5382894652be3a5576
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17336
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Kamil Godzwon <kamilx.godzwon@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This application can be used to test AER flows related to doorbell
errors.
Signed-off-by: Szulik, Maciej <maciej.szulik@intel.com>
Change-Id: Idc04d326f08f8e04455c77ab8265cc601485afbe
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17401
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
This patch makes functions related to Asynchronous Event and error
handling public, so that they can be used in custom nvmf transport
compiled out of SPDK tree.
Signed-off-by: Szulik, Maciej <maciej.szulik@intel.com>
Change-Id: I253bb7cfc98ea3012c179a709a3337c36b36cb0e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17237
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
unittest_sock unit tests fail on FreeBSD due to
ENOBUFS. Increase to buffer size to 4k for tests
to pass.
Fixes#2943
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I55d1164d49edb2d90ebfe5f720a615f052ee52d9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17370
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
For some systems, depending on the mem distribution, 2GB per node may be too much. Just lower it to 1GB as the only thing we are verifying here
is HUGENODE usage in this test.
Also, make sure we always log output from setup.sh.
Change-Id: If7c382b442e9523fe5ad5df03110a8cc5467f1d9
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17367
Reviewed-by: Jaroslaw Chachulski <jaroslawx.chachulski@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
This test scenario does the following:
- Attaches the controller to NVMe device.
- Injects an error timeout for admin command SPDK_NVME_OPC_GET_FEATURES
- Runs controller reset.
In this scenario the call for 'controller reset' should abort any outstanding requests that are waiting for completion defined in 'error injection' RPC call. Test fails if request(s) are not aborted (manually completed).
Signed-off-by: Yuri Kirichok <yuriy.kirichok@hpe.com>
Change-Id: I512de92e948d04cad538db378b44fd4d58d85bb3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17113
Reviewed-by: Michael Haeuptle <michaelhaeuptle@gmail.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
This will be useful in upcoming patch, where we
use the number of poll groups to dynamically pick
the buf_cache_size for each transport poll group.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Id166098244287c56f12cdd88ba27a17fa34a4348
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17331
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
If the controllers are left attached when the application is terminated,
the virtio library might receive notifications about memory being
unregistered from the mem_map. The apps in json_config test are
preallocating 1GB of memory, which should stop DPDK from releasing the
hugepages back to the system, but if they allocate more, that extra
pages can be released, which would generate the mem_map notifications .
This actually happened in #2951 - spdk_tgt was built with ocf support,
which preallocates an extra ~300MB mempools, exceeding 1GB, which caused
mem_map notifications to be sent when iscsi mempools were destroyed
triggering an assertion in virtio.
Another option to fix this would be to increase the size of the
preallocated memory, but detaching the controllers achieves the same
thing without requiring more resources.
Fixes#2951.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I743af39acfee68caba55ec4fccd593a696ab688b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17265
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
It fixes the following error under clang:
/usr/bin/ld.bfd: bdev_nvme_ut.o: in function `bdev_nvme_get_mdns_discovery_info':
/home/ksztyber/src/spdk/module/bdev/nvme/bdev_mdns_client.c:578: undefined reference to `spdk_jsonrpc_begin_result'
/usr/bin/ld.bfd: /home/ksztyber/src/spdk/module/bdev/nvme/bdev_mdns_client.c:599: undefined reference to `spdk_jsonrpc_end_result'
It probably works on gcc, because the bdev_nvme_get_mdns_discovery_info
isn't called anywhere in the unit test and gets optimized away.
Fixes#2950
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I2bb856afbc1a1a7e882dec72e4182ad9d3b9b24a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17260
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
It fixes the following error under clang:
/usr/bin/ld.bfd: vfio_user_ut.o: in function `nvmf_transport_create_async_done':
/home/ksztyber/src/spdk/lib/nvmf/transport.c:203: undefined reference to `spdk_mempool_lookup'
It probably works on gcc, because the nvmf_transport_create_async_done
isn't called anywhere in the unit test and gets optimized away.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I497aae4d211d556a531455481ee9a69067988cce
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17259
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
There's no reason to check the OS to decide whether to build some unit
tests if we can rely on the configure parameters. For instance, on
FreeBSD, RDMA unit tests can be built and run successfully, while vhost
is always disabled in configure.
Fixes#2941.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I0073783dcafa932f3a7f48fa553e04be7ae44f84
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17182
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
When an esnap clone blob's external snapshot arrives after the blob is
opened, it can now be hot-added to the blob. Presumably the new device
replaces a place-holder device that did not really atteempt IO.
Change-Id: I622feb84efa66628debf44f7e7cb88b6a012db6d
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16232
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
To avoid races that lead to use-after-free errors during esnap device
hot add/remove, we need a way to ensure that the destroy callback does
not free a bs_dev until all consumers are done.
This adds reference counting to the create_channel() and
destroy_channel() callbacks. The reference couunt is initialized to 1
and is decremented by destroy(). The destroy() and destroy_channel()
callbacks are updated to free the bs_dev only when the reference count
drops to 0.
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: Ie0b873717e431b33ce6548f878643dbc66d4f956
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16422
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
As the blobstore is being unlaoded, async esnap channel destructions may
be in flight. In such a case, spdk_bs_unload() needs to defer the unload
of the blobstore until channel destructions are complete.
The following commands lead to the illustrated states.
bdev_malloc_create -b malloc0
bdev_lvol_clone_bdev lvs1 malloc0 eclone
.---------. .--------.
| malloc0 |<--| eclone |
`---------' `--------'
bdev_lvol_snapshot lvs1/eclone snap
.---------. .------. .--------.
| malloc0 |<--| snap |<--| eclone |
`---------' `------' `--------'
bdev_lvol_clone lvs1/snap eclone
.--------.
,-| eclone |
.---------. .------.<-' `--------'
| malloc0 |<--| snap |
`---------' `------'<-. .-------.
`-| clone |
`-------'
As the blobstore is preparing to be unloaded spdk_blob_unload(snap) is
called once for eclone, once for clone, and once for snap. The last of
these calls happens just before spdk_bs_unload() is called.
spdk_blob_unload() needs to destroy channels on each thread. During this
thread iteration, spdk_bs_unload() starts. The work performed in the
iteration maintains a reference to the blob, and as such it
spdk_bs_unload() cannot do its work until the iteration is complete.
Change-Id: Id9b92ad73341fb3437441146110055c84ee6dc52
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14975
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This adds support for inflate and decouple for esnap clones. Since there
are no immediate consumers that will provide back_bs_dev->is_zeroes()
that can return true, a shortcut is taken in that inflate and decouple
of esnap clones are the same.
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: I4d2e6565126991acd650f073ce876466334e986d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11574
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
An esnap clone needs special handling as snapshots are created and
removed. In particular: the following must exist on the blob that
directly references the external snapshot and must be removed from
others:
- Ensure SPDK_BLOB_EXTERNAL_SNAPSHOT invalid flag exists only on the
esnap clone.
- Ensure BLOB_EXTERNAL_SNAPSHOT_ID internal xattr exists only on the
esnap clone.
- Clean up any esnap IO channels on a blob that is no longer an esnap
clone due to snapshot creation or removal.
See the diagrams and description in blob_esnap_clone_snapshot() in
blob_ut.c for details.
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: Ie4125d64d5bac9cfa7d6c7cc9a543d72a169f6ee
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11573
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
The channel passed to blob IO operations is useful for tracking
operations within the blobstore and the bs_dev that the blobstore
resides on. Esnap clone blobs perform reads from other bs_devs and
require per-thread, per-bs_dev channels.
This commit augments struct spdk_bs_channel with a tree containing
channels for the external snapshot bs_devs. The tree is indexed by blob
ID. These "esnap channels" are lazily created on the first read from an
external snapshot via each bs_channel. They are removed as bs_channels
are destroyed and blobs are closed.
Change-Id: I97aebe5a2f3584bfbf3a10ede8f3128448d30d6e
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14974
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Also, drop some code that's essentially not needed to improve
readability.
Change-Id: I5fda1c179983ce905661918e6f821eca486bb58d
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17218
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Try to create virtio blk transport. Transport is
created at application start, so EEXIST error is
expected.
This is aimed purely at using
rpc_virtio_blk_create_transport() from vhost_rpc.c
to increase code coverage.
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: Iba87ef73f8d5a962bf5eac28c7e6c93bdb65b67f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16436
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
On FreeBSD spdk_uuid_parse() will fail if the UUID doesn't have the
variant/version fields filled in correctly. So, use a proper UUID in
the test to make them pass on FreeBSD.
Fixes#2942.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I7c5384086c6828c7380f9d128878578301077b17
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17184
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
As per https://github.com/DPDK/dpdk/commit/71998eb61ff
Change-Id: Ie4e5a38976145e1037ef45593b4dc4265091482d
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17322
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
The change in question is already available upstream.
Change-Id: I26a641f8314a3292d57124056ac197e1f861685d
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17321
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Asan reports memory leaks from CRYPTO_zalloc which
seem to be caused by openssl library.
See https://github.com/spdk/spdk/issues/2947
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I964acc8ae8c3a6978ea01a0235a32661ffbc74c0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17217
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
xfsprogs refuse to create fs smaller than 300MB since:
https://git.kernel.org/pub/scm/fs/xfs/xfsprogs-dev.git/commit/?id=6e0ed3d19
To make sure mkfs does not fail, increase bdev size to 512MB.
Change-Id: Ic313db06cfab43f13f43a269c9cc058391bfb8ad
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17214
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Add an API to easily determine if a blob is an esnap clone, similar to
what already exists for snapshot, clone, and thin_provisioned.
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: Ie07cd09b30513893e82f1c85e94a24a93c79d71e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16862
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
When consumers open a blob with spdk_bs_open_blob_ext(), they can set
esnap_ctx in struct spdk_blob_open_opts to have that context passed
to bs->external_bs_dev_create().
Change-Id: I0c1a9cec0e5aed5ef2a7143103e822cbe400aabb
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14971
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This job is already done by the target test. Also, this makes sure
that nvme_ns_manage_cuse is always executed (note that we don't
consider it as a test to be skipped).
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I6d964230e4d961152912b1f8a0636712039eeae6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17080
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Use it in the nvme_cli_cuse to make sure proper drive was selected
for the test (i.e. supporting ns management) - some of the nvmes
installed in the CI pool don't support ns management, hence all
related commands fail as part of these tests.
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: Ib93933d7ee9b0d2eb283313d3f2bbb6f4af727a4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17079
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
They have exactly the same conditions, no need to keep it separate
for now.
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I6826ffe7b70e3d502f483cf1966930586a6f9f0b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16934
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Richael <richael.zhuang@arm.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
In order to do so we need to make sure that freq is lowered for
all thread siblings of a given core. Since DPDK and/or dynamic
scheduler do not take that into the account we need to do this
on our own.
Find thread sibling of the main cpu and imitate the DPDK's governor
work by adjusting its freq settings.
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I154a2a789903b66c2722160d7e252221083f5e3c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16930
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
During resize, we correctly determine if we have enough
md_pages for new extent pages, before proceeding with
actually allocating clusters and associated extent
pages.
But during actual allocation, we were incrementing
the lfmd output parameter, which was incorrect.
Technically we should increment it any time
bs_allocate_cluster() allocated an md_page. But
it's also fine to just not increment it at the
call site at all - worst case, we just check that
bit index again which isn't going to cause a
performance problem.
Also add a unit test that demonstrated the original
problem, and works fine with this patch.
Fixes issue #2932.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iba177a66e880fb99363944ee44d3d060a44a03a4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17150
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: 阿克曼 <lilei.777@bytedance.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Community-CI: Mellanox Build Bot
If blob_freeze_io() is called twice in a row,
and the second time occurs before the for_each_channel
for the first completes, the second caller will
receive its callback too soon.
Instead just simplify the whole process, always do
the for_each_channel and don't try to optimize it
at all. These are infrequent operations - correctness
and simplicity are in order.
A few additional changes:
1) Make same changes for unfreeze path.
2) Add blob_verify_md_op() calls, just to be sure
these are only called from md_thread. This was
already checked in calling functions, but as these
functions get called from new code paths (i.e.
esnap clones) it can't hurt to add additional
checks.
3) Add unit test that failed with original code, but
passes with this patch.
Fixes issue #2935.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ibefba554547ddf3e26aaabfa4288c8073d3c04ff
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17148
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Mike Gerdts <mgerdts@nvidia.com>
Community-CI: Mellanox Build Bot
This function can be used to kill all lingering vhost and qemu
processes that might have been left behind by vfio-user|vhost
tests.
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I17121f416e26732d53b5dfec0f010e5db8bd369f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15186
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Also, add at_app_exit() to make sure that all vhost and qemu
instances are terminated as well.
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I917f1cb1b02d18b78a482d754a19f509e580fc98
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15185
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Now that all crypto operations are using interfaces that support memory
domains, bdev_crypto can report support for memory domains.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I13a128a599f6560197fed3405599c2a6bb609703
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17041
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
This will stress the accel chaining code just a little bit more.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: If08625b8606a021a3494eb4aa8527d2b338bd255
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17040
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
It is quite common for a user to use the exact same iovec (in memory) to
describe buffers for two different operations. If that iovec was
describing accel buffer, accel would modify it replacing it with an
actual buffer. This is broken if that iovec was used by some other task
in a sequence, as accel wouldn't be aware that it has been changed too.
To address this, accel will use a new iovec from the aux_iovs array. It
means that accel buffers always *must* be passed using a single iovec.
Theoretically, users could chunk that buffer into several iovecs, but
spdk_accel_get_buf() always returns a single buffer, so, in practice,
this should never happen, and therefore is unsupported.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I25271bc032987dd6028fb7b3adde061657759b4b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17039
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
All these callbacks look identical now, so there's little point in
having three different functions doing the same thing.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I60d1426b5d2b20d924776699885e6a9dd176504a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17024
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
After changing to spdk_accel_append_*, this code is no longer necessary,
as accel operations are now executed by the base bdev/bdev layer.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I2e31249dfdf7abeee420744a5cc75b8273ecbad9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17023
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Similarly to reads, writes path is now also using the spdk_accel_append*
interface for performing encrypt operation.
Additionally, this patch also changes the way aux buffer is allocated -
spdk_bdev_io_get_aux_buf() was replaced with spdk_accel_get_buf(). This
ensures that the actual data buffer will be only allocated if it's
actually needed.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I3cd1d4f5753a95709d7b81de23d9227102a74261
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17022
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
The new spdk_accel_apend_decrypt() interface is now used for all read
requests for decrypting data. This makes it possible to chain decrypt
operations with other operations in a sequence. Support for encryption
in the write path will be added in the following patches.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I2f65044eeaf92232d839ddd166ae50889b3f386d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17021
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
This patch enables passing accel sequence for read requests. The
handling is pretty similar to writes, but the sequence is executed after
a request is completed by a bdev module.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I79fd7d4873265c81a9f4a66362634a1c4901d0c9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16975
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
It is now possible to submit a write request with a sequence of accel
operations that need to be executed before actually writing the data.
Such requests will be directly passed to a bdev module (so that it can
append subsequent operations to an accel sequence) if that bdev supports
accel sequences and the request doesn't need to be split. If either of
these conditions are not met, bdev layer will execute all the
accumulated accel operations before passing the request to a bdev
module.
The reason for not submitting split IOs with an accel sequence is that
we would need to split that accel sequence too. Currently, there's no
such functionality in accel, so we treat this case in the same way as if
the underlying bdev module didn't support accel sequences (it's executed
before bdev_io is split).
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I66c53b3a1a87a35ea2687292206c899f80aaed4a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16974
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
This channel will be used to execute accel operation sequences.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ied4bb57d14a50a923908ffb13ef4ba34ca65175c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16972
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
We *really* don't need to fill the whole bus with virtio-blk
controllers, the intention is just to verify that they can be created on
two different buses. So, instead of looping through 33 devices, specify
two addresses that will translate to bus 0 and bus 1.
This significantly reduces the amount of necessary resources needed for
this test.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I6d60bd0fa840596d2231f50fc10ccde6b47ac60e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17032
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
we can get sessions information by vhost_get_controllers
Signed-off-by: zhipeng Lu <luzhipeng@cestc.cn>
Change-Id: I8e63aea64d02b3467a62f30a712e1dcbf6fb8854
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16315
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Community-CI: Mellanox Build Bot
When a blobstore consumer creates or loads a blobstore, it should be
able to set a per-blobstore context pointer that will be passed back to
the consumer via bs->esnap_bs_dev_create().
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: I59c0ebe21eaf65c3d79a4ac3469715283f56313a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14970
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This may cause unwanted intermittent issues depending on how long
dnf syncs its repos - on occasion it takes too much time leading
to job timeouts under the CI. As an alternative, look under a specific
path to see if dpdk{,-devel}.rpm are already in place and install
them locally - VM images on the CI side will be already provisioned
with all needed dependencies.
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: Ic924574889b52d521db4799cbb12b4c238a2fe3a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16304
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
This structure can be allocated on the stack
where needed
Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: I92b7e777d4718c5f2dd1c09e8f1bed8fd6e6caf8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17075
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
nvme_ctrlr_disable() and nvme_ctrlr_disable_poll() were added almost a
year ago but they did not have any unit test.
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I4d9abe37e5aea18893509776b54de495a407ba67
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17000
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Michael Haeuptle <michaelhaeuptle@gmail.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Add command dword 13 field to the extendable structure
spdk_nvme_ns_cmd_ext_io_opts. This now enables us to pass dspec
and dsm fields.
Signed-off-by: Ankit Kumar <ankit.kumar@samsung.com>
Change-Id: Id4d3dac14fdbf0e2a57e0bf287551dfd827dd503
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16945
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Community-CI: Mellanox Build Bot
Removed bdev pmem implementation and makefiles
Removed bdev pmem rpc calls
Updated Python scripts
Updated match files for tests
Updated doc files
pkgdep and doc files will be updated when pmem
dependency is removed from libreduce
Change-Id: Ia1be5046159e04cd933ac13a0d5791e6d28219da
Signed-off-by: Marcin Spiewak <marcin.spiewak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17070
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Add default copy command support in bdev layer for backing devices that
does not support copy command.
Signed-off-by: Rui Chang <rui.chang@arm.com>
Change-Id: I5632e25544e95ac0c53ff91c4cd135dac53323ae
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16638
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
When IBV_EVENT_DEVICE_FATAL & RDMA_CM_EVENT_DEVICE_REMOVAL occurs,
destroy qpair immediately and do no assume that no successful WQE will
be received after rdma_disconnect.
Signed-off-by: sijie.sun <sijie.sun@smartx.com>
Change-Id: I23e44dd32c8adea301e5251659b1be519f5dfdf7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16314
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Community-CI: Mellanox Build Bot
IB device may be unplugged & hotplugged when modifying slaves of bonding
IB devices. This patch will try to recreate ibv device contexts, poller
and listeners after IB devices come back.
Signed-off-by: sijie.sun <sijie.sun@smartx.com>
Change-Id: I3288174bad847edc2d9859cb34aa93c6af8c673b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15616
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Patch 55f947933 ("bdev: remove spdk_bdev_ext_io_opts from spdk_bdev_io")
changed the way bdev_nvme submits IO to the NVMe driver causing
performance degradation for requests with iovcnt = 1, as they also had
to go through the path that executes the reset_sgl/next_sge callbacks.
This patch reverts those changes back to the original code checking
iovcnt and using the non-SGL functions if possible.
Suggested-by: Jim Harris <james.r.harris@intel.com>
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I5e7c6620d38b7690ff862d8cd0075afacc578217
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16961
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
This is the beginning of support for external snapshots. An external
snapshot is a read-only blobstore device (struct spdk_bs_dev) that can
be used as a blob's back device. Normally a blob will have no back
device (a normal blob), a zeroes back device (a thin provisioned blob),
or a blob back device (a clone blob). When a blob has an external
snapshot ("esnap") as its back device, it is called an esnap clone.
With this patch, esnap clones can be created but they are not yet
useful. Subsequent patches in the series will plumb the IO path, enable
various features, and allow lvol bdevs to be esnap clones.
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: I29206b628a2b03b6386a88532565e228df988e0e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14969
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Perform all tests on devices that do and do not support
spdk_bs_dev::copy.
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: Ic4c13ade9f45709b34a57f9fb7456d6c6a790a85
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16691
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
This reworks how blob_snapshot_rw() tracks the number of bytes read and
written. It has no functional change: it simply makes the patch that
follows less complex.
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: Ieeb738b6a814e7939931fecdfaf14b9f162d8431
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16861
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
If the bs_dev was opened read-write, continue to take a
read-many-write-one claim. If it was opened read-only, take a
read-many-write-none claim.
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: I25d977c6961f962423899fb891ec912cd847930a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16282
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
External snapshots, which will be introduced in a later commit, will
need read-only blob_bdev instances. This support is partially needed to
support underlying devices that are naturally read-only and partially to
provide an extra layer of protection against accidental writes.
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: Ibcb28d00ad644a6053aa5f4de15471c2cd8e348a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14968
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This patch removes references to deprecated PMEM from accel library.
The code that was executed when ACCEL_FLAG_PERSISTENT flag is set,
is no longer needed and is removed.
_sw_accel_copy() function is removed and replaced with memcpy(), as
after PMEM removal its functionality is the same as memcpy().
_sw_accel_dualcast() is no longer needed, replaced with direct calls
to memcpy()
Removed 'flags' parameter - it is no longer needed
accel_ut.c: removed references to PMDK
deprecation.md updated
ACCEL_FLAG_PERSISTENT flag will be removed in next patch.
Change-Id: I86130466fe7a5f6ee547df1517b803035ff41a7a
Signed-off-by: Marcin Spiewak <marcin.spiewak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16899
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
1.We should get current scaling governor from sysfs for comparison
2.The file name is scaling_governor not scaling_governors
Change-Id: I758b88d6b05a5d4b58bd103bfec8dfcb64453a5f
Signed-off-by: Richael Zhuang <richael.zhuang@arm.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16863
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michal Berger <michal.berger@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Added cppc_cpufreq case in set_cpufreq.
Change-Id: I8a7ceeb2891a9464a858b0d4ed1e3350b7ad1dc6
Signed-off-by: Richael Zhuang <richael.zhuang@arm.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16802
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Michal Berger <michal.berger@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
idle.sh sometimes fails because the app_thread's idle time is less
than busy time.
For rpc "framework_set_scheduler dynamic" will initialize the dpdk_governor
which finally call rte_power_init. This process is time consuming on some
systems which may make app_thread busy.
We can allow the app_thread to be busy for the first sample. This is one
solution. It also works if we call sleep for some time after calling rpc
framework_set_scheduler.
Change-Id: Ia2f4b2f35e94657ce093ce261438ab710399435a
Signed-off-by: Richael Zhuang <richael.zhuang@arm.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16825
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Michal Berger <michal.berger@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Patch rte_thash_x86_gfni introduced in change
4dcb94480b turned out
to be not unified as required so patching causes errors
in upstream vs-dpdk CI jobs.
The current master dpdk version is 23.03.0-rc1 and it
contains changes from the patch so we have to check for
all versions less than (not less-equal) 23.03.0.
Signed-off-by: Kamil Godzwon <kamilx.godzwon@intel.com>
Change-Id: Ie4d3a829bcd91ca07659388094bda2d61f7c1584
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16914
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
If a driver is registered and selected, it'll now be used to execute
sequences of accel operations. The driver has priority over accel
modules, so the modules will only be used to execute operations that the
driver cannot perform.
Once driver completes a task (or a number of tasks), it notifies accel
using standard spdk_accel_task_complete(). To let accel continue
processing a sequence, driver can call spdk_accel_sequence_continue().
This can be done when the driver executes all tasks (1), an error occurs
(2), or the driver doesn't know how to execute a given opcode (3). In
case of (3), that operation will be executed using appropriate accel
module and, while the rest of the sequence will be sent back to the
driver.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: If414c02073ffc731454e03d25c7ee02bef58463b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16548
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
In particular, cache cntlid, nsze and ncap to make sure namespaces can
be properly created and attached to. The test was assuming that the ns
should be attached with a controllers' list always set to 0, however,
that doesn't have to be the case for every nvme (the difference was
noticed under SAMSUNG MZQL21T9HCJR-00A07). Same goes for the formatted
block size where we cannot assume it's 512 bytes as it depends on the
active lbaf.
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: Iac9902052e8b2db44ede2b5178372d65c07b5b4a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16580
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
These are just wrappers around nvme cli used for caching ctrl and ns
features.
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I61b784a0a29e121ce813c1b405a56a07b4d859b0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16579
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
When we select CPUs for the scheduler tests we pick up all siblings
from given core. On some nodes in the CI these threads may have IDs
in ranges that simply overflow during the shift operation ( > 62).
To avoid this, use a list instead of a mask. Also, deny all threads
with IDs > 127 as DPDK doesn't support them anyway.
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: Ib83649a762cb31a460184d1e0b594c112aea2bab
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16604
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Mainly, don't trigger them on ERR, simply on EXIT - the ERR is
reserved for the main tracing trap set by autotest_common.sh and it
should not be overwritten.
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I435d1c08c29557692c134a087350f8538ebf92f9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16708
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Without it, this test fails under nvme ctrls which do report UUIDs
for their namespaces.
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I8f0137f8ce57f44c49f1708c8599a52e8092c262
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16707
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Also test a case when opening an lvol fails due to insufficient
resources.
Change-Id: I8b1b7a9c4d67e93691f89541374c7ef09a7d3f18
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16944
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
