Commit Graph

19402 Commits

Author SHA1 Message Date
sijie.sun
549be9ad81 nvmf/rdma: Recreate resources and listeners after IB device is hotplugged
IB device may be unplugged & hotplugged when modifying slaves of bonding
IB devices. This patch will try to recreate ibv device contexts, poller
and listeners after IB devices come back.

Signed-off-by: sijie.sun <sijie.sun@smartx.com>
Change-Id: I3288174bad847edc2d9859cb34aa93c6af8c673b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15616
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
2023-03-07 11:50:05 +00:00
sijie.sun
8ddc5cd4a7 nvmf/rdma: Destroy all related resources after IB device removed
When IBV_EVENT_DEVICE_FATAL & RDMA_CM_EVENT_DEVICE_REMOVAL occurs,
destory all userspace resources such as qp, poller and ibv_context.

Signed-off-by: sijie.sun <sijie.sun@smartx.com>
Change-Id: Ie4832e4804eb572d6ec3bdc44fb7f9339f443d7e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15615
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2023-03-07 11:50:05 +00:00
Konrad Sztyber
07be7ca0ad bdev/nvme: use non-ext IO functions if possible
Patch 55f947933 ("bdev: remove spdk_bdev_ext_io_opts from spdk_bdev_io")
changed the way bdev_nvme submits IO to the NVMe driver causing
performance degradation for requests with iovcnt = 1, as they also had
to go through the path that executes the reset_sgl/next_sge callbacks.

This patch reverts those changes back to the original code checking
iovcnt and using the non-SGL functions if possible.

Suggested-by: Jim Harris <james.r.harris@intel.com>
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I5e7c6620d38b7690ff862d8cd0075afacc578217
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16961
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2023-03-07 11:49:20 +00:00
Jim Harris
584d295245 nvmf/fc: fix memleaks
Submitted by @udayawati via GitHub comment on
issue #2872.

Fixes issue #2872.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Id662fc0178f6112dfe791733bda43f634107403f

Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16932
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Anil Veerabhadrappa <anil.veerabhadrappa@broadcom.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Mellanox Build Bot
2023-03-06 13:21:38 +00:00
Mike Gerdts
ce67e0c787 blob: clones of external snapshots
This is the beginning of support for external snapshots. An external
snapshot is a read-only blobstore device (struct spdk_bs_dev) that can
be used as a blob's back device. Normally a blob will have no back
device (a normal blob), a zeroes back device (a thin provisioned blob),
or a blob back device (a clone blob). When a blob has an external
snapshot ("esnap") as its back device, it is called an esnap clone.

With this patch, esnap clones can be created but they are not yet
useful. Subsequent patches in the series will plumb the IO path, enable
various features, and allow lvol bdevs to be esnap clones.

Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: I29206b628a2b03b6386a88532565e228df988e0e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14969
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2023-03-03 11:25:35 +00:00
Mike Gerdts
0f4bfd55e7 test/blob: test with and without copy offload
Perform all tests on devices that do and do not support
spdk_bs_dev::copy.

Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: Ic4c13ade9f45709b34a57f9fb7456d6c6a790a85
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16691
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
2023-03-03 11:25:35 +00:00
Mike Gerdts
18d9f71286 blob_ut: refactor read and write byte count
This reworks how blob_snapshot_rw() tracks the number of bytes read and
written. It has no functional change: it simply makes the patch that
follows less complex.

Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: Ieeb738b6a814e7939931fecdfaf14b9f162d8431
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16861
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2023-03-03 11:25:35 +00:00
Mike Gerdts
ab2eff0798 blob_bdev: take read-only-many claims on ro devs
If the bs_dev was opened read-write, continue to take a
read-many-write-one claim. If it was opened read-only, take a
read-many-write-none claim.

Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: I25d977c6961f962423899fb891ec912cd847930a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16282
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2023-03-03 11:25:35 +00:00
Mike Gerdts
bd5a784719 blob_bdev: support read-only devices
External snapshots, which will be introduced in a later commit, will
need read-only blob_bdev instances. This support is partially needed to
support underlying devices that are naturally read-only and partially to
provide an extra layer of protection against accidental writes.

Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: Ibcb28d00ad644a6053aa5f4de15471c2cd8e348a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14968
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2023-03-03 11:25:35 +00:00
Mike Gerdts
45e0a2a370 doc/blob: thin provisioning, snapshots, and clones
Describe how blobstores handle thin provisioning, snapshots, and clones.

Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: Ie6f1b69799a404a373269986fe8a89c36f381620
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11270
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2023-03-03 11:25:35 +00:00
Marcin Spiewak
fa272c9bc6 lib/idxd: Removed SPDK_IDXD_FLAG_PERSISTENT flag
Removed SPDK_IDXD_FLAG_PERSISTENT flag and associated code.

Change-Id: Ib4e038794792ae9866bdf344f1ec58dd04dbd483
Signed-off-by: Marcin Spiewak <marcin.spiewak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16986
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2023-03-03 11:20:03 +00:00
Marcin Spiewak
2903189c40 lib/accel: removed references to ACCEL_FLAG_PERSISTENT
- Removed ACCEL_FLAG_PERSISTENT and related code

Change-Id: Icc8890ad7143fab4bd4b544e82acdf9b84c0a1eb
Signed-off-by: Marcin Spiewak <marcin.spiewak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16951
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2023-03-03 11:20:03 +00:00
Marcin Spiewak
27d5b38523 lib/accel: remove PMEM support from accel library
This patch removes references to deprecated PMEM from accel library.
The code that was executed when ACCEL_FLAG_PERSISTENT flag is set,
is no longer needed and is removed.

_sw_accel_copy() function is removed and replaced with memcpy(), as
after PMEM removal its functionality is the same as memcpy().

_sw_accel_dualcast() is no longer needed, replaced with direct calls
to memcpy()

Removed 'flags' parameter - it is no longer needed

accel_ut.c: removed references to PMDK

deprecation.md updated

ACCEL_FLAG_PERSISTENT flag will be removed in next patch.

Change-Id: I86130466fe7a5f6ee547df1517b803035ff41a7a
Signed-off-by: Marcin Spiewak <marcin.spiewak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16899
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2023-03-03 11:20:03 +00:00
Tomasz Zawadzki
71a0db0e66 example/perf: add --use-every-core
This option overrides the current worker and ns
association logic with using every ns from each
worker.

This mimics the bdevperf -C option, and allows
to test scenarios in similar fashion.

Change-Id: Ie3734f86cd8fd1025bca5bd69e27b7316c2f3fbd
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14415
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2023-03-03 11:18:59 +00:00
Swapnil Ingle
d1b2a5c436 bdev_nvme: start accel_comp_poll poller lazily
With bdev_nvme accel_comp_poll is always created, even if it is not
really used e.g bdev_nvme with vfio-user transport.
accel_comp_poll poller is registered when accel_channel is created, in
order to start it lazily this patch creates accel_channel during first
usage.

Signed-off-by: Swapnil Ingle <swapnil.ingle@nutanix.com>
Change-Id: I5c710a038606ada00c9bfde6a4d9691ee8f6d204
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16957
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2023-03-02 08:36:03 +00:00
Richael Zhuang
dc32e1ba56 test/scheduler: fix bugs in set_cpufreq
1.We should get current scaling governor from sysfs for comparison
2.The file name is scaling_governor not scaling_governors

Change-Id: I758b88d6b05a5d4b58bd103bfec8dfcb64453a5f
Signed-off-by: Richael Zhuang <richael.zhuang@arm.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16863
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michal Berger <michal.berger@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2023-03-02 08:35:55 +00:00
Richael Zhuang
5971ef1d89 test/scheduler: fix lacking of cppc_cpufreq in set_cpufreq
Added cppc_cpufreq case in set_cpufreq.

Change-Id: I8a7ceeb2891a9464a858b0d4ed1e3350b7ad1dc6
Signed-off-by: Richael Zhuang <richael.zhuang@arm.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16802
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Michal Berger <michal.berger@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2023-03-02 08:35:55 +00:00
Kamil Godzwon
2f05e0593a dpdk: update submodule with AVX512F and GFNI fixes
This patch moves DPDK submodule by one commit:
(6fb6205c21) hash: fix GFNI implementation build with GCC 12

Signed-off-by: Kamil Godzwon <kamilx.godzwon@intel.com>
Change-Id: I84de7c3411906fe27dc33b4e2d4186a379c8a7cf
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17004
Reviewed-by: Michal Berger <michal.berger@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2023-03-02 08:35:49 +00:00
Richael Zhuang
822325ca28 test/schedule: fix idle.sh test failure
idle.sh sometimes fails because the app_thread's idle time is less
than busy time.

For rpc "framework_set_scheduler dynamic" will initialize the dpdk_governor
which finally call rte_power_init. This process is time consuming on some
systems which may make app_thread busy.

We can allow the app_thread to be busy for the first sample. This is one
solution. It also works if we call sleep for some time after calling rpc
framework_set_scheduler.

Change-Id: Ia2f4b2f35e94657ce093ce261438ab710399435a
Signed-off-by: Richael Zhuang <richael.zhuang@arm.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16825
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Michal Berger <michal.berger@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2023-03-01 19:23:54 +00:00
Kamil Godzwon
bad261b6ab test/patch: fix rte_thash_x86_gfni patch
Patch rte_thash_x86_gfni introduced in change
4dcb94480b turned out
to be not unified as required so patching causes errors
in upstream vs-dpdk CI jobs.

The current master dpdk version is 23.03.0-rc1 and it
contains changes from the patch so we have to check for
all versions less than (not less-equal) 23.03.0.

Signed-off-by: Kamil Godzwon <kamilx.godzwon@intel.com>
Change-Id: Ie4d3a829bcd91ca07659388094bda2d61f7c1584
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16914
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2023-03-01 13:29:50 +00:00
Konrad Sztyber
e6a8401a1d accel: execute accel sequences using a driver
If a driver is registered and selected, it'll now be used to execute
sequences of accel operations.  The driver has priority over accel
modules, so the modules will only be used to execute operations that the
driver cannot perform.

Once driver completes a task (or a number of tasks), it notifies accel
using standard spdk_accel_task_complete().  To let accel continue
processing a sequence, driver can call spdk_accel_sequence_continue().
This can be done when the driver executes all tasks (1), an error occurs
(2), or the driver doesn't know how to execute a given opcode (3).  In
case of (3), that operation will be executed using appropriate accel
module and, while the rest of the sequence will be sent back to the
driver.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: If414c02073ffc731454e03d25c7ee02bef58463b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16548
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2023-03-01 09:46:40 +00:00
Jim Harris
4037f0671b sock/posix: don't add 1 to pipe size
There's no need to add 1 to the pipe size, it
actually results in unaligned accesses after we've
cycled through the pipe buffer for the first time.

Fixes: eb98488 ("sock/posix: Internally buffer reads.")

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iff79d40694dab1afe128ccac0c7c7a28cd827a3d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16924
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2023-03-01 08:56:14 +00:00
Richael Zhuang
fed358a0e7 util: fix misaligned load for uint64_t type
The following error was reported when running gpt_ut which is related
to crc32_update().
"load of misaligned address 0x001ffeff78cc for type 'const uint64_t',
which requires 8 byte alignment".

This patch preprocesses the first several bytes to make the buf address
passed to __crc32_d or__crc32_cd is 8 byte aligned. And finally process
the trailing bytes.

For function spdk_crc32c_update in crc32c.c, memcpy was used to avoid
misaligned load problem. Update it with above solution to reduce extra
overhead.

Signed-off-by: Richael Zhuang <richael.zhuang@arm.com>
Change-Id: I7c7aaa41e1c042a96668158818b06729fb3ceec6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16801
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2023-03-01 08:56:01 +00:00
Sebastian Brzezinka
2d590e74b4 bdev/lvol: add param size_in_mib to replace size in bytes
`rpc_bdev_lvol_resize` and `rpc_bdev_lvol_create` can now use size
in MiB instead of `size` in bytes. This change make param `size`
deprecated and using both `size` and `size_in_mib` return error.

Since `bdev_lvol_resize` and `bdev_lvol_create` use size in MiB,
name of param should reflect it, previously used param `size` is
positional therefore there is no need to keep it as deprecated.

This patch fix issue: #2346

Signed-off-by: Sebastian Brzezinka <sebastian.brzezinka@intel.com>
Change-Id: Ibbe2c056bb63d9f82dee91c87fdd501ce441d5f8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16901
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2023-03-01 08:55:43 +00:00
Sebastian Brzezinka
57bd108ad7 lib/blob: notify user when bs_create_blob fail
This patch fix issue: #2346

Signed-off-by: Sebastian Brzezinka <sebastian.brzezinka@intel.com>
Change-Id: Ibbc95bcf47300653066a85769536853e7608f213
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16758
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2023-03-01 08:55:43 +00:00
Michal Berger
8ecf7ae50b test/nvme/cuse: Use common.sh to restore nvme devices with proper data
In particular, cache cntlid, nsze and ncap to make sure namespaces can
be properly created and attached to. The test was assuming that the ns
should be attached with a controllers' list always set to 0, however,
that doesn't have to be the case for every nvme (the difference was
noticed under SAMSUNG MZQL21T9HCJR-00A07). Same goes for the formatted
block size where we cannot assume it's 512 bytes as it depends on the
active lbaf.

Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: Iac9902052e8b2db44ede2b5178372d65c07b5b4a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16580
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2023-03-01 08:55:26 +00:00
Michal Berger
4033751407 test/nvme/cuse: Add some helper functions for cuse tests
These are just wrappers around nvme cli used for caching ctrl and ns
features.

Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I61b784a0a29e121ce813c1b405a56a07b4d859b0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16579
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2023-03-01 08:55:26 +00:00
Michal Berger
5ad2877dc4 test/scheduler: Use CPU list rather than cpumask
When we select CPUs for the scheduler tests we pick up all siblings
from given core. On some nodes in the CI these threads may have IDs
in ranges that simply overflow during the shift operation ( > 62).

To avoid this, use a list instead of a mask. Also, deny all threads
with IDs > 127 as DPDK doesn't support them anyway.

Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: Ib83649a762cb31a460184d1e0b594c112aea2bab
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16604
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2023-03-01 08:55:26 +00:00
Michal Berger
6f3127adb1 test/spdkcli: Adjust traps
Mainly, don't trigger them on ERR, simply on EXIT - the ERR is
reserved for the main tracing trap set by autotest_common.sh and it
should not be overwritten.

Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I435d1c08c29557692c134a087350f8538ebf92f9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16708
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2023-03-01 08:55:26 +00:00
Michal Berger
fe648cb987 test/spdkcli: Consider matching against nvme bdev's UUID
Without it, this test fails under nvme ctrls which do report UUIDs
for their namespaces.

Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I8f0137f8ce57f44c49f1708c8599a52e8092c262
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16707
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2023-03-01 08:55:26 +00:00
Jacek Kalwas
019cbb9335 nvmf: disable data buf mempool cache
Depending on the number of cores there are sporadic issues getting
elements of that pool although free elements are there during poll
group creation. Operation returns -ENOBUF. It results in odd notice
msg.

"nvmf_transport_poll_group_create: *NOTICE*: Unable to reserve the
full number of buffers for the pg buffer cache. Decrease the number of
cached buffers from 455 to 1366"

In this case 1366 is the actual number of available elements in the
pool. Few poll groups suceeds and few are ending up with the buffer
cache size set to 0.

Issue has been rootcaused as bug or behaviour change in DPDK v22.01.

Consider example:
We create DPDK mempool with 4K buffers, cache of 256. When first poll
group requests 512 buffers, DPDK mempool first looks in its per-core
cache, sees no buffers (mempool buffer cache doesn't get prepopulated)
and then requests 512 + 256 buffers from the backing pool. It returns
512 of the buffers to the user, and puts the other 256 buffers in the
cache ...it should only request 512 buffers total. For 8 cores and 512
buffers requested only 5 cores will get their buffers.

Disabling mempool cache seems to workaround the issue. More effective
cache is already implemented on nvmf generic layer.

Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: I3149dea95a4f24a75dd0074eda9468c4856d901d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16913
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2023-02-28 08:57:23 +00:00
Krzysztof Karas
b9298bf908 core locks: add error checking to unclaiming
Add error check to unclaim mechanism. Issue #2920
showed that unclaiming CPU locks might fail and
we should catch errors to determine the cause.

Change-Id: Ifdfb7db2595d73f8bae13418ef145ad80e1d07ef
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16958
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2023-02-28 08:57:13 +00:00
Krzysztof Karas
609f8d4379 bdev_nvme: adjust generation of UUID to conform to the standards
Change the way we generate UUIDs to use uuid_generate_sha1()
from libuuid.

Fixes #2788.

Change-Id: Id750a7de74fb37bc3ab3ab06b3254b49d73f4be3
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15935
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
2023-02-28 08:57:01 +00:00
Krzysztof Karas
7e88ff9e99 uuid: add spdk_uuid_generate_sha1()
Add implementation of uuid_generate_sha1() for systems
that do not have this function in their system libraries.

Use uuid_generate_sha1 from uuid.h inside a new function
spdk_uuid_generate_sha1(). The reason for this addition
is to prepare for UUID generation correction to conform
to standards.

First part of series addressing #2788.

Change-Id: Ib357aa1ee832e886288d176d8a47efdaa326f537
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16414
Reviewed-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2023-02-28 08:57:01 +00:00
Shuhei Matsumoto
511daff348 bdev/nvme: Factor out start operations from failover() and delete_ctrlr()
The following patches will add mutex for bdev_nvme_delete() to improve
handling race conditions. This refactoring make such change easier.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I8971684a708bce89872f08f75db86eb3b723f380
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16821
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2023-02-28 08:56:45 +00:00
Shuhei Matsumoto
f38d2f6a8e bdev/nvme: Delete alternate paths and an active path separately
If alternate paths and an active path are treated separately in
_bdev_nvme_delete(), it will be much easier to protect a path list
from delete operation by mutex.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: Ie3bed095fd92b80c0487ef7b136953ad03a174eb
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16820
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2023-02-28 08:56:45 +00:00
Shuhei Matsumoto
422f5f3d11 bdev/nvme: Factor out deleting a ctrlr from bdev_nvme_delete()
Factor out path deletion operation for one nvme_ctrlr into _bdev_nvme_delete().

_bdev_nvme_delete() traverses all remaining nvme_ctrlrs even after one
nvme_ctrlr was successfully deleted. To remember the successfule
deletion, use two return value variables, rc and _rc. For _rc returned by
_bdev_nvme_delete(), copy _rc to rc if _rc is zero, return _rc immediately
if _rc is not -ENXIO, or do nothing otherwise.

This improves the readability and makes us easier to add more changes.

Additionally, for _bdev_nvme_reset_io(), it is simpler to overwrite rc
to 0 if rc is -EBUSY rather than to return 0 if rc is -EBUSY. This is
very small change and hence done together.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I57eafd22918c8c976b9c51bedb9e4369976c1d5c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16819
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2023-02-28 08:56:45 +00:00
Shuhei Matsumoto
8173930051 bdev/nvme: Rename _bdev_nvme_delete() by bdev_nvme_delete_ctrlr()
The next patch wants to use _bdev_nvme_delete() as a function name.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: Iba103868e74795bf42c443a377c77e9c073329fb
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16818
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2023-02-28 08:56:45 +00:00
Shuhei Matsumoto
4757c05fa1 bdev/nvme: Factor out check if path should be deleted into a helper function
This improves the readability and makes us easier to add more changes.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I55d7925d70f2a204f65a81f3fc44cf96b69c3ebe
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16709
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2023-02-28 08:56:45 +00:00
Ben Walker
07ca24ec59 thread: Add spdk_thread_get_interrupt_fd_group that returns spdk_fd_group
Return the real spdk_fd_group object so it can later be nested.

Change-Id: I84c8a174c7d177799fa484b350269082c61b18a5
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15474
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: John Levon <levon@movementarian.org>
2023-02-28 08:52:27 +00:00
Ben Walker
059073c4d9 thread: In interrupt mode, exit thread by sending a message
Don't assume spdk_thread_poll() will ever get called. Instead, send
a message to process the exit.

Change-Id: Idd98e7e8164c5efebd0d7c9287e62731e7cbc998
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15551
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: John Levon <levon@movementarian.org>
2023-02-28 08:52:27 +00:00
Ben Walker
5ecfb22192 thread: In interrupt mode, remove pollers by sending a message
Move away from relying on spdk_thread_poll() to do clean up in interrupt
mode. In the future, we don't want to have spdk_thread_poll() called at
all.

Change-Id: I5318a7889601a3d3463e35419918b7305f68ee8d
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15550
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: John Levon <levon@movementarian.org>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2023-02-28 08:52:27 +00:00
Ben Walker
f3c1b59a29 thread: Always wrap interrupt handlers to set the thread correctly
Eventually, we want to allow merging of spdk_fd_groups, removing a level
of indirection. That means that some interrupt handlers won't
necessarily fire with the spdk_thread context already set. Set it in the
wrappers to ensure it's right.

Change-Id: Ief18d58cf3ee005c2969a9c0ee132b34b24cbd61
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15476
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: John Levon <levon@movementarian.org>
2023-02-28 08:52:27 +00:00
Ben Walker
01dca5ed72 thread: Pollers now register spdk_interrupts, unifying the code paths
This unifies the poller fds with the interrupt mechanism internally.

Change-Id: I57a270260981ff54670365dddb33a1d9bdb56781
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15754
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2023-02-28 08:52:27 +00:00
Marcin Spiewak
ba20950a53 lib/idxd: Fixed compilation issue (gcc 12.2.1)
Added initialization of	the prev_crc variable to avoid compilation error:

idxd.c: In function ?spdk_idxd_submit_copy_crc32c?:
idxd.c:1138:51: error: ?prev_crc? may be used uninitialized [-Werror=maybe-uninitialized]
 1138 |                                 desc->crc32c.addr = prev_crc;
      |                                 ~~~~~~~~~~~~~~~~~~^~~~~~~~~~
idxd.c:1081:18: note: ?prev_crc? was declared here
 1081 |         uint64_t prev_crc;
      |                  ^~~~~~~~

Change-Id: I6b93d5d85b52e20f8a2c313c41b740f66eebe1c7
Signed-off-by: Marcin Spiewak <marcin.spiewak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16900
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2023-02-28 08:15:40 +00:00
Artur Paszkiewicz
2b772dbbbc ut/vbdev_lvol: examine multiple lvols
Also test a case when opening an lvol fails due to insufficient
resources.

Change-Id: I8b1b7a9c4d67e93691f89541374c7ef09a7d3f18
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16944
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2023-02-24 21:01:11 +00:00
Artur Paszkiewicz
fd7641d575 bdev/lvol: retry lvol open if it fails due to insufficient resources
Fixes: #2910
Change-Id: Id28b5c4e7e002fe81fea774d989d1507d933133e
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16875
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2023-02-24 21:01:11 +00:00
Artur Paszkiewicz
0194aa5bd2 bdev/lvol: eliminate grow examine code duplication
These functions:
 _vbdev_lvs_grow_examine_finish
 _vbdev_lvs_grow_examine_cb
 _vbdev_lvs_grow_examine

are almost exact copies of:
 _vbdev_lvs_examine_finish
 _vbdev_lvs_examine_cb
 vbdev_lvs_examine

Remove the duplicated grow functions. Factor out the common entry point
into _vbdev_lvs_examine() which accepts a callback to perform the
required action (spdk_lvs_load or spdk_lvs_grow).

Change-Id: I9805f29e22812051608873207401e3786dab7103
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16942
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2023-02-24 21:01:11 +00:00
Artur Paszkiewicz
f64fe6fe34 bdev/lvol: remove _vbdev_lvol_examine_close_cb
Remove _vbdev_lvol_examine_close_cb() because it is identical as the
code at the end of _vbdev_lvs_examine_finish().

Remove the INFOLOG message because it is redundant (there's an ERRLOG
just before) and inconsistent with the lvolerrno != 0 case, where there
is no such message.

Change-Id: I8488a8b5fcbaf1895f4953081d98bc13f82be221
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16941
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2023-02-24 21:01:11 +00:00
Michal Berger
03e2374c93 test/nvme/cmb: Use get-pmr as fallback
Under latest kernels the cmb* attributes are not present under the
sysfs anymore. See:

https://bugzilla.kernel.org/show_bug.cgi?id=217037

this is a regression that was fixed in 6.2 kernel. Until we jump to
that release, we need to have a workaround in place.

Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: Ia659518224899cbb3e9cef19681a457c945a9c96
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16776
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2023-02-24 12:52:04 +00:00