Inline the internal of bdev_nvme_delete_ctrlr() and bdev_nvme_failover()
into _bdev_nvme_delete().
Change the _nvme_ctrlr_destruct() call from direct to message passing
to reduce lock hold time and avoid potential deadlock.
Then, protect nbdev_ctrlr via g_bdev_mutex_unlock and each nvme_ctrlr
via nvme_ctrlr->mutex.
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I5cc2cf781d2846c51bce631c12fceaeade860a0b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16822
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Michael Haeuptle <michaelhaeuptle@gmail.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
If we find that the discovery entry already exists, a
single break doesn't work - that just breaks out of
the TAILQ_FOREACH. So instead change it to free
the resolver object and return directly.
Fixes issue #2945.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ia31d6ecfa4fdc0a168eecc8ec4659da10a870770
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17209
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Patch 55f947933 ("bdev: remove spdk_bdev_ext_io_opts from spdk_bdev_io")
changed the way bdev_nvme submits IO to the NVMe driver causing
performance degradation for requests with iovcnt = 1, as they also had
to go through the path that executes the reset_sgl/next_sge callbacks.
This patch reverts those changes back to the original code checking
iovcnt and using the non-SGL functions if possible.
Suggested-by: Jim Harris <james.r.harris@intel.com>
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I5e7c6620d38b7690ff862d8cd0075afacc578217
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16961
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
With bdev_nvme accel_comp_poll is always created, even if it is not
really used e.g bdev_nvme with vfio-user transport.
accel_comp_poll poller is registered when accel_channel is created, in
order to start it lazily this patch creates accel_channel during first
usage.
Signed-off-by: Swapnil Ingle <swapnil.ingle@nutanix.com>
Change-Id: I5c710a038606ada00c9bfde6a4d9691ee8f6d204
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16957
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Change the way we generate UUIDs to use uuid_generate_sha1()
from libuuid.
Fixes#2788.
Change-Id: Id750a7de74fb37bc3ab3ab06b3254b49d73f4be3
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15935
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
The following patches will add mutex for bdev_nvme_delete() to improve
handling race conditions. This refactoring make such change easier.
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I8971684a708bce89872f08f75db86eb3b723f380
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16821
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
If alternate paths and an active path are treated separately in
_bdev_nvme_delete(), it will be much easier to protect a path list
from delete operation by mutex.
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: Ie3bed095fd92b80c0487ef7b136953ad03a174eb
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16820
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Factor out path deletion operation for one nvme_ctrlr into _bdev_nvme_delete().
_bdev_nvme_delete() traverses all remaining nvme_ctrlrs even after one
nvme_ctrlr was successfully deleted. To remember the successfule
deletion, use two return value variables, rc and _rc. For _rc returned by
_bdev_nvme_delete(), copy _rc to rc if _rc is zero, return _rc immediately
if _rc is not -ENXIO, or do nothing otherwise.
This improves the readability and makes us easier to add more changes.
Additionally, for _bdev_nvme_reset_io(), it is simpler to overwrite rc
to 0 if rc is -EBUSY rather than to return 0 if rc is -EBUSY. This is
very small change and hence done together.
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I57eafd22918c8c976b9c51bedb9e4369976c1d5c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16819
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
The next patch wants to use _bdev_nvme_delete() as a function name.
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: Iba103868e74795bf42c443a377c77e9c073329fb
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16818
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
This improves the readability and makes us easier to add more changes.
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I55d7925d70f2a204f65a81f3fc44cf96b69c3ebe
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16709
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
The spdk_bdev_ext_io_opts structure is used to pass extra options when
submitting a bdev IO request, without having to modify/add functions to
handle new options. Additionally, the structure has a size field to
allow adding new fields without breaking the ABI (and thus having to
bump up the major version of a library).
It is also a part of spdk_bdev_io and there are several reasons for
removing it from that structure:
1. The size field only makes sense in structures that are passed
through pointers. And spdk_bdev_ext_io_opts is indeed passed as a
pointer to spdk_bdev_{readv,writev}_blocks_ext(), however it is
also embedded in spdk_bdev_io (internal.ext_opts_copy), which is
also part of the API. It means that each time a new field is added
to spdk_bdev_ext_io_opts, the size of spdk_bdev_io will also
change, so we will need to bump the major version of libspdk_bdev
anyway, thus making spdk_bdev_ext_io_opts.size useless.
2. The size field also makes internal.ext_opts cumbersome to use, as
each time one of its fields is accessed, we need to check the size.
Currently the code doesn't do that, because all of the existing
spdk_bdev_ext_io_opts fields were present when this structure was
initially introduced, but we'd need to do check the size before
accessing any new fields.
3. spdk_bdev_ext_io_opts has a metadata field, while spdk_bdev_io
already has u.bdev.md_buf, which means that we store the same thing
in several different places in spdk_bdev_io (u.bdev.md_buf,
u.bdev.ext_opts->metadata, internal.ext_opts->metadata).
Therefore, this patch removes all references to spdk_bdev_ext_io_opts
from spdk_bdev_io and replaces them with fields (memory_domain,
memory_domain_ctx) that were missing in spdk_bdev_io. Unfortunately,
this change breaks the API and requires changes in bdev modules that
supported spdk_bdev_io.u.bdev.ext_opts.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I49b7524eb84d1d4d7f12b7ab025fec36da1ee01f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16773
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
It has been implemented in commit 56e2d632ce to reset all nvme_ctrlrs
of a bdev controller sequentially. Update the log.
Change-Id: Id87bf99d7e269c8ad0d6b2576e2329455601ec55
Signed-off-by: Richael Zhuang <richael.zhuang@arm.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16653
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
For JSON RPC, boolean response with false value may not be regarded as error.
Previously many cases were replaced to use
spdk_jsonrpc_send_error_response() explicitly. However, this pattern did
not become established and new bdev modules used
spdk_jsonrpc_send_bool_response() with false.
Once again, replace remaining cases in this patch.
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: Ie8858bc8ff7c36bc3a829977044a91d459db76f2
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16555
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
The NVMe bdev module has been refined since the bdev_nvme_apply_firmware
RPC was added. It is enough to open any one NVMe bdev even if one NVMe
controller has multiple namespaces. Hence, let's remove
spdk_bdev_first/next() calls from rpc_bdev_nvme_apply_firmware() rather
than replacing these by spdk_for_each_bdev(). Additionally, not only
spdk_bdev_close() but also spdk_put_io_channel() and
spdk_jsonrpc_send_*_response() should be done on the original thread.
Hence, redirect to the original thread always when a NVMe command is
returned, and rename firm_ctx->thread by firm_ctx->orig_thread for
clarification. orig_thread will be better than comment.
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I90b1986584b3926980fd265e4fded194eb5a2d00
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16541
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
It is enough to embed an instance of rpc_apply_firmware structure into
firmware_update_info structure.
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I41a480a0b8cd56f4b537512d5f83c5cbd5eee718
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16540
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Previously, bdev_nvme_apply_firmware() used a temporary character array
to reduce the calls of spdk_jsonrpc_send_error_response().
Change these to call spdk_jsonrpc_send_error_response() directly.
This will simplify the next patch to replace spdk_bdev_first/next() by
spdk_for_each_bdev()ss.
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I87ffc9416232dcaba30202ebbe0cb4b185a9563d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16539
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
This is required when volume is attached via different storage boxes
for example while migrating it.
Signed-off-by: Gili Buzaglo <gili.buzaglo@kioxia.com>
Change-Id: I680c781ae5de078972d6aef547804d60fff3c643
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16407
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
To allow SO_MINOR updates on LTS for the whole year it is supported,
the major version for all components needs to be increased.
This is to prevent scenario where two versions exists with matching
versions, but conflicting ABI.
Ex. Next SPDK release adds an API call increasing the minor version,
then LTS needs just a subset of those additions.
Increasing major so version after LTS, allows the future releases
to update versions as needed. Yet allowing LTS to increase minor
version separately.
Disabled test for increasing SO version without ABI change, as
that is goal of this patch. This check shall be removed with SPDK 23.05
release.
Looks like this was left over from prior LTS, to avoid that
make sure it is only skipped when running against v23.01.x as latest
release.
This patch:
- increases SO_VER by 1 for all components
- resets SO_MINOR to 0 for all components
- removes suppressions for ABI tests
Short reference to how the versions were changed:
MAX=$(git grep "SO_VER := " | cut -d" " -f 3 | sort -ubnr | head -1)
for((i=$MAX;i>0;i-=1)); do find . -name "Makefile" -exec \
sed -i -e "s/SO_VER := $i\$/SO_VER := $(($i+1))/g" {} +; done
find . -name "Makefile" -exec \
sed -i -e "s/SO_MINOR := .*/SO_MINOR := 0/g" {} +
Change-Id: I3e5681802c0a5ac6d7d652a18896997cd07cc8bf
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16419
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Add RPC bdev_nvme_get_path_iostat to show I/O statistics for IO paths
of the NVMe bdev.
Change-Id: I22e5ad112d5cfa6d96cf246dcd0e511ae71dc839
Signed-off-by: Richael Zhuang <richael.zhuang@arm.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14745
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
We may need to process multiple events to get all of
the events associated with a newly published service,
so a 1 second poller is actually a bit too long since
we will only get one event per poll iteration. Shorten
it to 100ms instead.
It doesn't appear there is any easy way to use the avahi
API to help with this, maybe we could have some context
variable used to detect if an event was processed or not,
and call avahi_simple_poll_iterate() multiple times if
necessary. This might be something worth investigating
in the future.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ifea7a99dc8b8b501392e29276b20528a873b1ae1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16327
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Boris Glimcher <Boris.Glimcher@emc.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
The io paths' stat will get lost when they are destroyed. Record
the stat in the nvme_ns structure.
Change-Id: I12fc0b04fac0d59e7465fe543ee733f2822a9cdb
Signed-off-by: Richael Zhuang <richael.zhuang@arm.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14744
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Added helper functions nvme_ns_alloc() and nvme_ns_free().
This is a preparation for next patch where we will allocate or free
I/O statistics of nvme_ns when this nvme_ns is allocated or freed.
Change-Id: I54b0dba61bd19923178ba4bed97677cf628ba509
Signed-off-by: Richael Zhuang <richael.zhuang@arm.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16300
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Currently we have stat per bdev I/O channel, but for NVMe bdev
multipath, we don't have stat per I/O path. Especially for
active-active mode, we may want to observe each path's statistics.
This patch support IO stat for nvme_io_path. Record each nvme_io_path
stat using structure spdk_bdev_io_stat.
The following is the comparison of bdevperf test.
Test on Arm server with the following basic configuration.
1 Null bdev: block size: 4K, num_blocks:16k
run bdevperf with io size=4k, qdepth=1/32/128, rw type=randwrite/mixed with 70% read/randread
Each time run 30 seconds, each item run for 16 times and get the average.
The result is as follows.
qdepth type IOPS(default) IOPS(this patch) diff
1 randwrite 7795157.27 7859909.78 0.83%
1 mix(70% r) 7418607.08 7404026.54 -0.20%
1 randread 8053560.83 8046315.44 -0.09%
32 randwrite 15409191.3 15327642.11 -0.53%
32 mix(70% r) 13760145.97 13714666.28 -0.33%
32 randread 16136922.98 16038855.39 -0.61%
128 randwrite 14815647.56 14944902.74 0.87%
128 mix(70% r) 13414858.59 13412317.46 -0.02%
128 randread 15508642.43 15521752.41 0.08%
Change-Id: I4eb5673f49d65d3ff9b930361d2f31ab0ccfa021
Signed-off-by: Richael Zhuang <richael.zhuang@arm.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14743
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Support to specify rr_min_io for multipath round-robin policy,
which makes I/O switches to another io path after rr_min_io I/Os are
rounted to current io path.
Change-Id: I09f0d8d24271c0178ff816fa63ce8576b6e8ae47
Signed-off-by: Richael Zhuang <richael.zhuang@arm.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15445
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Added helper function bdev_nvme_clear_current_io_path for clearing
current_io_path.
The next patch will clear a new variable of the nvme_bdev_channel
when clearing current_io_path. It is convenient to add a helper
function to clear current_io_path.
Change-Id: I7d445638648ea232fedcbf16736e9c4c73cc9c71
Signed-off-by: Richael Zhuang <richael.zhuang@arm.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15681
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Support selecting io path according to number of outstanding io of
each path in a channel. It's optional, and can be set by calling
RPC "bdev_nvme_set_multipath_policy -s queue_depth".
Change-Id: I82cdfbd69b3e105c973844c4f34dc98f0dca2faf
Signed-off-by: Richael Zhuang <richael.zhuang@arm.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14734
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Non-optimized path was not cached for active/active policy unexpectedly.
Fix the bug in this patch.
This will simplify the code and will be helpful to add more algorithms a
little.
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I233d9fa6eca73b1e0fb850c7ea4b3474aff6fc1d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16190
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Richael <richael.zhuang@arm.com>
Unify _bdev_nvme_find_io_path() and bdev_nvme_find_next_io_path()
into _bdev_nvme_find_io_path() by modifying nvme_io_path_get_next().
For active/passive policy, _bdev_nvme_find_io_path() is called only if
nbdev_ch->current_io_path is NULL. Hence, the prev parameter is not
necessary anymore.
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: If61b8a24b768a1d571c0033b91d9d9bd487b5cf7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16189
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Richael <richael.zhuang@arm.com>
nbdev_ch->current_io_path is NULL at initialization.
bdev_nvme_find_io_path() calls _bdev_nvme_find_io_path() if
nbdev_ch->current_io_path is NULL. If there is no optimized path,
nbdev_ch->current_io_path is not filled. Hence, as long as there is no
optimized path, bdev_nvme_find_next_io_path() is never called.
However, bdev_nvme_find_next_io_path() fills nbdev_ch->current_io_path
even if the found path is non optimized.
This is a contradiction and a bug.
To fix this bug, as a preparation, refactor _bdev_nvme_find_io_path() to
cnosolidate bdev_nvme_find_next_io_path() and _bdev_nvme_find_io_path()
into a single function because bdev_nvme_find_next_io_path() and
_bdev_nvme_find_io_path() were almost identical.
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I7bc4107ce6ae6f116dee03c9bb67d9e4061ce775
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16188
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Richael <richael.zhuang@arm.com>
When an I/O gets an I/O error, the I/O path to which the I/O was
submitted may be still available. In this case, the I/O should be
retried to the same I/O path. However, a new I/O path was always
selected for an I/O retry.
For the active/passive policy, the same I/O path was selected naturally.
However, for the active/active policy, it was very likely that a
different I/O path was selected.
To use the same I/O path for an I/O retry, add a helper function
bdev_nvme_retry_io() into bdev_nvme_retry_ios() and replace
bdev_nvme_submit_request() by bdev_nvme_retry_io(). bdev_nvme_retry_io()
checks if nbdev_io->io_path is not NULL and is available. Then, call
_bdev_nvme_submit_request() if true, or call bdev_nvme_submit_request()
otherwise. For I/O path error, clear nbdev_io->io_path for
clarification. Add unit test to verify this change.
Linux kernel native NVMe multipath already takes this approach. Hence,
this change will be reasonable.
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I7022aafd8b1cdd5830c4f743d64b080aa970cf8d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16015
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Richael <richael.zhuang@arm.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
The following patches will clear bio->io_path when the error is a
network path error. As a preparation, cache bio->io_path to a local
variable io_path.
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I0a25f4a70dd1064702b2037283bddd6e789dfae1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16014
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This is in the same effort as gathering all retry functions into the
same place. This will make the following patches easier.
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: If2a017ebda45eb1eaf0dc93b15d02b4122dc3be0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15958
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Relocate abort_retry_ios() in a file to gather all retry functions
in the same place.
I/O retry is very complex and an improvement will be added in the
following patches. This will make the following patches easier.
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I84751bd078db4aef753dcc533320cee9aa09c9b2
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15957
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
The following patches will change I/O retry to use the same io_path if
it is still available. However, bdev_nvme_submit_request() always calls
bdev_nvme_find_io_path() first. For I/O retry, if possible, we want to
skip calling bdev_nvme_find_io_path() and use nbdev_io->io_path instead.
To reuse the code as much as possible and not to touch the fast code
path, factor out request submit functions from
bdev_nvme_submit_request() into _bdev_nvme_submit_request().
While developing this patch, a bug/mismatch was found such that
bdev_io->internal.ch was different from ch of
bdev_nvme_submit_request(). Fix it together in this patch.
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: Id003e033ecde218d1902bca5706c772edef5d5e5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16013
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Error counters for NVMe error was added in the generic bdev layer but
we want to know more detailed information for some use cases.
Add NVMe error counters per type and per code as module specific
statistics.
For status codes, the first idea was to have different named member
for each status code value. However, it was bad and too hard to test,
review, and maintain.
Instead, we have just two dimensional uint32_t arrays, and increment
one of these uint32_t values based on the status code type and status
code. Then, when dump the JSON, we use spdk_nvme_cpl_get_status_string()
and spdk_nvme_cpl_get_status_type_string().
This idea has one potential downside. This idea consumes 4 (types) *
256 (codes) * 4 (counter) = 4KB per NVMe bdev. We can make this smarter
if memory allocation is a problem. Hence we add an option
nvme_error_stat to enable this feature only if the user requests.
Additionally, the string returned by spdk_nvme_cpl_get_status_string()
or spdk_nvme_cpl_get_status_type_string() has uppercases, spaces, and
hyphens. These should not be included in JSON strings. Hence, convert
these via spdk_strcpy_replace().
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I07b07621e777bdf6556b95054abbbb65e5f9ea3e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15370
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
If calloc failed, there was a NULL pointer deref.
Signed-off-by: plestk <plestringant@kalray.eu>
Change-Id: I8e4104589a283349e7251c32b1decdfdd850b9ec
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15779
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Added io_outstanding in struct nvme_io_path to record outstanding
I/O number in each path, which will be used by multipath to select
I/O path.
io_outstanding gets updated for I/O sent to a namespace and not get
updated if sent to a controller.
For FLUSH case, it calls bdev_nvme_io_complete() directly and
io_outstanding is not updated for this case.
spdk_bdev_io_get_buf() is executed in the generic bdev layer.
Hence, we do not update io_outstanding for spdk_bdev_io_get_buf().
Change-Id: I47b515e0f254e5daa7e1e88799a832032b23ff34
Signed-off-by: Richael Zhuang <richael.zhuang@arm.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15032
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Directly call bdev_nvme_io_complete() for FLUSH case.
This is a preparation for io_outstanding update in nvme_io_path. For
IO sent to a namespace, io_outstanding gets increment/decrement. For
IO sent to a controller, io_outstanding is not updated. For FLUSH
case, it's not sent to a namespace nor a controller. Directly call
bdev_nvme_io_complete() here to make it easier to understand why
io_outstanding not gets updated for FLUSH.
Change-Id: Idcf4b37a0816a327256fe87004705fb65df1b06c
Signed-off-by: Richael Zhuang <richael.zhuang@arm.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15517
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Add an option "--generate-uuids" to bdev_nvme_set_options
RPC to enable generation of UUIDs for NVMes devices that
do not provide this value themselves. The identifier is
based on a serial number of the device, so a bdev
using this NVMe will always be assigned the same UUID.
Part of enhancement from #2516.
Change-Id: I86d76274e5702d14ace89d83d1e9129573f543e2
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15151
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
per Intel policy to include file commit date using git cmd
below. The policy does not apply to non-Intel (C) notices.
git log --follow -C90% --format=%ad --date default <file> | tail -1
and then pull just the 4 digit year from the result.
Intel copyrights were not added to files where Intel either had
no contribution ot the contribution lacked substance (ie license
header updates, formatting changes, etc). Contribution date used
"--follow -C95%" to get the most accurate date.
Note that several files in this patch didn't end the license/(c)
block with a blank comment line so these were added as the vast
majority of files do have this last blank line. Simply there for
consistency.
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Id5b7ce4f658fe87132f14139ead58d6e285c04d4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15192
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
