This closes races between concurrent spdk_bdev_module_claim_bdev()
and/or spdk_bdev_module_release_bdev() calls affecting the same bdev by
holding bdev->internal.spinlock while claiming and releasing a bdev. It
also closes a potential TOCTOU bug in that optimizing compilers probably
already eliminate in bdev_finish_unregister_bdevs_iter() and documents
that bdev->internal.claim_module is protected by
bdev->internal.spinlock.
This can be removed when the bdev_register_examine_thread deprecation
is removed.
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: Ib48552df065d5172139a61bbc00b391f36552c0c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15282
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Since bdev_examine() can happen on any thread and it happens without any
other lock being held on the spdk_bdev_module, it is possible for
multiple threads to try to simultaneously increment
module->internal.action_in_progress. Decrements may also race.
This commit adds bdev_module->internal.spinlock and holds it while
modifying module->internal.action_in_progress.
This can be removed when the bdev_register_examine_thread deprecation
is removed.
Change-Id: I9c401eeb3c7c97c484e16fa9cfd82668b32e508b
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15281
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
This introduces a deprecation for calling spdk_bdev_register() and
spdk_bdev_examine() on a thread other than the app thread. The
deprecation period starts in SPDK 23.01 and removal is expected in SPDK
23.05.
The intent of this deprecation is to ensure that bdev modules'
examine_config() and examine_disk() callbacks are only ever called on
the app thread. This largely a formalization of what has long happened
due to the RPC poller running on the first thread started by
spdk_app_start().
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: Ic9d7b87b6522be20357d2eab2d0c77cd5753452f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15690
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Define struct spdk_bdev_io_error_stat privately in lib/bdev/bdev.c.
Add a pointer to struct spdk_bdev_io_error_stat to struct
spdk_bdev_io_stat.
Allocate spdk_bdev_io_error_stat for bdev and RPC, but do not allocate
spdk_bdev_io_error_stat for I/O channel.
Dump the contents of spdk_bdev_io_error_stat only if its total is
non-zero.
As a result of these, only spdk_bdev_get_device_stat() can query
spdk_bdev_io_error_stat for the bdev_get_iostat RPC. This will be
acceptable.
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: Idae868afe65347a96529eedc3dcc692101de4a29
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14826
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
The following patches will make some of io_stat helper functions
public APIs. Then, for consistency, bdev_ + verb + _io_stat will
be better naming rules.
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: If36d4ed29253e87954c23c270e8414731d083f03
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15896
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
The internal mempools were replaced with the newly added iobuf
interface.
To make sure we respect spdk_bdev_opts's (small|large)_buf_pool_size, we
call spdk_iobuf_set_opts() from spdk_bdev_set_opts(). These two options
are now deprecated and users should switch to spdk_iobuf_set_opts().
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ib1424dc5446796230d103104e272100fac649b42
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15328
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
This is done in a couple of places, so it makes sense to extract it to a
separate function.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Id34b2545d9912c2b7b65b1277711e9683db92658
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15327
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Found with misspell-fixer.
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: If062df0189d92e4fb2da3f055fb981909780dc04
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15207
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Both max and min should be reset periodically. We can use the queue
depth sampling poller to reset these but the queue depth sampling poller
is optional. We extend the bdev_reset_iostat RPC to support mode to
reset all or only max/min fields.
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I9ce54892f6e808f6a82754b6930092f3a16d51ff
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15444
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Add max/min_read/write/unmap_latency_ticks into the struct
spdk_bdev_io_stat.
When initializing or resetting the instance of the struct
spdk_bdev_io_stat, initialize max to 0 and min to UINT64_MAX.
Then update max if a new value is larger than the current max,
and update min if a new value is smaller than the current min.
For the bdev_get_iostat RPC, it prints max and prints min if min is not
UINT64_MAX or 0 if min is UINT64_MAX.
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I1b30b3825c15e37e9f0cf20104b866186de788a2
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14825
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Add a helper function bdev_reset_device_stat() to reset I/O statistics.
This funciton is used for the bdev_reset_iostat RPC.
We do not have any plan to use bdev_reset_device_stat() outside
lib/bdev. Hence, we do not add this as a public API.
Then, add a new RPC bdev_reset_iostat to reset I/O statistics of a
single bdev or all bdevs.
Resetting I/O statistics affects all consumers. Add a note to CHANGELOG
and doc/jsonrpc.md.
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I97af09107b5c3ad1f9c19bf3cbf027457c4fbae7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15350
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Hold not only io_stat pointer but also num_blocks and blocklen in local
variables.
This will shorten and simplify bdev_io_update_io_stat(), and improve
readability and changeability.
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I527b72538a169a1faafd32863ff539306a8763a9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15732
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
The following patches will add max/min latencies and more optional
counters. This factorization will improve the readability.
In addition to factorization, add spdk_likely to check if completed
successfully or not.
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I57581ece2b73d486aa138f8d26a5afaf6953a322
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15480
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
For consistency, rename a JSON dump function by bdev_io_stat_dump_json()
and change the parameter order.
Other public APIs and function pointers in the generic bdev layer,
spdk_bdev_dump_info_json(), spdk_bdev_fn_table::dump_info_json, and
spdk_bdev_fn_table::write_config_json have a json_write_ctx pointer
as the last parameter. For consistency, swap a statistics pointer and
a json_write_ctx pointer.
This is another preparation to extend I/O statistics to include error
counters and module specific counters to output these via the
bdev_get_iostat RPC.
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I6f3bb6f2752f7da856d4fe66c0f1f8a2eedc176b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15731
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Move a JSON dump functionbdev_get_iostat_dump() for I/O statistics
into lib/bdev/bdev.c.
The next patch will rename the function and change the parameter order.
This is another preparation to extend I/O statistics to include error
counters and module specific counters to output these via the
bdev_get_iostat RPC.
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I6a90d15fcbaa2e2a250167754135623bc9e7f362
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14837
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Add helper functions, bdev_io_stat_alloc(), bdev_io_stat_free(),
and bdev_io_stat_get() for struct spdk_bdev_io_stat.
Then replace a bdev_io_stat_add() call by bdev_io_stat_get() at
spdk_bdev_get_device_stat() because the saved data is queried first.
This is another preparation to extend I/O statistics to include error
counters and module specific counters to output these via the
bdev_get_iostat RPC.
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I9547757421a1de1b8cb44e0f8ade4b5c2bcad4e6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15443
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
The following patches will extend I/O statistics to include error
counters and module specific counters to output these via the
bdev_get_iostat RPC.
In this case, the size of the struct spdk_bdev_iostat will be variable.
As a preparation, allocate spdk_bdev_io_stat dynamically.
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I1979a9d867859d5cb5d05717bfcc677f07fa03f8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15479
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Community-CI: Mellanox Build Bot
The following patches will extend I/O statistics to include error
counters and module specific counters to output these via the
bdev_get_iostat RPC.
In this case, the size of the struct spdk_bdev_iostat will be variable.
As a preparation, allocate spdk_bdev_io_stat dynamically.
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I50b57f792b451cf748ea8eb0611fe65d693d5a14
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15478
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Community-CI: Mellanox Build Bot
Replace comments saying that particular locks must be held with
assertions that enforce that those locks are held. Remove the comments
so that there is no chance of comments and code getting out of sync in
the future.
This also fixes a caller of bdev_close() that did not hold a required
lock.
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: I3a540f1ad9b9826f925c523986334aa8fcd302f2
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15440
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This is unsafe, because we touch need_buf_* queues, which aren't
thread-safe. Also, documented this requirement in
spdk_bdev_io_get_buf()'s description.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Iabc141e051c543fdd51f079ae212f69e980d8148
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15668
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Added new API 'spdk_bdev_histogram_get_channel' to get histogram of
a specified channel for a bdev. A callback function is passed to it
to process the histogram.
Change-Id: If5d56cbb5fe6c39cda7882f887dcc9c6afa769ac
Signed-off-by: Richael Zhuang <richael.zhuang@arm.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15539
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Print out the specific values in this SPDK_ERRLOG,
this can help to find where the error is.
Change-Id: I2a38aa2d4270e0bbf554ddb348a73d40967d1b16
Signed-off-by: wanghailiangx <hailiangx.e.wang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15618
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: GangCao <gang.cao@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Mellanox Build Bot
SPDK threads generally run on dedicated cores and locks should be rarely
contended. Thus, putting a thread to sleep while waiting on a mutex does
not free up CPU cycles for other pthreads or processes. Even when
running in interrupt mode, lock contention should be low enough that
spinlocks are a net win by avoiding context switches.
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: I6e2e78b2835bbadb56bbec34918d998d75280dfd
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15438
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
`BDEV_IO_NUM_CHILD_IOV` and `BDEV_RESET_IO_DRAIN_RECOMMENDED_VALUE`
are public macro definitions without `SPDK_` prefix, so we add the
`SPDK_` prefix to them.
Change-Id: I4be86459f0b6ba3a4636a2c8130b2f12757ea2da
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15425
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
The bdev hot remove might be an async process. The bdev_open will
return an error during the hot remove process. If someone invoke the
bdev_get_bdevs API when a bdev is in the middle of a hot remove
process, the spdk_for_each_bdev function will stop its loop when a
bdev_open return an error. Thus the bdev_get_bdevs will only return
partual bdevs or even return an empty list if the hot remove bdev is
the first bdev in the loop. When spdk_for_each_bdev and
spdk_for_each_bdev_leaf loop for each bdevs, if a bdev returns an
error, we skip that bdev instead of stop the whole loop.
Signed-off-by: Peng Yu <yupeng0921@gmail.com>
Change-Id: Ib35b817e23e47569fc5762a883b4ff8e322ae173
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15322
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Mike Gerdts <mgerdts@nvidia.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Remove automatic generation of UUIDs for bdevs
that do not provide this value themselves.
This is to clarify whether this field can be
depended upon.
Modified match files to reflect change in UUID
generation.
Disabled nullglob shell option, as it deletes
empty arrays during word splitting. Bdevs with no
aliases would instead of "[]", have nullpointer
printed, which makes resulting JSON invalid.
Part of enhancement proposed in #2516.
Change-Id: Ic1d5f8f8d001ae1a219e876aef2a19b1ff0b2f2c
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15150
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Mellanox Build Bot
For I/Os controlled by QoS, TRACE_BDEV_IO_DONE is collected after
redirecting to the original thread. Hence, TRACE_BDEV_IO_START should
be collected on the original thread too.
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I15411be823450ee5ddaa7582509a7aa068476fc5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14824
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
per Intel policy to include file commit date using git cmd
below. The policy does not apply to non-Intel (C) notices.
git log --follow -C90% --format=%ad --date default <file> | tail -1
and then pull just the 4 digit year from the result.
Intel copyrights were not added to files where Intel either had
no contribution ot the contribution lacked substance (ie license
header updates, formatting changes, etc). Contribution date used
"--follow -C95%" to get the most accurate date.
Note that several files in this patch didn't end the license/(c)
block with a blank comment line so these were added as the vast
majority of files do have this last blank line. Simply there for
consistency.
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Id5b7ce4f658fe87132f14139ead58d6e285c04d4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15192
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
Add API spdk_bdev_io_get_submit_tsc to get submit tsc of a bdev I/O,
which can be used in bdev modules to avoid calling expensive
spdk_get_ticks().
Change-Id: Ifbcecb1bc663344997c5e73b72a1dfb5d0422946
Signed-off-by: Richael Zhuang <richael.zhuang@arm.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14989
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Copy operation is defined by source and destination LBAs and LBA count
to copy. For destiantion LBA and LBA count we reuse exiting fields
`offset_blocks` and `num_blocks` in `struct spdk_bdev_io`. For source
LBA new field `src_offset_blocks` was added.
`spdk_bdev_get_max_copy()` function can be used to retrieve maximum
possible unsplit copy size. Zero values means unlimited. It is allowed
to submit larger copy size but it will be split into several bdev IOs.
Signed-off-by: Evgeniy Kochetov <evgeniik@nvidia.com>
Change-Id: I2ad56294b6c062595c026ffcf9b435f0100d3d7e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14344
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Community-CI: Mellanox Build Bot
And also related function pointers and APIs:
spdk_bdev_for_each_channel_msg;
spdk_bdev_for_each_channel_done;
spdk_bdev_for_each_channel_continue;
Change-Id: I52f0f6f27717d53c238faf2f998810c9c5ee45d4
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14614
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Add new bdev property split_on_write_unit which, if set to true, causes
writes to be split to match write_unit_size and fail if not aligned to
or not multiple of write_unit_size.
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Change-Id: Id49f58a3288ddf5cfe4921ce4020ae4bcdd67298
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11390
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
A new parameter io_drain_timeout has been added to spdk_bdev
structure. If this value is unset, the bdev reset behavior
does not change.
The io_drain_timeout controls how long a bdev reset must wait for IO
to complete prior to issuing a reset to the underlying device.
If there is no outstanding IO at the end of that period, the reset
is skipped.
Change-Id: I585af427064ce234a4f60afc3d69bc9fc3252432
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14501
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
These functions start from a given offset and seek for next
data or for next hole. For bdevs that do not support seeking,
it is assumed that only data and no holes are present
Signed-off-by: Damiano Cipriani <damiano.cipriani@suse.com>
Change-Id: I6bc831970223333b25683f60ce3fcbbfebb5bb81
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14361
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Community-CI: Mellanox Build Bot
The generic bdev layer has a public API spdk_bdev_get_qd() but its
value is the most recently measured value and it requires qd sampling
to be enabled. We will have bdev modules to want to wait until
all bdev_ios are aborted by a reset. Unfortunately, spdk_bdev_get_qd()
is not suitable for the custom bdev module. Furthermore,
spdk_bdev_channel::io_outstanding is not accessible from bdev modules.
Hence, add a new public API spdk_bdev_get_current_qd().
This function should be used only from the bdev module and it should
be ensured that the bdev is not unregistered during execution.
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: Ica30a8d8fe3264e28f0772a39bdf5f9ba72933e1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12791
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Some use cases want to abort every bdev_io submitted to the bdev by
traversing the bdev channels.
However, struct spdk_bdev_channel is private in lib/bdev/bdev.c.
Hence, add a helper function spdk_bdev_for_each_bdev_io() to execute
the function on the appropriate thread for every bdev_io submitted
to the bdev.
This function should be used only from the bdev module and it should
be ensured that the bdev is not unregistered during execution.
We keep this function as generic as possible because we may have
other use cases in future.
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: Ic0209361bd1228ea8d4cb3241d0df07106be58d9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12751
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
If the bdev does not natively support compare we use
the fall-back which performs a read instead of a compare
operation. We then compare the results of the read with
the buffer provided by the user. In case the bdev has
metadata, there are two options:
1) md is interleaved -> the md will be part of the data
buffer allocated for the read and compared accordingly
2) md is separate -> currently we do not compare
the metadata but just ignore it.
This patch fixes 2) by comparing the md buffer after
the read is done.
Signed-off-by: Jonas Pfefferle <pepperjo@japf.ch>
Change-Id: I1018b8c02540bffcba69408eb283bdc8f06bb747
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14132
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
bdev_io is allocated from a memory pool and is not zeroed on reuse.
So set bdev_io->u.bdev.ext_opts = NULL for io ops where it is not
supported (yet) so we can test against it.
Signed-off-by: Jonas Pfefferle <pepperjo@japf.ch>
Change-Id: Ia579ea6b0787cf62572ea3a6bf2251867602e952
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14056
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Community-CI: Mellanox Build Bot
This will allow us to map spdk_bdev_io events
to nvme_request events coming in a future patch.
Since we pass the nvme_bdev_io to the nvme driver
(not the spdk_bdev_io), we need to add tracepoints
for the nvme_bdev_io so that spdk_trace can
do the spdk_bdev_io->nvme_bdev_io->nvme_request
mapping.
An alternative would have been to pass the spdk_bdev_io
as the cb_arg to the nvme driver, but that change
seemed to invasive, and I think we will find other
uses for the nvme_bdev_io events anyways.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Id7519e689b01875093359f41a1ca2af912061a8b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13994
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Wait for all bdevs to finish examination before
proceeding with the spdk_bdev_finish shutdown
logic. This ensures the bdev layer and its
modules are not trying to examine bdevs after
the bdev layer has reported it has shut down.
Theoretically, bdev modules could all defer their
fini callbacks until any outstanding examinations
are complete, but it is WAY simpler to just use
the existing spdK_bdev_wait_for_examine API
instead.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: If90cc2a786281d348b82de8beb17ac37ba269c64
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13850
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
We have to process whole QoS queue on each QoS poll. It may contain
IOs that still have quota or not affected by QoS rules at all. If we
stop on the first queued IO, all IOs will be limited by the minimum
QoS rule even if they're not affected by this rule.
Here is an example and simple test. We have a NVMf target with Null
bdev and QoS configured with read bandwidth limited to 10 MB/s and
write bandwidth limited to 100 MB/s. First we start nvme_perf with
only write IOs and we see that reported bandwidth is 100 MB/s. Then we
start another instance of nvme_perf with only read IOs. We see that
reported read bandwidth is 10 MB/s but we also see that write
bandwidth also drops to 10 MB/s.
Signed-off-by: Evgeniy Kochetov <evgeniik@nvidia.com>
Change-Id: I1edf09d038e65f873deef19ecb0f4bf9725a5ca5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13767
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
