Add an API so that the user can enable/disable the bdev IO
timeout. Also, add the bdev io timeout handling callback.
So it means to let the upper user determine how to handle
the IO timeout scenario reset the device or abort the IO.
Change-Id: I9c7138ca46c74c045b687adab59a18d6bccc4996
Signed-off-by: Jin Yu <jin.yu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/469228
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Previously, when additional resets were submitted while a reset was in
progress, those resets were queued and then executed serially as part of
executing the original reset. Doing multiple resets on a bdev in quick
succession is not useful if the first reset succeeds and is very
unlikely to be useful in the negative case. Instead, we should batch
resets and complete them all at once when the current reset succeeds or
fails.
Change-Id: If10e0f37526860eaeeb41a8803d6298a3eff3212
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/474599
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Add the tsc trace for the parent bdev_io.
Change-Id: Idf5c5df3953fa1697c384a4f83b591584dd30a80
Signed-off-by: Jin Yu <jin.yu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476588
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
The reset cmd and parent of splited IO would not go into
the _bdev_io_submit() so they don't record the submitted
time. The timeout IO check needs the submitted time so we
add them in this patch.
Change-Id: Ic7217171ec878d4a6dfa80d6635957ca0186928b
Signed-off-by: Jin Yu <jin.yu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476050
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Add a link to the bdev channel for linking all the bdev
IOs that were submitted to this channel so that we can
monitor each IO's consuming-time.
Change-Id: I1e425b2059f20fd7b158eb3d6b023ce8629e7a30
Signed-off-by: JinYu <jin.yu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/469227
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
The _ was supposed to indicate an internal function,
but really leaving off the spdk in the function
is the standard way to denote an internal function.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I74facafb67b793502838b9c1b5f90aec2c88c69b
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/475033
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
We try to reserve the spdk_ prefix for functions that
are part of the public SPDK API.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I0a95ba0f9db04b588a2e948427f8a9f53a7b5740
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/475032
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
This function is not part of the SPDK public API, so
remove the spdk_ prefix from it to make it more clear
this is internal to the SPDK bdev library.
Also add "channel" to the name since this function is
operating on the channel parameter.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I57897ab4bf601b90551259b7cf6efa63152ed02f
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/475031
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
This is not part of the public SPDK API, so remove
the spdk_ prefix to make it clear this is internal
to the bdev library.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ibda76e4d8e34dde0c2fe638cb965e5ba2d9e47b5
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/475030
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
This function is not part of the SPDK public API, so
remove the spdk_ prefix to make that more clear when
reading the code.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I5ea4c04c474a6cf0862f11921daa97c9df728e96
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/475029
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
In emulated zero copy mode on devices with separate metadata support
the start zero copy request could assert after read completion,
during _bdev_io_unset_bounce_buf; the original iovs would be NULL, but
original md_buf would have garbage data.
Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Change-Id: Ia52b012c1b7462ebe490c29f5493168775646848
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/473455
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This change allows setting of the NVMe completion queue
CDW0 in spdk_bdev_io_complete_nvme_status.
Before that change, handling of vendor specific NVMe IO
commands was limited since there wasn't a way to return
command specific info back to the initiator.
Change-Id: I250d5df3bd1e62ddb89a011503d42bd4c8390f9b
Signed-off-by: Michael Haeuptle <michael.haeuptle@hpe.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/470678
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
When splitting bdev's, if we run out of child IOVs, we have to
make sure that we end block aligned. That may require us to
shorten or eliminate one or more child IOVs to be picked up
on the next split. If we eliminate enough such that there
are no IOVs for this split IO, just continue and the next
split completion will kick off another split to pick up
the remaining data to be transferred.
Fixes issue #981
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: If2bfbe5f0862295e1d74cbea00692890a2178967
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/471313
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
When we ran out of child_iov space, ensure the iovs to be aligned
with block size. However the calculation was wrong.
(to_next_boundary_bytes % blocklen) meant not to_last_block but to_next_block.
So calculate to_last_block_size by reducing to_last_block_size from blocklen.
The data was collected when the issue occured. So add unit test
by using the data.
Fixes#979
Reported-by: Geoffrey McRae <geoff@hostfission.com>
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I62a50bada450288ea7c60aec0e557c2a53cd8916
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/470806
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This patch adds reference counter and gurading mutex to bdev_desc
structure to keep track of in-flight messages and avoid release of
descriptor until all of messages has been processed. With reference
counter existing remove_scheduled field in descriptor structure is not
needed anymore.
Signed-off-by: Evgeniy Kochetov <evgeniik@mellanox.com>
Signed-off-by: Sasha Kotchubievsky <sashakot@mellanox.com>
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Change-Id: I97f78955362b04131abf202ba04e6d60343f3faf
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/469620
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Some internal bdev.c static function will be shared
with new zoned bdev module.
Change-Id: Ifbb8bf443f67b2daf97858b15d474ecce98a9efb
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/469100
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
This patch is entry point for extending bdev
interface to support devices with zoned namespace
semantics.
spdk_bdev_is_zoned() will allow user to check if
bdev is zoned bdev.
Change-Id: Id9ea9898d406d1d942bf3081b00ebcb574ac2b5e
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/460641
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Added write_unit_size field to bdev structure.
It describes required number of logical blocks
for write operation. For legacy bdevs this value
will be equal to logical block size.
For bdevs working on top of Open Channel/Zoned
Namespace SSDs or RAID 5 volumes write size unit
could be greater than logical block size and
upper layer should perform write operations
with size of multiple of write unit size.
Change-Id: I55eb6687912a7d0d1157874b2778e7d6c2d44e37
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/463802
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
This patch adds new interface for opening bdev and
implements new style remove event. With that changes
user can be notified about different types of events
that occur in regards to bdev. spdk_bdev_open_ext
function uses bdev name as an argument instead of bdev
structure to remove race condition where user gets
the bdev structure and bdev is removed after getting
that structure and before open function is called.
spdk_bdev_open is now deprecated.
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I44ebeb988bc6a2f441fc6a0c38a30668aad999ad
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455647
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Every exit path in spdk_bdev_set_qos_rate_limits() should
go through _spdk_bdev_set_qos_limit_done() as soon as
ctx is allocated and qos_mod_in_progress set to true.
This patch fixes one path were it did not occur.
With this change qos_mod_in_progress is set to false,
when failure in allocating qos parameters occurs.
Change-Id: I04a45dfdcde9160fd2701b44f5fde26fb0245177
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/465659
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
This is a part of a change that will introduce
new spdk_bdev_open_ext function. This will make
possible to pass callback function while opening
bdev which will be called when some bdev events
occur.
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I3e552edd5c90dacd39885e75e9114e48dddf50ea
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/463157
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Ran into this while putting the compress tests in CI, an error
message will print following the completion of the app, checked
with Jim and it is actually not an error when lvols are involved
so change to warn now as this will show up in everyone's build
log for bdev testing.
Change-Id: Ib509b89acf5b6bfbeb349692ab59bbea4964557f
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/463803
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
If subsystem bdev module initialize failed, it will call
spdk_bdev_init_complete(-1) -> (subsystem bdev)->fini
_spdk_bdev_finish_unregister_bdevs_iter -> spdk_bdev_module_finish_iter
In abovt path, spdk_bdev_module_finish_iter will repeate to call
bdev_module->module_fini function. Some bdevs will call spdk_io_device_unregister
which never call spdk_io_device_register. It will coredump when assert false in
spdk_io_device_unregister dev is null.
To fix this, let's check whether g_bdev_mgr.module_init_complete is equals true
and then call the bdev_module->module_fini.
Change-Id: Ia9a13318720d954e40eb2d666574bcb86e5f49e3
Signed-off-by: Tianyu Yang <yangtianyu2@huawei.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462382
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Now that vhost closes bdev descriptors on the same thread
that opened them, we can reintroduce thread asserts into
the bdev layer.
This reverts commit 283abcb9a2.
Change-Id: I1acc455df0674b808ecf2fa58dffd183db6cf3c2
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459168
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
In case some module has `async_init = true` and
some other module that comes after it fails to initialize,
then callback from asynchronously initialized module
may call `spdk_bdev_init_complete()` first, then failed module
will call `spdk_bdev_init_complete()` later.
This currently results in NULL dereference because
first call to `spdk_bdev_init_complete()` sets `g_init_cb_fn = NULL`.
This change prevents first call to `spdk_bdev_init_complete()`
by saying that failed module is not finished with initialization.
This patch fixes#847
Change-Id: Ib6b231d5ea27896ad88d7f11b8732921077b3d4d
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/461230
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
In future patch in new spdk_bdev_open_ext function we will call
spdk_bdev_get_by_name function and after that call and before
calling old spdk_bdev_open routine bdev can be removed.
We need to add mutex which will prevent that. Any future code
should use this mutex when accessing the bdevs list to get
a bdev and perform some operation on it.
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I785a1791346aebdd394fc51ad0e7fbfbabf317c9
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458457
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
It looks like currently we only check bdev names for NULL, but
not for "empty" string.
For example this rpc command:
sudo scripts/rpc.py construct_aio_bdev aio_disk "" 512
Will result in construction of AIO bdev with empty name:
sudo scripts/rpc.py get_bdevs
[...]
"name": "",
"aliases": [],
[...]
Change-Id: I41204096c8cf210a4dc40a8225d1c9dad353f533
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/460150
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
For devices that don't have a UUID, the UUID is generated at
registration time. That means that some devices will not have the same
UUID from run to run, but this seems no worse than having no UUID at
all.
Change-Id: Icf6b8517ffcffabafa2b73176dc03d896d0017fe
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459604
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
In file included from bdev_ut.c:43:
/root/yidong/spdk/lib/bdev/bdev.c:4373:9: warning: Access to field 'bdev'
results in a dereference of a null pointer (loaded from variable 'desc')
return desc->bdev;
^~~~~~~~~~
This is related to issue #822.
Change-Id: I8cd2bafadeff9846169bc9ca67b3c4110e9c0da8
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459529
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
When we run out of bdev_io's child iovs and we had
to round down I/O size to nearest block size boundary,
we used to decrease the existing child_iovcnt and
set a new "child_iov_run_out" flag to terminate the
uppermost splitting loop.
We can get rid of that new flag by just not decreasing
child_iovcnt when rewinding the last few iovs - it
will make the uppermost loop naturally terminate using
the existing checks.
Change-Id: Ie40c7ce135e7fb8fe284afdf7beeebd10af85cb7
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459911
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Here is the an example to describe existing issue:
There is a Write request with 64KiB data length, and this IO is cross the IO
boundary. We assume that the parent IO will have 2 children requests, one is
33KiB length, the other one is 31KiB. Here is the view of parent iovs, the
first 33KiB length data has 33 iovs:
iov.[0].iov_length = 1024;
.
.
iov.[31].iov_length = 256;
iov.[32].iov_length = 768;
.
.
iov.[64].iov_length = 1024;
In function _spdk_bdev_io_split(), then you can see that for the 33KiB length
child request, exiting code will run out of child child_iov space and return
error due to last one data buffer is not block size aligned.
Here we can rewind the existing offset to last block size aligned buffer to
avoid the error case, for backend which need aligned data buffer such as
AIO backend, the request will go through spdk_bdev_io_get_buf() again to
do the data copy, otherwise for those backend devices such as NVMe with
hardware SGL support, 256 data segment is fine for them.
Change-Id: I96ebdf29829d86f9b38fab28a7406eedc9fa44ef
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453604
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
There is one existing example usage case to describe the issue:
Users(e.g. Vhost-blk target with Windows Guest) call spdk_bdev_readv_blocks()
to submit a 128KiB length data READ request, and the data buffer
provides by vhost isn't aligned, but the backend block device
requires aligned data buffer, so existing function call trace:
spdk_bdev_readv_blocks()-->
spdk_bdev_io_submit()-->
spdk_bdev_io_get_buf()
spdk_bdev_io_get_buf() will allocate buffer from large data
buffer pool for 128KiB length, of course, it will return error
with existing logic.
So here, no matter what the data length is, we can go through
the split process first for both READ and WRITE.
However, there is one scenario that for iSCSI READ request,
the iSCSI layer will not allocate data buffer for the request,
so for this case if the IO boundary is required we should keep
the logic as before.
Change-Id: I67661f5fa4c3c7c561b45c86146759aa3477adbf
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453133
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
If bdev is configured to use separate buffers for metadata transfer,
but it doesn't support the write zeroes command, pass empty zeroed
buffer to make sure the metadata is cleared out as well.
Change-Id: If6f024266067e5764a28a276296f651d31da4792
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457628
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
The function spdk_bdev_desc_get_bdev() was added,
but remained unused. Couple places done the exact same thing,
so now they just use it directly.
Change-Id: I018f57b7b4bcfe22ec6a97cf5261b9a257ba93de
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456957
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
_spdk_bdev_io_split() will be used to requests that have IO buffers.
Change-Id: I5ba0b83e47b84066e6c5032de2e02404155896d3
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457549
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
The function spdk_bdev_io_get_io_channel() was added,
but remained unused. Couple places done the exact same thing,
so now they just use it directly.
Change-Id: Ifa332ce3a489256ccf2f5d0dd9c74a3215c544ce
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456435
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
The function spdk_bdev_io_get_thread() was added,
but remained unused. Couple places done the exact same thing,
so now they just use it directly.
Change-Id: Ia25abf57eb88c8df47c83e4edd761a9e02cc1618
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456436
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This patch adds *_with_md family of functions allowing for IO with
metadata being transferred in a separate buffer.
Change-Id: I842d5a00a532cf5d0b0f0738535ea46903674140
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/451465
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Existing code in spdk_bdev_write_zeroes_blocks() will call spdk_bdev_free_io()
for the error case, which will cause assertion because the bdev_io isn't
submitted to the backend yet, so we will check the condtion first to
avoid the error case.
Change-Id: If27d78217f709a3315e74c00869d345abd6b9a69
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453491
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Move actual submission code to _spdk_bdev_io_do_submit, used by
both normal submission path and QoS path.
Previous patch(review.gerrithub.io/c/442127) adds the missing
bdev_io->internal.in_submit_request flag to QoS submission path.
But QoS submission path doesn't handle nomem_io yet.
This patch makes QoS submission path handle nomem_io in the same way
as the normal path and extracts actual submission code into do_submit
function, so that further modification of the submission logic will
apply to both paths automatically.
Change-Id: I41fa88d239c3a2bd9783d812826e32e7c887818d
Signed-off-by: lorneli <lorneli@163.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455252
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: GangCao <gang.cao@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
When parent IO was splitted into several children requests, SPDK
may return parent completion callback with error status before
all the children requests are finished.
Change-Id: I63221a0ae1a5925a7fcd9744b4f5d8079c641252
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453611
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
These two paths do the same thing, only spdk_bdev_io_get_buf tries to
allocate the buffer directly, while spdk_bdev_io_put_buf reuses the
returned buffer for one of the waiting requests.
Change-Id: I341a011f3a16a99de399c8b56fa73ccd9c7fe4fa
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/451466
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Previously whenever bdev module went through init>fini>init,
the async modules would trigger assert in spdk_bdev_module_action_done().
This was because starting value for action_in_progress was being
set only once - during registration of the modules.
With this change, modules are set as async just before their init start.
Change-Id: If177a8dc97812b1b264cb61d5ad202ce47fca3b5
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/454614
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
This was not used by any of the trace register descriptions.
Let's remove it rather keeping it around if we don't need it.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Idda809e2911db5be555ff6aa13695484a14bf665
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452734
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Mempools are based off of a ring structure which allocates its elements
as a power of two. It also only exposes n-1 elements to the user. So
when we create a mempool with 2^n elements in it, we have to allocate a
ring with 2^n+1 entries. By decreasing the number of elements in these
key mempools by 1, we can save a decent amount of memory.
Change-Id: I942c9dd4cf59096969bc2559fb46fd2084a07f09
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/448875
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
For bdevs that don't support zero copy, emulate the
API using regular read and write operations.
Change-Id: Iabd7ff474bf740b096f38bd47196987cbd89e915
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/416465
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Add a ZCOPY operation to obtain buffers that represent
data regions on the backing block device.
Change-Id: Ie941c16ee051d0009e3888b52b8f41773bba47b3
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/386166
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Bdev descriptors could be closed only from the same thread
that opened them. This restriction was suddenly introduced
at one point without making sure all the SPDK code respects
it. Vhost can still close descriptors from any arbitrary
thread and fixing that would require some more effort.
With this patch we remove the thread-specific assert from
spdk_bdev_close() and hence allow vhost to work properly
in debug builds. Vhost can still have a possible data
race with bdev hotremove notification, but let's get rid
of the abort() from the usual code path first.
Change-Id: I6fac66a5ebc907b1c5418fff618f0b64cd9b69f4
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/451561
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
We used to wait only for those descriptors which
specified the hotremove notification callback. The
bdev could've been removed before the descriptor
was closed and the subsequent spdk_bdev_close would
simply segfault.
This patch modifies spdk_bdev_unregister to always
wait for all descriptors to be closed before actually
unregistering the bdev. This consolidates the bdev
unregister behavior for descriptors with and without
the hotremove callback.
Change-Id: I9b358209c6abd301b6fe8660e27bc6fa4ef485d6
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/450175
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
set_bdev_qos_limit RPCs must come after the RPCs that
create its bdev.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I7008745f9c56d9bc579decc7fe3c4d0d58950754
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/451414
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
spdk_dma_*malloc() is about to be deprecated.
Change-Id: Idb18f7ef44634deaf63139f0f872881c9cdaefac
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/450256
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This API had good intentions, but as more complicated
use cases came up where base bdevs could come and go,
we've realized that the bdev layer will need another
mechanism to query bdev modules on these types of
relationships between a virtual bdev and its base
bdevs. We removed all code related to tracking
the array of base bdevs a long time ago.
Change all existing callers to use spdk_bdev_register.
Document spdk_vbdev_register as deprecated for now,
and change its implementation to just call
spdk_bdev_register for simplicity sake.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I3b40ed96480c0fa7184db42953a9f4e4c167fed1
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/450076
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
When bdev is registered it will send "bdev_register" notification and
"bdev_unregister" when unregister is complete.
Change-Id: I3b37a4cb766a473f3da63a571036add366e78fa5
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/448839
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
As the bdev_io maybe NULL, so put the assignment after assert(bdev_io != NULL).
Change-Id: I58d6fcf97931b2f431ad5680b87b098e0c9a5e9b
Signed-off-by: JinYu <jin.yu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449296
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Currently if module claims a bdev in examine_config,
it cannot start asynchronously. This patch changes this
behaviour by calling examine_disk on modules which previously
claimed bdev trough examine_config.
Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Change-Id: I85b603590c6dab50e59ef7e68f292cb9abc47d98
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/448132
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Increase the size of small and large buffers in the bdev layer
to store DIF. Increase is the amount necessary to store 16 byte
metadata to the block formatted with 512 + 16. 512 + 16 is the
current maximum ratio of metadata per block.
This change will be done in the iSCSI library in the next patch
and other libraries may be done later.
Hence add and use an new macro SPDK_BDEV_BUF_SIZE_WITH_MD(x) for
convenience.
Change-Id: I4b5498f56c9baf3e3ed93dd1c757998d7ce65141
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/444558
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Unregister calls are not guarded. Fix this by chekcing status before
doing unregister.
Change-Id: I593e27efdae17f6d89362fd8e4edccf2af2b7281
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/445894
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Retry mechanism will stuck if there is no outstanding
IO after completion with ENOMEM. This should never
happen.
Change-Id: I58ebf8cc8a498231beef43ac20f58eeaad89afda
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/446610
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
The calling of spdk_bdev_finish is in wrong place,
the bdev subsystem recycling will be called by
_spdk_subsystem_fini_next. If we call spdk_bdev_finish
here, it will be called twice, so it is unnecessary.
And this patch will be used to address the issue
reported in:
https://review.gerrithub.io/#/c/spdk/spdk/+/447613/
Change-Id: I03dff8b84bd0d7b49675ce8300b3da623963e140
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/447731
Reviewed-by: GangCao <gang.cao@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
iSCSI target will need to get data block size except for metadata.
Other SPDK application may need to get the same information
future. Hence this patch adds an new API spdk_bdev_get_data_block_size
to bdev layer. In the header file, spdk_bdev_get_data_block_size
is located next to spdk_bdev_is_md_interleaved to avoid confusion
by new users.
Change-Id: I0fd2a6d0bcf6a4c18c583f70d96cc5035fc57fe9
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/445082
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: wuzhouhui <wuzhouhui@kingsoft.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
If success is false in each bdev module's spdk_bdev_io_get_buf_cb,
call spdk_bdev_io_complete with SPDK_BDEV_IO_STATUS_FAILED, and
then return.
Change-Id: I6f106d8d39a3616f7305201fa2efc4805d4d00ee
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/c/446046
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
When the specified buffer size to spdk_bdev_io_get_buf() is greater
than the permitted maximum, spdk_bdev_io_get_buf() asserts simply and
doesn't call the specified callback function.
SPDK SCSI library doesn't allocate read buffer and specifies
expected read buffer size, and expects that it is allocated by
spdk_bdev_io_get_buf().
Bdev perf tool also doesn't allocate read buffer and specifies
expected read buffer size, and expects that it is allocated by
spdk_bdev_io_get_buf().
When we support DIF insert and strip in iSCSI target, the read
buffer size iSCSI initiator requests and the read buffer size iSCSI target
requests will become different.
Even after that, iSCSI initiator and iSCSI target will negotiate correctly
not to cause buffer overflow in spdk_bdev_io_get_buf(), but if iSCSI
initiator ignores the result of negotiation, iSCSI initiator can request
read buffer size larger than the permitted maximum, and can cause
failure in iSCSI target. This is very flagile and should be avoided.
This patch do the following
- Add the completion status of spdk_bdev_io_get_buf() to
spdk_bdev_io_get_buf_cb(),
- spdk_bdev_io_get_buf() calls spdk_bdev_io_get_buf_cb() by setting
success to false, and return.
- spdk_bdev_io_get_buf_cb() in each bdev module calls assert if success
is false.
Subsequent patches will process the case that success is false
in spdk_bdev_io_get_buf_cb().
Change-Id: I76429a86e18a69aa085a353ac94743296d270b82
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/c/446045
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
There are some cases that virtual bdev open and close
the device and QoS will be disabled at the last close.
In this case, when a new bdev open operation comes again,
the QoS needs to be enabled again.
Change-Id: I792e610f4592bad1cac55c6c55261d4946c6b3e2
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.gerrithub.io/c/442953
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Currently struct bdev_io holds io_channel that the I/O was submitted
on through bdev_io::bdev_channel, but bdev_io::bdev_channel is private
in bdev.c and cannot be referenced in other files.
Hence add an new API spdk_bdev_io_get_io_channel API to get io_channel
coniveniently.
Change-Id: Ic2e2fde845d324f7a1637e3c75080727a62de5ec
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/c/443843
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
This patch is for DIF check types.
Add enum spdk_dif_check_type to DIF library.
Add a field dif_check_flags to struct spdk_bdev and add
spdk_bdev_is_dif_check_enabled to bdev APIs.
Added enum is intended to improve usability. If no enum, the
caller will have to get raw data of flags and mask each bit.
Change-Id: Ia46a37a9684dc968dcc51963674f0a9963e0cd4d
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/c/443339
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This patch is for DIF settings.
Add fields dif_type and dif_is_head_of_md to struct spdk_bdev and
add APIs spdk_bdev_get_dif_type and spdk_bdev_is_dif_head_of_md to
bdev APIs.
The fields dif_type and dif_is_head_of_md are added to the JSON
information dump.
Change-Id: I15db10cb170a76e77fc44a36a68224917d633160
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/c/443184
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
To support DIF, bdev will need to expose the following information:
- Metadata format
- Block size
- Metadata size
- Metadata setting (interleave or separate)
- DIF settings
- DIF type 1, 2, or 3
- DIF location
- DIF check types
- Guard check
- Reference tag check
- Application tag check
This patch is for the metadata format. Subsequent patches will do for the DIF
setting and DIF check types.
Add fields, md_len and md_interleave, to struct spdk_bdev and add APIs,
spdk_bdev_get_md_size and spdk_bdev_is_md_interleaved, to bdev APIs.
The fields, md_len and md_interleave, are added to the bdev JSON infomation dump.
DIF will be used only in the NVMe bdev module and the upcoming virtual
DIF bdev module first. But additional required storage by md_len and md_interleave
will be very small and they are simple. Hence add them to struct spdk_bdev simply.
Change-Id: I4109f6a63e6f0576efe424feb0305a9a17b9b2e8
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/c/443183
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This function gets used as a function pointer, which
seems to keep the compiler from trying to inline the
function. Stack manipulation was showing up in the
perf profile pointing to this. Marking the function
as inline gets it actually inlined in the hot I/O
path.
Improves bdevperf microbenchmark from 78M to 85M IO/s.
Cores are virtually identical - 11.4M on core 0 and
10.4-10.6M on remaining cores.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iadced071dfc07fc09db6da3571c930988b2dc3fd
Reviewed-on: https://review.gerrithub.io/c/443278
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
This keeps the hottest structures at the head of the
cache and helps improve performance.
Improves microbenchmark (8 null bdevs on 8 lcores,
bdevperf seq read with qd=1) from 67M to 78M on my
Xeon E5-v3 system. Core 0 performance remains about
the same (10.7-10.8M) but others cores improve from
around 8.0M each to 9.4M.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ia3ccf94ab39b6f911127f0bd1016e352027b11fc
Reviewed-on: https://review.gerrithub.io/c/443277
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Unmap, discard, write zeros will be sent down from
higher stack. Remove these IOs for the QoS limit.
Change-Id: Ieb3cc19f31c43f8ddf8f8d2fd338f442ef48b679
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.gerrithub.io/c/442673
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Liang Yan <liang.z.yan@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
_spdk_bdev_io_submit uses the bdev_io->internal.in_submit_request
flag to ensure we unwind in cases where the I/O is completed
inline (i.e. malloc or null bdevs). But when an I/O gets queued
for QoS, and then we iterate through the queued I/O in
_spdk_bdev_qos_io_submit(), this flag was not getting set
when those I/O would get submitted to the underlying bdev. This
would allow for _spdk_bdev_qos_io_submit recursion, resulting
in all kinds of different types of memory corruption.
Fixes#613.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I29263f4e7b2ead60f08b60474d210defa803348c
Reviewed-on: https://review.gerrithub.io/c/442127
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Liang Yan <liang.z.yan@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: GangCao <gang.cao@intel.com>
This patch adds the support of read and write separate
bandwidth rate limits control with the configuration file.
Below is the example (in MiB) for the configuration section:
[QoS]
Limit_Read_BPS Malloc0 100
Limit_Write_BPS Nvme0n1 200
Change-Id: I0221516ce70c3fbb07b9e80c1c814ed5ba271c88
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.gerrithub.io/c/416672
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This patch added two new function pointers (queue_io,
update_io) for related qos operations like iops and
bandwidth rate limits.
Change-Id: I2ffd67c5f1c421eab448fd5e95f809da55805fcd
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.gerrithub.io/c/438157
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
There are some patches to linux kernel for separating
out discard IOs that are written to /proc/diskstats and
the various /sys/block/*/stat files. The following additional
fields were added to these files:
Filed 12 -- # of discards completed
Filed 13 -- # of discards merged
Filed 14 -- # of sectors discarded
Filed 15 -- # of millisecond spent discarding
SPDK could provide these raw information to bdev_io_stat. Users
can use these information to calculate more states of block device.
Signed-off-by: Yanbo Zhou <yanbo.zhou@intel.com>
Change-Id: I517d67f0ff0159baf04e24732a8fd0ccefcb9c46
Reviewed-on: https://review.gerrithub.io/c/439057
Reviewed-by: wuzhouhui <wuzhouhui@kingsoft.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
The underlying io_channel created by a bdev module
could be never freed in various cases due to missing
spdk_put_io_channel() calls.
Change-Id: If1aa736f67f0354c4f9dcdf61d3f1e6b859b4c48
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/438803
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: wuzhouhui <wuzhouhui@kingsoft.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Since bdev->internal.mutex protects variables in bdev->internal,
there's no need to init the fields of spdk_bdev_desc struct and
free the struct when claim_module exists within the mutex lock.
Reducing critical area could help to reduce lock competition
caused by opening bdev concurrently.
Change-Id: Ie6b2a63e44a97cfed914fa6d8adef2c5ec3ac300
Signed-off-by: lorneli <lorneli@163.com>
Reviewed-on: https://review.gerrithub.io/c/438514
Reviewed-by: wuzhouhui <wuzhouhui@kingsoft.com>
Reviewed-by: GangCao <gang.cao@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
When create channel, the fields that _spdk_bdev_channel_destroy_resource()
checked are always be set to non-null. Remove these unnecessary if
statements makes issue exposed more easily if something goes wrong.
Change-Id: I2d505c87176d4d49eb1528a258e4bea6477e0fe6
Signed-off-by: wuzhouhui <wuzhouhui@kingsoft.com>
Reviewed-on: https://review.gerrithub.io/c/438799
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Since division is more expensive than right shift operator, in
function spdk_bdev_bytes_to_blocks, use right shift instead of
division if the blocklen of bdev is a power of two.
Change-Id: Ib3dbc792e86582bba30b3dc028efbd12c69075ba
Signed-off-by: lorneli <lorneli@163.com>
Reviewed-on: https://review.gerrithub.io/438318
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
In the case the QoS thread has not properly initialized yet,
needs to go through the regular QoS enabling process to notify
all the channels and also disable the QoS properly. The channel
and poller related staff also needs to be handled together
with the thread.
Change-Id: Ifc2b2cdfb1181aa6418ad1d43ae5905c0c317549
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.gerrithub.io/437519
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Previously, if want to know which mask bit is used for specific
trace group, the only way is to check source code. Now list
each trace group with its trace tpoint group mask bit in
usage message
Change-Id: I7a85fe9c0885f1919f6ffbdc97dab81f1986fb07
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/435448
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
This will be required in following histogram patches.
Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Change-Id: I2eee6629243b7a4838a80dc1de33ae485c58081e
Reviewed-on: https://review.gerrithub.io/433874
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
There are some use cases such as multipath and RAID expansion where a
vbdev could have been registered before one of its base bdevs.
Currently we unregister bdevs at shutdown in reverse order of their
registration. Continue to do that in general, but skip any bdev that
is still claimed. Any bdevs skipped in this way will eventually be
unregistered once any bdevs that have claimed it have completed
unregistration.
Change-Id: Iafde9558430bc5ce56e8608ef50bcb2b5fbfbf71
Signed-off-by: Andrey Kuzmin <akuzmin@jetstreamsoft.com>
Reviewed-on: https://review.gerrithub.io/432136
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Now, that _spdk_bdev_io_get_buf offers allocating aligned buffers,
add possibility to store original buffer and replace it by aligned
one for the time of IO.
Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Change-Id: If0ed306175631613c0f9310dccaae6615364fb49
Reviewed-on: https://review.gerrithub.io/429754
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This is requirement for following patch. Requests that will
reuquest bounce buffer can only allocate limited size buffer.
Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Change-Id: I850b614305d66065733381ceb7bd67d4b1cad6b3
Reviewed-on: https://review.gerrithub.io/430783
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This patch changes the name of the field. Following patches
will introduce logic that will guarantee that buffers
provided to bdev module will be aligned to value
specified in this field
Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Change-Id: I5329b9fe26ef2417bc7beae86518cc643b263f97
Reviewed-on: https://review.gerrithub.io/430782
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
SPDK poller function will return the different value for
the different status. Update the QoS poller for this change.
Change-Id: Ia384bf00a23713df663c317b1997ead441c5adcb
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.gerrithub.io/428573
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Initiailize fields later assumed to be NULL
Change-Id: I61e054dd275c6c04fb3f826adc445e56f0add331
Signed-off-by: shahar salzman <shahar.salzman@kaminario.com>
Reviewed-on: https://review.gerrithub.io/428304
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I96b5410a92f176aef11e00829fdebd36910ac2d4
Signed-off-by: shahar salzman <shahar.salzman@kaminario.com>
Reviewed-on: https://review.gerrithub.io/428302
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Large read I/O will be typical in some use cases such as
web stream services. On the other hand, large write I/O
may not be typical but will be sufficiently probable.
Currently when large I/O is submitted to the RAID bdev,
the I/O will be divided by the strip size of it and then
divided I/Os are submitted sequentially.
This patch tries to improve the performance of the RAID bdev
in large I/Os. Besides, when the RAID bdev supports higher
levels of RAID (such as RAID5), it should issue multiple
I/Os to multiple base bdevs by batch fasion in the parity
update. Having experience in batched I/O will be helpful
in the future case too.
In this patch, submit split I/Os by batch until all child IOVs
are consumed or all data are submitted. If all child IOVs are
consumed before all data are submitted, wait until all batched
split I/Os complete and then submit again.
In this patch, test code is added too.
Change-Id: If6cd81cc0c306e3875a93c39dbe4288723b78937
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/424770
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
When destroy qos, spdk_bdev_qos_destroy() allocates a new qos, and swap
old one. After spdk_bdev_unregister() frees the new qos, the old qos poller
may still reference new qos via bdev->internal.qos. Fix this error by
using old qos in _spdk_bdev_qos_io_submit().
Reported in 72aac51430.1539054028/ubuntu16.04/build.log
Change-Id: Id1bce6c8b1cefae604dd2c69e8f3482ec34b1b54
Signed-off-by: wuzhouhui <wuzhouhui@kingsoft.com>
Reviewed-on: https://review.gerrithub.io/428444
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
