ivampiresp/Spdk
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

vhost_blk: speed up the vhost device boot

Browse Source 
When starting qemu with vhost-user-blk multiqueue(set num-queues to
more than 1), the vhost device will be started/stopped many times
(related to the queue num), as the vhost-user backend doesn't know the
exact number of queues used for this device.
The target have to stop and start the device once got a valid IO queue.

When stoping and starting the vhost device, the backend bdev io device
will be deleted and created repeatly.

If the backend bdev is a distribution system, the cost is large as the
network RTT.

In this patch, add a dummy_io_channel to hold a reference to the io
device, so that the io device will not be deleted.

Change-Id: I5737248ec52bee06342ff0873bb89fd0a51665c2
Signed-off-by: Li Feng <fengli@smartx.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2020
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
This commit is contained in:
Li Feng 2020-04-24 22:19:23 +08:00 committed by Tomasz Zawadzki
parent b0bac20ad6
commit bcc28e05d1
1 changed files with 23 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

23
lib/vhost/vhost_blk.c
View File

@ -93,6 +93,8 @@ struct spdk_vhost_blk_dev {
struct spdk_vhost_dev vdev; struct spdk_vhost_dev vdev;
struct spdk_bdev *bdev; struct spdk_bdev *bdev;
struct spdk_bdev_desc *bdev_desc; struct spdk_bdev_desc *bdev_desc;
/* dummy_io_channel is used to hold a bdev reference */
struct spdk_io_channel *dummy_io_channel;
bool readonly; bool readonly;
}; };
@ -816,6 +818,7 @@ vhost_dev_bdev_remove_cpl_cb(struct spdk_vhost_dev *vdev, void *ctx)
struct spdk_vhost_blk_dev *bvdev = to_blk_dev(vdev); struct spdk_vhost_blk_dev *bvdev = to_blk_dev(vdev);
assert(bvdev != NULL); assert(bvdev != NULL);
spdk_put_io_channel(bvdev->dummy_io_channel);
spdk_bdev_close(bvdev->bdev_desc); spdk_bdev_close(bvdev->bdev_desc);
bvdev->bdev_desc = NULL; bvdev->bdev_desc = NULL;
bvdev->bdev = NULL; bvdev->bdev = NULL;
@ -1246,10 +1249,24 @@ spdk_vhost_blk_construct(const char *name, const char *cpumask, const char *dev_
goto out; goto out;
} }
/*
* When starting qemu with vhost-user-blk multiqueue, the vhost device will
* be started/stopped many times, related to the queues num, as the
* vhost-user backend doesn't know the exact number of queues used for this
* device. The target have to stop and start the device once got a valid
* IO queue.
* When stoping and starting the vhost device, the backend bdev io device
* will be deleted and created repeatedly.
* Hold a bdev reference so that in the struct spdk_vhost_blk_dev, so that
* the io device will not be deleted.
*/
bvdev->dummy_io_channel = spdk_bdev_get_io_channel(bvdev->bdev_desc);
bvdev->bdev = bdev; bvdev->bdev = bdev;
bvdev->readonly = readonly; bvdev->readonly = readonly;
ret = vhost_dev_register(vdev, name, cpumask, &vhost_blk_device_backend); ret = vhost_dev_register(vdev, name, cpumask, &vhost_blk_device_backend);
if (ret != 0) { if (ret != 0) {
spdk_put_io_channel(bvdev->dummy_io_channel);
spdk_bdev_close(bvdev->bdev_desc); spdk_bdev_close(bvdev->bdev_desc);
goto out; goto out;
} }
@ -1270,6 +1287,12 @@ vhost_blk_destroy(struct spdk_vhost_dev *vdev)
int rc; int rc;
assert(bvdev != NULL); assert(bvdev != NULL);
/* if the bdev is removed, don't need call spdk_put_io_channel. */
if (bvdev->bdev) {
spdk_put_io_channel(bvdev->dummy_io_channel);
}
rc = vhost_dev_unregister(&bvdev->vdev); rc = vhost_dev_unregister(&bvdev->vdev);
if (rc != 0) { if (rc != 0) {
return rc; return rc;