data_ro means that write, write_zeroes and unmap operations are not
allowed.
md_ro means that resize, set_xattr and remove_xattr are not
allowed.
There is no code yet that can activate this - it is coming in a future
patch. Two usages are planned though:
1) a user explicitly marks a blob as read-only - this is persisted so that
future loads of the blob will ensure the blob cannot be modified - neither
metadata nor data
2) a future feature flag framework (how's that for alliteration) may allow
a blob to be opened, but not allow metadata modifications, if there are
feature flags set in the blob's or blobstore's metadata that the
application does not understand
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I247fd900430c56f7176edfb80dddd5a1a6c8dc87
Reviewed-on: https://review.gerrithub.io/388663
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Make this generic and not directly dependent on
the event framework. That way our libraries can
register pollers without adding a dependency.
Change-Id: I7ad7a7ddc131596ca1791a7b0f43dabfda050f5f
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/387690
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
At the moment there was no way to a user of blobstore api to know,
how many clusters are availible to him. Total_clusters describes
number of clusters for metadata and user data.
New field added total_data_clusters, keeping number of clusters
that can be used to create blobs - meaning just user data.
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I60555217644557410844f74628375a6b46fd2ac7
Reviewed-on: https://review.gerrithub.io/385633
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Change-Id: I59cbef4ce1bfe8af113c66c2c9cb9f208440c0aa
Reviewed-on: https://review.gerrithub.io/383887
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
It is possible that a user will call spdk_bs_unload() with blobs
list not-empty. Instead of just asserting that, now the call fails
with appropriate error.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I83818453d6c90ff9b5bf657c90e12b2f9d5ca013
Reviewed-on: https://review.gerrithub.io/383220
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Currently exposed API allows to load/unload and to
initialize blobstore on a device.
A spdk_bs_destroy() call is added in order to reach
functional parity with spdk_bs_init(). It was not
possible to remove blobstore from device from within
SPDK previously.
spdk_bs_destroy() takes blobstore pointer as argument
(instead of bs_dev), because blobstore has to be already
loaded to destroy it.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I2c493a4407868fcf08fd1766a19fc8463f634ef5
Reviewed-on: https://review.gerrithub.io/382019
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
We only sync the metadata and data in the runtime of blobstore, which
means we only update the used md bitmap and used clusters bitmap in memory.
if the system crushed, we have no chance to sync the used md bitmap and
used clusters bitmap into disk, then next time when we try to load the
blobstore, all the data will lost, this patch add the logic to recover the
valid data from last dirty shutdown. We will go through all the metadata pages
to find all valid data and rebuild them.
Change-Id: Ieb7c5f932206b1b68fdde0cee35f2d2cb3a4f309
Signed-off-by: Cunyin Chang <cunyin.chang@intel.com>
Reviewed-on: https://review.gerrithub.io/376470
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Introducing bstype as a way to identify and verify
blobstore type.
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I50267b5408625be10fe0c146ae329016d5509b4a
Reviewed-on: https://review.gerrithub.io/380476
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Fixes github issue #29.
Because of how we handle the blobid and pagenum in blobstore,
it was possible to have blobstore inadvertently open the wrong
blob if open is provided a blobid where the lower 32 bits match
an existing blob but the upper 32 are clear.
Patch does the following:
- removes assert() that caught this on MD load and replace with
an error given that this condition can be induced via the API
- cleanup of pagenum and blobid conversion/handling to make it
clearer how they're related and converted
- new UTs that would have failed w/o the new check in place
Change-Id: I2b49b237922b3b8cfc4df296f5bc20195e41dc41
Signed-off-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/380872
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Three checks are added to options passed to spdk_bs_init,
with appropriate errors returned:
- whether any of the options is set to 0
- device size has to be bigger than cluster size
- pages reserved for metadata exceed total number of clusters
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Idee3c194b653e737ec7c7a768f1973ff72452c5b
Reviewed-on: https://review.gerrithub.io/379676
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Now bs_dev is destroyed only in two instances:
- within spdk_bs_init() on failure path
- vbdev_lvs_create() if spdk_lvs_init() errors out,
before even calling spdk_bs_init()
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I7b8af39fbe83907b0c47797f0f55ca3b941729d9
Reviewed-on: https://review.gerrithub.io/379848
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Fixes condition where blobstore was prematurely calling the
application callback on spdk_bs_unload(), if the application
tries to do something too quickly bad things happen.
To avoid application changes with how the g_devlist_mutex is
held, it is no longer held while calling
_spdk_io_device_attempt_free() because the app unload CB is
called from that function and may want to call
spdk_io_device_unregister() from its unload CB. So the lock
is now held and releases strictly around the list its
protecting which allows the CB from _spdk_io_device_attempt_free()
to be called without issue.
Change-Id: Ib451cfe6b33ea0c3f9e66c86785316f9d88837c7
Signed-off-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/377872
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
This patch add support crc for metadata pages, we will also
add crc for supper block, used md and used clusters bitmask
pages in the following patches.
Change-Id: Ie36fcc16b39296d06721f1f8eb5689260194c558
Signed-off-by: Cunyin Chang <cunyin.chang@intel.com>
Reviewed-on: https://review.gerrithub.io/377901
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
The system could crash anytime, we need sync the "clean" flag
into disk as soon as we load the blobstore. Then if the system crashed,
we will find the blobstore is not clean shutdown next time when we load
the blobstore, and we could run the recover process then.
Change-Id: I6189678e970ffe979a224e02be6cede0ee44dde8
Signed-off-by: Cunyin Chang <cunyin.chang@intel.com>
Reviewed-on: https://review.gerrithub.io/376276
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
This moves the thread name setting code into the generic SPDK thread
setup code, so now all spdk_threads can be named, not just ones created
by the event framework.
Change-Id: I6c824cf4bcf12fe64a8e2fc7cdc2d6c949021e40
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/375220
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I9b1c8a1be8e6bce705ba6c549957e82e492038cc
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/375623
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Adding ability to delay scheduled tasks until _bs_flush_scheduler is called.
This allows to test blobstore when tasks do not finish immediately.
New unit test added for spdk_bs_unload.
Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I9c1b3c0c2ebb1e16c032fd136d11ef351d8f3fe0
Reviewed-on: https://review.gerrithub.io/374767
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Explicitly allocating spdk_bs_dev on init_dev() allows to check if dev_destroy for it was called.
If it was not, then ASAN will provide information on that.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I333185958dd8c29180954077a6c326b82e5949ae
Reviewed-on: https://review.gerrithub.io/374766
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Most of the work here revolves around having to split
an I/O that spans a cluster boundary. In this case
we need to allocate a separate iov array, and then
issue each sub-I/O serially, copying the relevant
subset of the original iov array.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I0d46b3f832245900d109ee6c78cc6d49cf96428b
Reviewed-on: https://review.gerrithub.io/374880
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Fail spdk_bs_init() if the dev being used has an
LBA size that is larger than a metadata page or not evenly
divisible by the size of a metadata page.
Change-Id: I0e0ca747ecd5b6039c20fb6a885382bde4527158
Signed-off-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/374182
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: I151672e49c442dc1420b5ef6b5a6a47eb78e7464
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/366517
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
