This new UT is checking validity of md sync being run
along another md sync with changed metadata.
At this time added case where first md sync is being
interrupted by removal of xattr with second md sync.
This interruption is perfomed at increasing number of
poller executions, until the number is enough to
complete first md sync.
There are two expected states of used_md_pages array,
either with xattr or without. The state is verified
after md syncs.
This UT will be expaned in similar manner by other
operations (than xattr) that casue changes in persisted metadata.
Related to #960
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I251dca92ffc3080d8dc503a7f1ff342aa59adef9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/774
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Making use of recently added utility to poll exact number of times.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I23a9dc702c899285626522ace8ec77a9ba0f7757
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/773
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
spdk_put_io_channel() was designed to be called on the same thread
that called spdk_get_io_channel(). spdk_put_io_channel() sends a
message to its own thread, to allow the context to unwind before
releasing the resources. This had the side effect to allow an
incorrect thread to call spdk_put_io_channel(). This patch will fix
that.
Bdevperf tool had a design flaw that needed the side effect, but
it was fixed recently. We do not know if we have any other case.
Hence add assert to spdk_put_io_channel() to find other case.
We found that unit test for blobstore had called
spdk_put_io_channel() and fix it together in this patch.
Besides, correct the comment for spdk_put_io_channel() in
include/spdk/thread.h not to create any other case in future.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I6ec7bf074818abef43b23ca40bc9385adac70a75
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/479390
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Extent table and extent page descriptors are now
set to be default way clusters are serialized on disk.
With this patch UT are ran with and without
extent table.
Changed two asserts in test, since amount is dependent on
which type of serialization is used.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ica58fce6a4effd014d7dd40ee26edd0fa3196d0f
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/481901
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
This will be used to add another run of whole UT suite
with extent pages on/off.
Next patch in series will be enabling both types of
extent serialization for all UT.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ia8b4b8822edefb90ffc13cf777885f9af95e4545
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482170
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
All the non-ext version of the call is doing, is calling
ext with NULL as opts. Then default opts are used in
its place.
This change was facilitated by next on in series,
where all blob opts will be initalized in UT
with parameter use_extent_table either set to
true or false.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I62b642c1808b38a5f7c94a5900f25f4978a4ec39
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482859
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
After creation of blobs in both tests, only clusters indexed from
0 to 10 are supposed to be used. Index 0 for md and 1-10 for data
of single blob since it was create thick provisoned.
Cluster allocations are done in order so if there was a bug for
overflow amount of clusters claimed, first in order would be
one with index 11.
This patch adds asserts after each bs load for first data cluster
that is supposed to be used and for first data cluster that is
not.
During the tests those should remain constant.
When creating/deleting snapshots, the blobs are affected
by changing their type to/from thin_provisioned.
Added asserts to verify their state at every blob open.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I38418da55850d5b8468e578b3c42c5b817ae8045
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482661
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
g_bserrno from blob deletion or snapshot creation,
should not be checked. It is implementation
dependent whether the error (or success) from those
calls actually means that enough data was persisted
on disk.
This test case should work even if we set the threshold
high enough that no failed opperations occur.
On the other hand some parts of those calls do cleanup
in them, meanwhile there is enough metadata data on disk already.
Such as cleaning up unused clusters or pages issue
writes, but at that point the blobs already are in expected
state.
Thus removed assert for g_bserrno, as failure is not indicative
of impossibility to recover.
While here, removed the spdk_bs_unload(). This UT are for
testing power fail safety. Never should it be the case that
enough writes occured in create/delete, but blobs are not
in the expected state.
When such bug would be introduced, it could be covered up
by spdk_bs_unload() cleanly closing up the blobstore.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ic69c3061f2cc1fe04bf895632cdb11efb2fe6912
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482660
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Extent Pages claim and insertion can be asynchronous
when cluster allocation happens due to writing to a new cluster.
In such case lowest free cluster and lowest free md page
is claimed, and message is passed to md_thread.
Where inserting both into the arrays and md_sycn happens.
This patch adds parameters to pass the Extent Page offset
in such case.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I46d8ace9cd5abc0bfe48174c2f2ec218145b9c75
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/479849
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Accept a clear method option on blob create by adding clear_method
to the opts structure passed in to _spdk_bs_create_blob(). Store
these 2 bits in md_ro_flags so that earlier versions without an
understanding of these bits can not alter metadata.
The new metadata values will be used later in the series.
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I5440645ca20b426778d13b2e544b65dc2b3b83c7
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/472204
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Previously this UT was only checking the snapshot counts.
This was correct, but since the blobs in this tests are
clones of each other - it would make sense to check it as well.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Id9dec7f0a292cbb52da8196bfc77679fc0f49d8b
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478976
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
For deletion to actually deliver results, poll_threads()
has to be called.
g_bserrno came from previous API call, not delete itself.
This patch fixes it so that g_bserrno is from the delete itself.
Next patch in series adds checks for number of clones.
Having such checks in first place,
would allow to notice lack of poll_threads() here before.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I3477f5c5b99106b70ee3d5706906a4dc94283180
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478975
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This patch fixes minor unit test deallocations around io_channels.
It has not exposed any issues in blobstore code, but should always be right
to prevent covering up any issues later.
1) Two spdk_bs_free_io_channel() were missing
2) Dirty shutdown should still free bs resources with _spdk_bs_free()
Sample log at the end of UT, before this patch:
thread.c: 200:_free_thread: *ERROR*: thread 0x617000000080 still has channel for io_device blobstore
thread.c: 200:_free_thread: *ERROR*: thread 0x617000000080 still has channel for io_device blobstore
thread.c: 200:_free_thread: *ERROR*: thread 0x617000000080 still has channel for io_device blobstore
thread.c: 180:spdk_thread_lib_fini: *ERROR*: io_device blobstore not unregistered
thread.c: 180:spdk_thread_lib_fini: *ERROR*: io_device blobstore not unregistered
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I187bc61bb6e094c9c740a987e7d14760551a0503
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/475872
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Previously blob_insert_cluster_msg UT specified the cluster
which will be used to store LBA. So for blob the cluster [1],
was always stored as cluster 0xF in blobstore.
This patch changes _spdk_bs_claim_cluster() with preselected
cluster in blobstore to _spdk_bs_allocate_cluster() that
will choose first free one in the map.
It will help in further patches that add more logic
on extent table and extent pages in _spdk_bs_allocate_cluster().
Otherwise this patch 'hard coded' values might get more complex
than needed.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I238329b2191e7ebd3f73c1fe85d7e1da0249979e
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/475493
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
When serializing extents, run-length encoding is supposed to
1) RLE all sequential LBAs
2) RLE zero LBAs (unallocated)
There is one special case, with sequential LBAs that start
with 0 LBA. This is RLE as 1) case, but results in descriptor
matching case 2). Which causes loss of allocated clusters.
This requires following conditions to be met:
- blobstore has just a single cluster reserved for MD
- blob is thin provisioned
- first allocation occurs on cluster_num=1
For last part to be true, very first write for blob has to be
issued to LBA between cluster_size and 2*cluster_size.
Causing allocation of second cluster in blobstore and assiging
it LBA equal to number of LBAs per cluster.
To fix this, case 1) disallows to RLE zeroes.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I136282407966310c882ca97c960e9a71c442c469
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/475494
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This was inconsistent on all CU_ASSERT macros,
so now it is fixed.
Change-Id: I6d20f98528c4041baf02cb419ab9644a45b807c4
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459760
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Blob id only is matched to the very first page of md for
that particular blob.
During loading blobstore, we shouldn't verify
further pages in chain against the blobid.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ifc7863ddcb403aedc264c14e6b4c3915bd30dc41
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/460607
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Length of xattr descriptor is equal to length of xattr struct,
xattr name and the len of stored value.
There is no limit to how much can be stored in memory for xattr.
On disk xattr size is limited to single page and within that to
max descriptors that can fit in it.
This size is known at compile time.
Before this patch it was possible to add xattr exceeding
what was possible to be written to disk. This caused issues
when serializing the metadata during spdk_blob_sync_md()
or spdk_blob_close(). Making those fail without specific info
to the user and not actually writting such descriptor.
Since maximum length of xattr descriptor is known at compile time,
this patch compares against this value when setting the xattr.
It will immediately report back to user with error, and will
not store xattr in memory (thus not serialize it).
This patch should not affect any backward compatibility for blobs.
Too large xattrs weren't written to disk before,
API for blobstore stays the same - only reporting ENOMEM
when it should.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I6f4af4d079e47f084e20d7a4969d9a78ec1f8610
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/460450
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Currently we are missing cleanup routine for case when
power failure interrupts creating snapshot. This patch
add such routine.
For the case where we find blob with a parent snapshot ID
matching newly created snapshot we can finish whole process
during recovery by processing forward with setting snpashot
as read only, removing xattr and syncing. We should remove
snapshot only if there is no blob with parent pointing at
snapshot.
Fixes github issue #760
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I2f0e298164e07a2b4dfa5367e8878facef640702
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455216
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
This is unit test for power failure event during snapshot
deletion. At the same time this is an example how to
use new power failure event functionality in unit tests.
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I9212392f665576fa16edd28c609199d0e02dc434
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/454608
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Starting with this patch it is possible to remove a
snapshot if there is only a one clone created from it.
In such case snapshot can be removed without any data
copying. This is achieved with following steps (case
with only one clone):
1. Open snapshot (Snapshot1) that shall be removed
2. Check if the Snapshot1 has no more than 1 clone (Clone1)
3. Remove Clone1 entry from Snapshot1
4. If the Snapshot1 has a parent snapshot (Snapshot2):
4a. Add Clone1 entry to the Snapshot2 clones list
4b. Remove Snapshot1 entry from Snapshot2 clones list
5. Open Clone1 blob
6. Freeze I/O operations on Clone1
7. Temporarily override md_ro flag for Snapshot1 and Clone1
for MD modification
8. Merge Snapshot1 and Clone1 clusters maps into Clone1
clusters map
9a. If Snapshot2 is present switch parent ID and backing
bs_dev on Clone1
9b. If Snapshot2 is not present set parent ID to
SPDK_BLOBID_INVALID and backing bs_dev to zeroes_dev
10. Sync MD on Clone1
11. Sync MD on Snapshot1
12. Restore MD flags for Clone1 and Snapshot1
13. Unfreeze I/O on Clone1
14. Close Clone1 blob
15. Remove Snapshot1
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I800724b981af894e01e1912d0077c5b34a2ae634
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/445576
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
In some cases user may want to flag blob for removal
then do some operations (before removing it) and while
it happens there might be power failure. In such cases
we should remove this blob on next blobstore load.
Example of such usage is delete snapshot functionality
that will be introduced in upcoming patch.
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I85f396b73762d2665ba8aec62528bb224acace74
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453835
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This patch verifies that functionality of only releasing cluster
that was originaly claimed in race thread condition.
Change-Id: I4d60ffdb4fd6612d61a761835aae2fd1bf81b7fd
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452626
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Block operation that should not be done simultaneously.
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I3cab510377a49be4e5847ba37a6218f0025c0db6
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/450014
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
1,add a judgement to ensure spdk_blob_close() runs correctly.
2,make sure the command has been completed before _spdk_bs_free()
which would avoid heap-use-after-free.
Change-Id: I698521d8d4d8e1d8a7374665d27d6eec6d0b85f5
Signed-off-by: JinYu <jin.yu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449517
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
spdk_thread_poll()
This is an optimization if the calling function already knows the
current time.
Change-Id: I1645e08e7475ba6345a44e0f9d4b297a79f6c3c2
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/443634
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Now that spdk_thread_poll can be called from within
a thread, some of the unit tests can be a bit simpler.
Change-Id: Ib7b92c3f59862e5c128f5e0cc8a08f65e2b89e93
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/441160
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
When last clone of a snapshot is being deleted
we remove that snapshot from snapshots list.
We should not do that as it still works as a
snapshot and it is read-only, but it does not list
as a snpashot from get_bdevs. Instead remove snapshot
entry from the list when blob that represents that
snapshot is being removed.
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I8d76229567fb0d9f15d29bad3fd94b9813249604
Reviewed-on: https://review.gerrithub.io/436971
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Change-Id: I994b5d46faffd34430cb39e66225929c4cba90ba
Reviewed-on: https://review.gerrithub.io/414935
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Valgrind found this on my system. The unit test itself
was passing correctly (we wrote uninitialized data and then
successfully read back the same uninitialized data) - but
Valgrind doesn't like code that makes any decisions based
on reading uninitialized memory.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I762f5dd0cb368328bd5eba9cecf7db404b457fad
Reviewed-on: https://review.gerrithub.io/424109
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
This patch adds UT for IOV split operation.
Next patch will refactor split of vectored read/write a little.
Change-Id: I77fbaf042fd894b23ad5ae846f54c28498e8e62b
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/422768
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Unify the regular mocks and the pointer mocks. Simplify
several of the #defines.
Change-Id: Ica8c69dbb70a685a55b5961b73fd7872f451c305
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/418884
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Previously cluster sizes only set as default were tested,
this patch adds test for load when different cluster_size is used.
Change-Id: I585adff25c81de809435ef235314cd4b8b116261
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/420607
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
It is not an error if bdev is smaller than cluster size so convert this
error to infolog. This fixes false error message dring examine process.
To return proper error message when creating blobstore using RPC the
_spdk_bs_alloc was adjusted to return errocode that is propagated up and
converted to "No space left on device".
Fixes#316
Change-Id: Ic9803720a55125fcfa34263346f2d9e1aae03a53
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/420054
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
This patch prevents to copy cluster data when there is not
backing blob to improve cluster allocation performance
in thin provisioned blobs.
Change-Id: Ie766d2e5274daa74c2b13b2198a20205e3417467
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-on: https://review.gerrithub.io/417938
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Before this patch when serializing extents,
unallocated clusters were treated as separate lba.
This caused metadata to grow without need.
Change-Id: I5d66466dda5f5e6d4d53f4ed5bd0bac18c74be96
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/419180
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
This patch adds more intensive unit tests for reading and
writing splitted operations.
Change-Id: I64921fa31689e660281674e8b2ad4253faa7cf04
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-on: https://review.gerrithub.io/410031
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Save the original size of the disk to metadata when it is first created.
On load verify that the disk did not change size.
Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Change-Id: I535940ee188425ee3b394effd99653cc073d541e
Reviewed-on: https://review.gerrithub.io/410896
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This patch adds an API to decouple blobs parent removing dependency
on it. Blob stays thin after this operation.
Also unit tests for blobstore inflate are improved and reused with
decouple parent functionality.
Change-Id: I96dfee467c78cf4f4d929ec7bc05263f7a23a8aa
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-on: https://review.gerrithub.io/410829
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
When snapshot is created from a clone, clones parent is not
inherited.
This patch also updates unit tests covering this case.
Change-Id: I42eb00fe2f33504c5b5c5beded20c4ea65eaff67
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-on: https://review.gerrithub.io/414804
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Change-Id: I6182eb3a77d23db7088703492d71349e3a4b6460
Reviewed-on: https://review.gerrithub.io/399366
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
The patch disables writing dirty bit during blobstore loading.
Instead, dirty bit is written prior to the first metadata update.
Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Change-Id: I7be81009a99f09048bf23749c8f6ef5e9f7b3751
Reviewed-on: https://review.gerrithub.io/410884
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Change-Id: I26e9d1e86c93f0b96b5bff7c9af5a453199c3e29
Reviewed-on: https://review.gerrithub.io/408138
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Inflate call can be used on thin provisioned blob or clones.
Function allocates all unallocated clusters on specified blob and:
- For clones, copies data from backing blob.
- For thin provisioned blobs, clusters are zeroed.
After this call all dependency from specified blob is removed
what allows deletion i.e. snapshots.
Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Change-Id: Ibff569e45b12068b2fb46557156be348b36c252b
Reviewed-on: https://review.gerrithub.io/399367
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Thin provisioned blobs mark unallocated clusters with
cluster ID 0. During recovery from a dirty shutdown,
we must not try to claim cluster 0 - we should ignore
them instead.
Fixes issue #291.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: If0dd42416f5de8d9972073bf6ed44eb8bc655415
Reviewed-on: https://review.gerrithub.io/410065
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
