Commit Graph

4877 Commits

Author SHA1 Message Date
paul luse
e2a2b637c9 bdev/compress: incorporate reducelib read and write
Change-Id: I280e9d51bf23ef101b8a3ba8a68a0c1eb10cb65c
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/438274
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-03-29 04:41:32 +00:00
Ziye Yang
b151999f06 blobfs: Add the return value check for calling cache_append_buffer
Reason: We need to detect this issue. Otherwise, it will
cause null pointer issue later

Change-Id: I4d9aec0570e0b46274ebf3b9642b4727049a4a3b
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/448594
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-03-29 02:38:33 +00:00
Jim Harris
11b38a585a app, log: clarify how to enable log flags
Let's point users to use --enable-debug now instead of
CONFIG_DEBUG=y.  Also to be completely pedantic, use
"configure" instead of "build" to make it more clear
this flag needs to be passed to the configure script,
not make.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I4a6f6313a4f8e87cbb1d79cb68645305abb0e106

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449339
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-03-29 02:35:48 +00:00
Jim Harris
df082c931e reduce: fix chunk_is_compressed calculation
Stupid bug - a chunk is compressed when the number
of io_units for the compressed data does *not* equal
the number of io_units per chunk.

Paul found this during some of his initial DPDK
framework integration testing.  I have some better
unit tests coming up to test this further, but want
to get this obvious fix in for now.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iaf9c78a1c1d6045070e7625c6a54ab3d227c8ea7

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449498
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-03-29 02:34:21 +00:00
Darek Stojaczyk
fb51565a59 env/dpdk: add spdk_pci_fini()
It's mostly needed for the next patch, but even
now it provides some value by printing errors if
there any leaked (still attached) PCI devices
at shutdown.

Change-Id: I8459a6049b3c6612d9f1d99444bf3acfd474a839
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449082
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-03-28 14:50:46 +00:00
Wojciech Malikowski
04814b72a8 lib/ftl: Internal IO retry mechanism in case ENOMEM from nvme layer
Added internal retry IO queue. In case ENOMEM from
nvme layer request is put to retry queue. When some
inflight operations are completed try to make progress
on IOs in retry queue in first place.

Change-Id: Ie9d3d20bd34431ee57f9454f242b0cdca349c804
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/448461
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-03-28 14:17:11 +00:00
Wojciech Malikowski
f8399d4539 lib/ftl: Update internal IO position in one place
Change-Id: Ie925659e8ffca1fb8aaa1252118bc670b3b274fd
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449015
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-03-28 14:17:11 +00:00
Wojciech Malikowski
e27f64fb98 lib/ftl: Use ftl_io_done() to determine if IO is completed
Change-Id: I76b58764153793f26b167071041ee17673fec63f
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449160
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-03-28 14:17:11 +00:00
Darek Stojaczyk
23d7ff31fc vhost: change vsession->lcore only within that lcore
There is currently a small window after we stop
session's pollers and before we mark the session
as stopped (by setting vsession->lcore to -1). If
spdk_vhost_dev_foreach_session() is called within
this window, its callback could assume the session
is still running and for example in vhost scsi
target hotremove case, could destroy an io_channel
for the second time - as it'd first done when the
session was stopped. That's a bug.

A similar case exists for session start.

We fix the above by setting vsession->lcore directly
after starting or stopping the session, hence
eliminating the possible window for data races.

This has a few implications:
 * spdk_vhost_session_send_event() called before
   session start can't operate on vsession->lcore,
   so it needs to be provided with the lcore as
   an additional parameter now.
 * the vsession->lcore can't be accessed until
   spdk_vhost_session_start_done() is called, so
   its existing usages were replaced with
   spdk_env_get_current_core()
 * active_session_num is decremented right after
   spdk_vhost_session_stop_done() is called and
   before spdk_vhost_session_send_event() returns,
   so some active_session_num == 1 checks meaning
   "the last session gets stopped now" needed to be
   changed to check against == 0, as if "the last
   session has been just stopped"

Change-Id: I5781bb0ce247425130c9672e0df27d06b6234317
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/448229
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-03-28 14:16:56 +00:00
Darek Stojaczyk
2cddd571ee vhost: add spdk_vhost_session_start_done/stop_done
Split spdk_vhost_session_event_done() into two separate
functions. This is just a preparation for the next patch.

Change-Id: I05e046e4b963387f058d2b822d7493c761eebbbb
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/448228
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-03-28 14:16:56 +00:00
Darek Stojaczyk
64d76e50cc vhost: call session_event_done() always under the global vhost lock
In the next patch we will put much more responsibility
on spdk_vhost_session_event_done(), so here we make
sure it's always called under the global vhost mutex.

Specifically, spdk_vhost_session_event_done() will set
vsession->lcore, which any other thread might try to
concurrently access via spdk_vhost_dev_foreach_session().

Change-Id: I7a5fde4be4e8bdfdbbb24ac955af964f516bdb68
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/448227
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-03-28 14:16:56 +00:00
Darek Stojaczyk
bbfbadf59a vhost: add spdk_vhost_trylock()
We'll make use of it inside the vhost device backend
code. The function itself is generic enough to be put
in the public vhost.h header rather than vhost_internal.h.

Change-Id: I60602c61d8bba665dcf9c6d27af2e910c208a7be
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/448226
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-03-28 14:16:56 +00:00
yidong0635
0d076e264e bdev/raid: remove unnecessary assert.
Assert(raid_bdev->base_bdev_info) at here is unnecessary.
So remove it.

Change-Id: I7d87fbc95d8c1434ab346fd11179433547b74798
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449379
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-03-28 14:14:38 +00:00
Darek Stojaczyk
6e9eca7874 virtio: switch to spdk_*malloc().
spdk_dma_*malloc() is about to be deprecated.

Change-Id: Iacf9f6536ba5baca7b245e639d0d42a89720ba58
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/448173
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-03-28 07:39:31 +00:00
Jim Harris
2edc652913 reduce: save num_io_units and chunk_is_compressed to req object
When a chunk cannot be compressed enough to save one I/O
unit, we will just write the uncompressed data to disk.  In
this case, we want the helper routine to use the decomp_buf
instead of the comp_buf.

To facilitate that, save some information in the request
object on the number of io_units to be read from or written
to disk, and whether the data is compressed or not.  We
could easily deduce if the data is compressed based on the
number of io_units but saving it in the structure will
save a few instructions and simplify the code a bit.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I1537130577943e19fe4dfcee58eac453eca3c29b

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449259
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-03-28 03:45:29 +00:00
Jim Harris
309f7791b9 reduce: plumb basic compress/decompress callbacks
The unit tests don't really try to compress anything
yet, but this at least gets the pipeline in place.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ic413850b30e4d9631f3ece2bab40d9026225e5b2

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449097
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-03-28 03:45:29 +00:00
Jim Harris
72da9b7562 reduce: use consistent callback names
There are a sequence of operations needed for both
read and write I/O.  For example, on a write I/O,
we may need to do a read, modify the chunk, then
write the data back to disk.

For read I/O, the name of the callback is
_read_read_done - the first "read" indicates the
I/O type, the second "read" indicates the disk read
part of the sequence.

The write I/O steps aren't using this same naming
convention, so change that here.  This will be
important in upcoming patches where we'll be adding
compress/decompress steps into the I/O pipeline -
having a consistent naming strategy will make the code
easier to follow.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Icd525874918378118d807acdbe5dd42d5be03126

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449258
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-03-28 03:45:29 +00:00
Jim Harris
f9ac7dcf11 reduce: move code into _read_complete_req helper function
This preps for some future changes.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I380258938e2412062b43c3d460a7322c5afdf819

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449096
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2019-03-28 03:45:29 +00:00
Jim Harris
09aff2d0fd reduce: add intermediate buffers for compression operations
We will actually use these in future patches.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I730ef90da6d93922dd17a5ac16594e66132a5f11

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449086
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2019-03-28 03:45:29 +00:00
JinYu
5874e2ac6c nvme:complete I/O and abort rest I/O before destroy io_qpair
Logically we should complete any I/O in the completion queue and
abort the rest of the I/O after we delete the submission queue
and completion queue, so that we would not lost any complete I/O.
We alse should complete I/O and abort I/O before destroy I/O qpair
even though the ctrlr/device has been removed.

Change-Id: Ieb28ad7b4a3a7be553f70178b29ca870b5413191
Signed-off-by: JinYu <jin.yu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449316
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-03-28 01:06:17 +00:00
Paul Luse
1717b5d580 bdev/compress: add reduce integration
Basic integration, init and load sequences

Change-Id: I12595a251f38168aad15e95275651cb4ce40ea7d
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/435764
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-03-27 19:17:46 +00:00
JinYu
2f3147c0b2 bdev: fix potential segmentation fault bug
As the bdev_io maybe NULL, so put the assignment after assert(bdev_io != NULL).

Change-Id: I58d6fcf97931b2f431ad5680b87b098e0c9a5e9b
Signed-off-by: JinYu <jin.yu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449296
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-03-27 18:38:52 +00:00
Paul Luse
db541f8eb3 bdev/compress: add compression vbdev module
Just the starting point, base functions and structs.

Change-Id: I2ff2b672aa675a19b89e4449f1cc4aa664007f6f
Signed-off-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/435747
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-03-27 14:51:22 +00:00
Jim Harris
20145fd714 reduce: prepend rw to request buf and buf_iov
We will need separate temporary buffers for compress/decompress.
A single temporary buffer won't do because a user's read/write
operations may not cover a full chunk - so we'll need one buffer
to read/write the compressed data, and another buffer for
the uncompressed data.

So for now, just prepend rw to the existing fields - this
will signify these fields are used for the read/write operations
to the backing storage device.  We'll add additional ones in future
patches for the buffers that will hold the uncompressed data.

Note: the vol->buf_mem and ->buf_iov_mem are not prepending "rw"
because they will be used for the compress/decompress temporary
buffers as well.  We'll just double the size of these buffers
when allocating them for the volume.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I7df56ab72769f4689f3abac3354446e9cf78d423

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449085
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-03-27 14:20:26 +00:00
Jim Harris
502ab5b66d reduce: rename vol->reqbufspace to buf_mem
This matches better with the existing buf_iov_mem.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I3b85708672553c281666becc8eaf9f9028da8782

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449084
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: wuzhouhui <wuzhouhui@kingsoft.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-03-27 14:20:26 +00:00
Jim Harris
5a04099b01 reduce: save compressed data size in chunk map
Upcoming patches will try to compress the data before
writing it to backing I/O units, and we'll need to save
the exact number of compressed bytes.

For now, since we're not actually compressing data yet,
just save the uncompressed data size to this field.  There
will be cases when we cannot realize any compression
savings and will just store the uncompressed data, so
handling this now is one less path we'll need to add later.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I499a89590349e4816fe0fef77cc1bfabde6871bf

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449078
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-03-27 14:20:26 +00:00
Jim Harris
3bb1f95570 reduce: add struct for chunk maps
Previously we were just passing around uint64_t pointers,
since the chunk map was literally only an array of uint64_t.
But we need to store the compressed data size in the chunk
as well.  Before adding the compressed data size, let's do
the prep work here to just create the chunk map structure
which for now only includes an array of uint64_t.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Icbfa6e509463d4f560444d3db8acd9e87917835f
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449076
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: wuzhouhui <wuzhouhui@kingsoft.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-03-27 14:20:26 +00:00
Vitaliy Mysak
2c651f101c ocf: batched request processing in queue poller
Process 32 OCF requests during poller invocation.
This will result in more flat events distribution.

This patch uses new API function: ocf_queue_run_single().

Change-Id: I0bc2ef783018fdbcf6f6e938fd7ca136ade61599
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/446635
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2019-03-26 23:30:31 +00:00
Vitaliy Mysak
bef0c6edd2 ocf: rpc: extend get_ocf_bdevs for multicore cases
Extend existing get_ocf_bdevs call to make it easier
  to get information about attached cores.

Implementation is based adding additional optional argument "name"
  to existing call. Based on "name" bdevs are filtered.
  Backward compatability of RPC interface is preserved.

This patch also adds tests for the case when name is given.

Change-Id: I4300ebe37e936bc5cca8e066b5f09db462a87cf7
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/444841
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2019-03-26 23:30:31 +00:00
Darek Stojaczyk
530f481259 blobstore: switch to spdk_*malloc().
spdk_dma_*malloc() is about to be deprecated.

Change-Id: I0da485ec68329ee9f819cf6dfdb8744c5993e6f2
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/448171
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-03-26 23:27:29 +00:00
Chunyang Hui
38902a5a27 rpc: add get_spdk_version rpc method
Change-Id: Iacc1766a60fb1e15a1013475f83f7f13cd45fe12
Signed-off-by: Chunyang Hui <chunyang.hui@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/448613
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-03-26 22:54:28 +00:00
Michal Mielewczyk
b949f1318e ocf: Added zeroing memory returned from mempool
OCF requires memory retrived from mempool to be filled with zeros.

When mempool is created, information about element size is stored to be used
when new memory is retrived.

This is fix for issue #671

Signed-off-by: Michal Mielewczyk <michal.mielewczyk@intel.com>
Change-Id: Ieb533e3bdae0665dae18e7b3a379da0ed843c35a
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449155
Reviewed-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-03-26 22:53:09 +00:00
Konrad Sztyber
02b0230296 bdev/ftl: treat null UUID as no UUID
Check if the UUID in the configuration is null (all zeroes) and treat it
as if no UUID has been supplied.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ic91c77591528a8aaa4cf5c0241e6bde51b3757f1
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/448628
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-03-26 12:01:48 +00:00
Konrad Sztyber
9674a7b6cb bdev/ftl: better config errors logging
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ie3814a905efa6122d56a5f2dfc9710d016e8449e
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/448713
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-03-26 12:01:48 +00:00
Shuhei Matsumoto
3c4199d6c6 env_dpdk: Run DPDK in legacy memory mode through spdk_env_opts
Some customized SPDK application needs DPDK to run legacy memory mode,
but it is  unusual. So this patch uses env_context, the opaque option
variable in the struct spdk_env_opts, and if the application sets
"--legacy-mem" to it, spdk_env_opts_init() skips adding
"--match-allocations" for DPDK 19.02 or later and skips adding
"--legacy-mem" for DPDK 18.05 or before.

Change-Id: I6f40c726c66c29f0aabfeeaecd00611954dc774f
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/448263
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-03-26 12:00:36 +00:00
Jim Harris
205111718d nvme: add nvme_pcie_qpair_ring_cq_doorbell
Similar to recently added nvme_pcie_qpair_ring_sq_doorbell.
Prepares for using this code for coalescing cq doorbell
writes in the future.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I565e103acf73b3d305e72a4440e1cc678c95faa0

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/448871
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-03-26 02:59:53 +00:00
Changpeng Liu
ca76e519f8 nvmf: verify each NVMe commands for reservation enabled's NS
The filter function can be used for IO commands, because all
the Admin commands related with reservations are not supported
in SPDK for now.

Change-Id: I44f0bf0017bafaee87d5f8ac03b0fd368f44c810
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/436941
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-03-26 02:57:11 +00:00
Seth Howell
a9fc7e1db8 rdma: use LAST_WQE_REACHED event in the SRQ path
This event is generated by NICs utilizing the SRQ feature when the last
RECV for that qpair is processed. I have confirmed this feature.

Change-Id: Ib6d6b6d02987f789b4d5dd3daf734e3351ee1974
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/448063
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-03-25 17:23:51 +00:00
Changpeng Liu
add76a3515 bdev/nvme: change hotplug poller with asynchronous probe API
Change-Id: Id198e8ec79c84743ef5025e1552c4ce45ca30445
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/445078
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: wuzhouhui <wuzhouhui@kingsoft.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-03-25 13:50:23 +00:00
Changpeng Liu
8129ede1c8 bdev/nvme: use asynchronous API to probe user specified controller
NVMe controller can be attached via asynchronous API now,
we added the RPC 'construct_nvme_bdev' with asynchronous
probe support so that the initialization can be processed
later.

Change-Id: Ic60fdde6af9c4ba9a07b874852cfba044acb06c
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Change-Id: Ic60fdde6af9c4ba9a07b874852cfba044acb06c8
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/445054
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: wuzhouhui <wuzhouhui@kingsoft.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-03-25 13:50:23 +00:00
Jim Harris
4fb2260117 lib/reduce: queue overlapping I/O to same logical chunk
Write operations are not in-place, meaning that once it
is completed, it's chunk and backing io units are freed.
Reads in progress to these backing io units, from other
I/O to the same chunk, could cause data corruption.
Simultaneous writes to the same chunk can cause similar
issues.

There are cases where we can relax this in the future -
for example, overlapped reads to the same chunk poses
no problems.  We'll leave that as a future exercise.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I3a4d9e8e3ab920f823084cd1bae91f179a4386ff
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/448333
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-03-25 04:39:26 +00:00
Vishal Verma
dfb60590c9 bdev: Add io_uring bdev module
This adds bdev io_uring module.

Change-Id: I9a8171d7c871673b189bff59a89d06679da4e191
Signed-off-by: Vishal Verma <vishal4.verma@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/445334
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-03-22 18:51:48 +00:00
Ziye Yang
fa2d95b3fe blobfs: remove the assert.
The caller function needs to judge whether the buf is
allocated. Putting assert here is easy for debugging,
but not be friendly for real use.

Change-Id: I3c8e66bc95cbd2f9bc87f26335f249c32e339da4
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/448697
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-03-22 18:51:32 +00:00
Jim Harris
b1c5a962d4 lib/reduce: create helper functions for starting read/write ops
Upcoming changes will require queuing an I/O if it overlaps
with another I/O to the same logical chunk.  So split out
code (especially from the writev path) into separate functions,
so that we can execute an spdk_reduce_vol_request object
whether it's just been created via readv/writev, or after
another I/O to the same chunk has been completed.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I2a216c92e4999313ecc9f7cf157dccd65c9bb30f

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/448332
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: wuzhouhui <wuzhouhui@kingsoft.com>
2019-03-22 18:28:43 +00:00
Jim Harris
d7c80bfb44 lib/reduce: save logical_map_index in the request object
This will help optimize some upcoming patches which
guard against overlapped I/O to the same chunk in the logical
map.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ia1370fffd3bf3d55dc9ce88a993ed157ecee272a

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/448331
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: wuzhouhui <wuzhouhui@kingsoft.com>
2019-03-22 18:28:43 +00:00
Jim Harris
1036358ecf lib/reduce: rename variable to old_chunk_map_index
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I39479f44684e2c5e6652d07237ac40949dcf0aca

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/448330
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: wuzhouhui <wuzhouhui@kingsoft.com>
2019-03-22 18:28:43 +00:00
Wojciech Malikowski
cfdc60c05a lib/ftl: Avoid splitting read's in case contiguous PPAs
Change-Id: I026902096686da37c0ca7086bb5d91b35d90ffa6
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/447340
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Young Tack Jin <youngtack.jin@circuitblvd.com>
Reviewed-by: Claire Jihyun In <claire.in@circuitblvd.com>
2019-03-21 20:11:35 +00:00
Changpeng Liu
2e6dbe7539 nvme: reduce default Admin timeout to 30 seconds
120 seconds is too long for controllers which can't be
setup during initialization, because this value is only
used for Admin commands so also rename as it is.

Change-Id: I0a3d3192252c0f6fc0bef4d8b868eaef2ae40fe3
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/448601
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-03-21 18:22:28 +00:00
Changpeng Liu
6129e78d26 nvme: small code cleanup for nvme_transport_ctrlr_scan
The identify data is only valid if spdk_nvme_connect()
was used with Discovery Controller, so move this code
into the section where it belongs to.

Change-Id: I1897f38277eafc192552a09556a568e9152bb72d
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/448500
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-03-21 18:22:28 +00:00
yidong0635
fc43fbba04 rdma: fixed heap used after free issue.
With ASAN to run this cases, it will report issue about heap used after free
in spdk_nvmf_rdma_qpair_destroy. Resources have been released before,
change the order to in this tailq to release resources.

ERROR: AddressSanitizer: heap-use-after-free on address
0x6080000080e0 at pc 0x0000006e1e3f bp 0x7fd48b6c3df0 sp 0x7fd48b6c3de0
READ of size 8 at 0x6080000080e0 thread T3 (reactor_1)
0x6e1e3e in spdk_nvmf_rdma_qpair_destroy spdk/lib/nvmf/rdma.c:813

Change-Id: Ia1c12bca84955a2de60399e6b265c9b8901bb51e
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/448534
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-03-21 18:00:04 +00:00