Commit Graph

5092 Commits

Author SHA1 Message Date
Jim Harris
b6206d657c trace: shorten max name from 44 to 24 characters
This restriction helps reduce the amount of padding when
printing out the event trace, allowing it to fit in a
small number of columns.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ifa31e5a6967c7b9bc7028069effb71533f80596f

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452736
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-05-02 08:41:56 +00:00
Jim Harris
11ca2e0d2c trace: print error message if event name is too long
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I8ecf371700c8c0ca5bae03e393b0c32bca258170

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452735
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2019-05-02 08:41:56 +00:00
Jim Harris
617184be3b trace: remove short_name
This was not used by any of the trace register descriptions.
Let's remove it rather keeping it around if we don't need it.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Idda809e2911db5be555ff6aa13695484a14bf665

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452734
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2019-05-02 08:41:56 +00:00
Jim Harris
af25204404 bdev/nvme: always enable FTL
FTL doesn't have any kind of special package requirements.
It is getting pretty good traction in the community, so
let's enable it by default.

Note that we will disable FTL on FreeBSD.  FTL uses
CIRCLEQ which is not available on FreeBSD.  Let's not
spend time trying to get FTL to work on FreeBSD until
there's a demand to do so.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I15525b6c4e6ee52f49adf74d55f9484fe08a6dcc

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452752
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-05-02 08:41:56 +00:00
Ben Walker
8df1f6b220 iscsi: Rename spdk_iscsi_conn_migrate to spdk_iscsi_conn_schedule
This function is in charge of scheduling new connections
onto cores. It used to be used to dynamically migrate
connections, but we no longer do that.

Change-Id: I2fa04a7dfb43bf9fae7902376ba5ba6d18744469
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452727
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-05-02 08:33:48 +00:00
Ben Walker
31dd18697a iscsi: Use threads in iscsi_conn_check_shutdown
This can use threads instead of events.

Change-Id: I8a55ad512038dd75218a9a6af17d135eba6949f7
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452726
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-05-02 08:33:48 +00:00
Anil Veerabhadrappa
2061874474 lib/nvmf: Validate requested SQ size for both admin and IO queue
During connect call based on queue type (AQ or IOQ), SQ size should be
validated against max sq size for that particular queue type.

Change-Id: I977d7556e4d04e37004d16c87efffd3b467fa62c
Signed-off-by: Anil Veerabhadrappa <anil.veerabhadrappa@broadcom.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452376
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-05-01 18:51:28 +00:00
Seth Howell
6cc18a64aa rdma.c: Don't set recv->qpair to NULL
We can use the rpoller->srq to check if a qpair is valid when processing
recv completions.

Change-Id: I6aa360adc48a3312ddcf79f10e2a65b502a7314f
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452247
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-05-01 18:48:13 +00:00
Darek Stojaczyk
7b3f411063 thread: fix deadlock in spdk_for_each_thread error handling
We could return from spdk_for_each_thread() without
unlocking the global threading mutex, rendering the
entire SPDK application useless.

Change-Id: I641c7b36db996f9a36310332279648014579de83
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452087
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-05-01 18:46:34 +00:00
Darek Stojaczyk
0dcc507778 copy: declare some global variables as static
They're only used in that single file.

Change-Id: I929d00156bdd0e81adaac0281560d7f6e13394f8
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452425
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-05-01 18:45:29 +00:00
Konrad Sztyber
a37999b44b lib/ftl: clear flush bitmap during IO completion
Change-Id: I073af62b0af58d2e63bc5d8df8d13fd8ecf94ec9
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/451155
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-05-01 18:35:40 +00:00
Darek Stojaczyk
07689e948c scsi: alloc temporary data buffer with regular calloc
spdk_dma_malloc() is not required here, as the buffer
is neither DMA-able nor shared between processes.

The buffer is used to store the scsi response as it's
prepared. At the end it's always memcopied to possibly
multiple DMA-able buffers in the scatter-gatter list
provided by the upper layer.

Change-Id: I92fcede67dc24c532b13399f6364b071b2aeeb56
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/451555
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-05-01 17:59:34 +00:00
Ben Walker
1b6ddcc713 nvme: Use AVX 512 instruction to copy SQE
Change-Id: I54521ad7af0ac51cbf5e54b019ab3e8f03ee90b8
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/451782
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-05-01 17:59:00 +00:00
James Bergsten
2d4f7e20a6 nvme: add spdk_nvme_ctrlr_cmd_io_raw_no_payload_build
Send the given NVM I/O command, I/O buffers, lists and all to
the NVMe controller.

This is a low level interface for submitting I/O commands directly.

This can only be used on PCIe controllers and qpairs.

This function allows a caller to submit an I/O request that is
COMPLETELY pre-defined, right down to the "physical" memory buffers.
It is intended for testing hardware, specifying exact buffer location,
alignment, and offset.  It also allows for specific choice of PRP
and SGLs.

The driver sets the CID.  EVERYTHING else is assumed set by the caller.
Needless to say, this is potentially extremely dangerous for both the host
(accidental/malicionus storage usage/corruption), and the device.
Thus its intent is for very specific hardware testing and environment
reproduction.

Signed-off-by: James Bergsten <jamesx.bergsten@intel.com>
Change-Id: I595fe02fe0dfa9c3ceba1ac116b6900357b02d2c
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/451994
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-05-01 17:47:27 +00:00
Ziye Yang
5f1e468117 bdev/crypto: clean up _complete_internal_read
Also remove an unnecessary empty line and fix typos

Change-Id: I54e63e39dda23063c3fcbdd709cafec4278b130a
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/451921
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-05-01 17:46:46 +00:00
Seth Howell
33f60621af lib: resize key mempools
Mempools are based off of a ring structure which allocates its elements
as a power of two. It also only exposes n-1 elements to the user. So
when we create a mempool with 2^n elements in it, we have to allocate a
ring with 2^n+1 entries. By decreasing the number of elements in these
key mempools by 1, we can save a decent amount of memory.

Change-Id: I942c9dd4cf59096969bc2559fb46fd2084a07f09
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/448875
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-05-01 17:45:29 +00:00
Vitaliy Mysak
31b367ba97 ocf: add callback argument to vbdev_ocf_construct()
Add callback for construct function.
It is not asynchronous yet, but will be after
  adopting asynchronous OCF API

Also adopt RPC construct call for this change.
This is done in this patch because (1) change is small (2) leaving
  implementation as TODO still requires implementing a mock because we
  decided that callback should be required rather than optional.

Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Change-Id: Ib6a78e2caf34ac057d4da53ad5dda47163e8a089
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452146
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-05-01 17:44:03 +00:00
Vitaliy Mysak
8d3d32b982 ocf: pass vbdev in callback of mngt_ interface
Add vbdev as another argument in callback of vbdev management operations.
This is nice for async version of vbdev_ocf_construct() which uses mngt_
  interface and creates vbdev structure.

Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Change-Id: I8c88e0a881855b9f22fcf1f8174c888f3cfd6828
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452145
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-05-01 17:44:03 +00:00
Darek Stojaczyk
57706d1824 thread: add spdk_thread_get_cpumask()
Change-Id: I2811bdbbecb28c83403ebe94834afd9a808f12a4
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452203
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-05-01 17:36:44 +00:00
Jim Harris
f71fed216c blobfs: add trace events
For now, just do events for open, close and start/end
for writing the length xattr.  These will be useful for
debugging the intermittent RocksDB failures.

Note these events may only be recorded on the polling
SPDK thread.  SPDK events cannot be recorded on
non-SPDK threads (i.e. RocksDB user or background threads).

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I72fb302db23b399f583f56194c237340587f3027

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452262
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-05-01 17:10:43 +00:00
Jim Harris
dd1c38cc68 trace: allow arg1 to represent a string
This requires changing arg1_is_ptr to arg1_type.
We will use this to print the first 8 characters of
a blobfs filename when collecting event trace data.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I1b321d99145e82b42dcf6d901ce9d6298158edae

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452259
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-05-01 17:10:43 +00:00
Tomasz Zawadzki
fde382d1ee blobstore: release same cluster as claimed during initial insert
When new writes come from different threads, cluster allocations can
happen many times at once. The corresponding cluster number for the map
is determined via _spdk_bs_allocate_cluster() and kept in ctx->new_cluster.

The cluster itself is inserted into the map only on md_thread.
When there is conflict of two threads allocating same cluster,
message is returned to the losting thread to release the cluster.

Before this patch, on such failure the cluster to release
was calcualted from the page. This resulted in releasing the
cluster claim for thread that actually won it.

This patch makes it so that cluster allocated and save in ctx
is used instead.

Change-Id: Id10811b887f673f9b89e41e0637d4422f1d7270d
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452625
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-04-30 14:43:09 +00:00
Jim Harris
e740ba637c blob: Don't look at cluster map prior to checking frozen
Change-Id: I3858c637d421b58e74fa5573d257e59fed92824a
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452268
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-04-29 16:24:24 +00:00
Maciej Szwed
ee430f97b0 blobstore: Add _spdk_bs_delete_blob_finish function
This patch add new _spdk_bs_delete_blob_finish function
which will be helpful in future changes.

Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I2d492b6102f33ad35b7b6fe408f709f54b7b2341

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452251
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-04-26 23:08:35 +00:00
Maciej Szwed
5fb0e244ed blobstore: Add _spdk_bs_is_blob_deletable function
This patch adds new function which is used to check
if blob can be removed when requested.

Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: Iafa82fba9bf67ffd15cf639f4665087f054b6b7d
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452242
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-04-26 23:08:35 +00:00
Maciej Szwed
8a9c101446 blobstore: Add _spdk_bs_get_snapshot_entry function
This patch creates new function that will be helpful
with further implementation of 'delete snapshot'
feature.

Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I66f138ba217fb4a4186f2703900a2952cdb8e438
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452240
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-04-26 23:08:35 +00:00
paul luse
600341b6c5 bdev/compress: replace RPC parm name with pm_path
We decided to name compress bdevs with COMP_ and then the name of
the underlying bdev so the RPC parm name was stale.  Previously
hardcoded the path to PM for early dev so replaced name with pm_path.

Note that a sample create looks like this now:

rpc.py construct_compress_bdev -b NVMe0n1 -p ~/pm_files -d compress_isal

And that devs need to pay attention for the pathname they provide, its
easy to leave orphaned pm files around and they can get big.

Change-Id: Ifb5245c922461bbecec4bef266bdeb25e8b87f31
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452235
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-04-26 20:07:19 +00:00
paul luse
7bffdb2db9 bdev/compress: fix size of compressed bdev
Needs to be based on the compressed vol size which is smaller
than the backing io device.

Fixes issue 763

Change-Id: I917c98e86a0755e503d5ba3d4b6c02e42f9ed709
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452158
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-04-26 20:07:19 +00:00
Seth Howell
d05c553827 rdma: don't spam people with async event messages.
It used to be that we would get async events very infrequently. However,
with the introduction of SRQ, this number has gone up tremendously.
Change the way we report our these events so that we don't spam/confuse
people running the target.

Change-Id: I33070281fa854cbc17784d61bbbb870196ca8780
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452159
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-04-26 18:10:56 +00:00
Seth Howell
ec47f92b9b rdma: fix potential heap-use-after-free in srq shutdown
If there are outstanding recvs for a qpair when it is destroyed, we need
to clear the qpair from it before reposting it. Otehrwise, we have a
potential heap-use-after-free of double free (depending on whether the
recv completion is in error state or not).

See github issues #730

Change-Id: Ic2009c761cbcc5e89174f62fbd0872d0489c67ca
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452122
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-04-26 11:16:22 +00:00
Jim Harris
9e1116ea83 blob: pass correct op to spdk_bs_user_op_alloc in iov case
At some point this parameter may have been a bool, but it's
not now - so we need to explicitly pass SPDK_BLOB_READV or
SPDK_BLOB_WRITEV.

Otherwise we end up trying to write the iovec array as the
data buffer when executing this operation later - since we
passed "false" which is treated as 0 which is
SPDK_BLOB_WRITE (not SPDK_BLOB_WRITEV).

Fixes issue #603.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I50df736216c4a88b40604722c021e204c7fac623

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452157
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-04-26 06:54:07 +00:00
Seth Howell
3856d82b50 subsystem: check for NULL bufs in reservation ops.
At the RDMA level, we allow processing requests that should contain a
data transfer, but specify a length of zero to be passed up the stack
without a data buffer. See spdk_nvmf_rdma_request_get_xfer. In the case
of the reservation requests, we weren't checking whether req->data was
NULL before trying to copy into it causing us to segfault if we got a
malformed reservation request.

Found when using the fuzzer.

Change-Id: I320174ec72a8d298ab6ca44ef6a99691631f00ca
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/451786
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-04-25 22:52:12 +00:00
Ben Walker
795134891e blob: Don't reallocate cluster array if it didn't change size.
Other threads are referring to this memory.

Change-Id: I9689a1b64a1f032ad479ec85095d2575dcab3a7f
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452130
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-04-25 22:47:46 +00:00
Jim Harris
ec820daba3 rocksdb: use C++ constructor/destructor for thread_ctx
There's some more work needed to clean this up, but this
at least gets RocksDB working for now after the following
patch which enables dynamically specifying the RocksDB
commit ID.

Note that for now, we've modified the RocksDB BGThreadWrapper
to call SpdkInitializeThread() explicitly.  We'll be able
to move that in the future, but for now we check whether the
channel has already been allocated so that this extra
SpdkInitializeThread() effectively becomes a nop.

Note that for the main thread, g_fs hasn't been set yet, so
we can't allocate the thread_ctx.  So we still need an
explicit SpdkInitializeThread() call in the main thread.

This has the nice side effect of removing the need for SpdkEnv
to override Env::StartThread - so remove all of the code associate
with that.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I1e8d12b74e688953e15d5d6df58b93e3f5b74c3d

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452112
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-04-25 20:04:04 +00:00
Chunyang Hui
6b48e743a3 Opal: Add revert tper cmd option
Reset the device to its factory defaults.

Change-Id: I43f7dc8fb7bd5226283a4762beac0e2cf016f698
Signed-off-by: Chunyang Hui <chunyang.hui@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/445253
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-04-25 18:20:42 +00:00
Maciej Szwed
4b1012d5bf blobstore: change _spdk_bs_blob_list_remove() type to void
_spdk_bs_blob_list_remove function returns only 0
so it can be changed to void function.

Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I1827d783a7cd18caf831275554e33f398b0b962f

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/451783
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-04-25 03:34:17 +00:00
Wojciech Malikowski
58c4dac9d9 lib/ftl: Limit write queue depth to one per chunk
There are no guarantees of the order of completion
of NVMe IO submission queue so wait for write completion
on specific chunk before submitting another write to it.

To control chunk occupancy split IO to child requests
and release chunk in IO completion callback.

Change-Id: I44147a21b528a7f33fb92b9e77d7de8f5b18f8ff
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449239
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Young Tack Jin <youngtack.jin@circuitblvd.com>
Reviewed-by: Claire Jihyun In <claire.in@circuitblvd.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-04-24 23:24:21 +00:00
Wojciech Malikowski
049624823b lib/ftl: Rely on internal IO position during ftl_io_done()
Change-Id: I7a7667e1c4e6321150e5bc8eb4287fb61847ccb5
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/450174
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Young Tack Jin <youngtack.jin@circuitblvd.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-04-24 23:24:21 +00:00
Maciej Szwed
e6100af425 blobstore: block simultaneous operations on blobs
Block operation that should not be done simultaneously.

Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I3cab510377a49be4e5847ba37a6218f0025c0db6

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/450014
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-04-24 22:43:47 +00:00
Maciej Szwed
478c2e6f64 blobstore: Move snapshot check on deletion after blob is opened
Currently someone can create clone while blob is being
opened for deletion. This patch moves check for blob if
it is a snpashot after the blob is opened.

Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: Ia4b8b7cb956522a29784aa349d677eb87886e4db

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/451682
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-04-24 22:43:47 +00:00
Chunyang Hui
7250ec64db Opal: add take ownership cmd options
Also make some modification for invoking process in
nvme_manage tool.

Change-Id: Ib54db43d7336d3e839e7d9317c292b9a57b38f80
Signed-off-by: Chunyang Hui <chunyang.hui@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/445059
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-04-24 22:40:04 +00:00
yangtianyu 00383654
1dd07870e3 bdev/null : Clean up module resources that failed to initialize.
Currently, if the Bdev subsystem initialization fails, the initialization failed
module does not call the fini function cleanup resource.

However, null module use the 'spdk_io_device_register' and spdk_zmalloc to request
resources. In failed path, it will cause resource leak when exiting the app.

To avoid this, it will cleanup resources when module initialization fails.

Change-Id: Ib1a89e5238252d69b6e64ea02a0bd826661349a5
Signed-off-by: Tianyu Yang <yangtianyu2@huawei.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/450105
Reviewed-by: GangCao <gang.cao@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-04-24 22:25:47 +00:00
paul luse
933270eafd bdev/crypto: add new uninit call needed for dpdk 19.02
This patch won't fix issues related to the need for the latest
ipsec with 19.02 but it is needed for that patch to pass.

Change-Id: I8e2d984d85b355d88edfb9c90c159c8aa0a5362d
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/451788
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-04-24 22:00:10 +00:00
Konrad Sztyber
ef5ede4b51 lib/ftl: allow parent/child relations between IOs
Extended the IO descriptors with the ability to group them together in a
parent / child(-ren) relationship. It allows to delay parent completion
until all children are done and enables the children to have additional
context in their completion routines (as opposed to having a single IO
describe all requests).

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I6ad111dc231813e397697109a333e2a5d7f5941d
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449068
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Young Tack Jin <youngtack.jin@circuitblvd.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-04-24 21:57:06 +00:00
Changpeng Liu
3f4426878a nvmf: disable the protection if the backend doesn't contain valid type
It's not an error if the NVMe hard drive was formatted to 512 + 8 but
has no protection type, so we will also disable the protection for
NVMoF target.

Change-Id: I07e605cff9545f46c642f7ca783a4727a26abece
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/451926
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-04-24 21:47:18 +00:00
Changpeng Liu
4582e9fb4a nvme: add additional check to avoid being divided by zero error
When a Namespace was removed all the field will be zeroed, which
may lead to being divied by zero error when IO is running, especially
with perf tool.  The perf tool doesn't add hogplug support, so
we add the additional check here to avoid such issue.

Fix issues #728 and #629.

Change-Id: I0e387c8c1bd4f3d40130377e2e0f5143f43be6a3
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/451762
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-04-24 21:46:15 +00:00
Darek Stojaczyk
47cf47e482 vhost/scsi: don't send events when eventq is unset
We do technically support initiators without eventq or
controlq, but the lun hotplug/hotremove path expected the
eventq to be always present.

This was causing vhost to randomly crash in the fuzz tests.
Specifically, the crash happened if lun hotplug was handled
while a VM was in the middle of switching from BIOS to OS.

We fix it by checking if eventq is set before putting
any event there. LUN hotplug and hotremove won't work
without an eventq, but the entire session will be restarted
after new queues are initialized. This will make the VM
retrieve all up-to-date luns after OS initialization is
complete.

Change-Id: I5d28cbedad8fb2a35ede5a491aeb7fdc52faad06
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/451789
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-04-24 16:46:09 +00:00
Darek Stojaczyk
5bf0a3838c iscsi: allocate sessions array using regular calloc
spdk_dma_malloc() is not required here, as the session
array is nether DMA-able nor shared between processes.

Change-Id: Ia9218d00eea8a3207abf07c1d78eaa585ffcad2c
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/450552
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-04-24 16:11:26 +00:00
paul luse
a50100900d bdev/compress: return error to reducelib if comp/decomp fails
Change-Id: Ibdea83543069e80d73358e95fa43f7b27c351d1a
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/451064
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-04-24 15:07:37 +00:00
Tomasz Zawadzki
bbc49ab42f lib/thread: free cpumask on spdk_thread_create() failure
Change-Id: I2e814aa3188639edff47b6bb3f56609417e6c034
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/451791
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-04-24 15:07:02 +00:00