Use the default thread implementation instead.
Change-Id: I844b02ba3c45d323e9f16b328a31b9d38dc8f754
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/422466
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
This is the callback type used for message passing. The
old name is easy to confuse with the callback to pass
a message to a thread or the upcoming callback to
spawn a thread.
Change-Id: I5fd63b57c4be2a4262a197850e6de4901be03ee7
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/435941
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Add init_crc parameter as seed value to spdk_crc16_t10dif API to generate
a CRC value spanning multiple separate buffers.
This will be necessary for upcoming DIF/DIX patches.
Having init_crc parameter is general, and so change the existing API
without adding seed version of the existing API.
Change-Id: I0ac7919b18013967e41829dcedd3e4e73204d5d6
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/437204
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This makes the timeout check for each qpair in the group
efficient. If there are many qpairs in the group, we
can scale.
Change-Id: I75c29a92107dc32377a2ef7edb5ac92868f1c5df
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.gerrithub.io/435277
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Remove the unnessary fields in spdk_nvmf_tcp_transport
Change-Id: I632608ba654b30f3511f5e1d925c6743c9100365
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/437271
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
When last clone of a snapshot is being deleted
we remove that snapshot from snapshots list.
We should not do that as it still works as a
snapshot and it is read-only, but it does not list
as a snpashot from get_bdevs. Instead remove snapshot
entry from the list when blob that represents that
snapshot is being removed.
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I8d76229567fb0d9f15d29bad3fd94b9813249604
Reviewed-on: https://review.gerrithub.io/436971
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This more closely matches the sequence of events
a user will take in creating a compressed volume.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I2311aed49e2a623feb9c5fbdbd25fb3201aeeeb5
Reviewed-on: https://review.gerrithub.io/436065
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Parameterize the existing unit tests to maximize
code reuse.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iaa6b1bd52ad48ad4c19f6e6419e3e669b2d61e56
Reviewed-on: https://review.gerrithub.io/434117
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
For write operations, copy data to req->buf and write
to disk.
If chunk already specified in logical map, read the
chunk first into req->buf, and overwrite with data
specified by the write operation.
If chunk not specified in logical map, fill logical
blocks not specified by the write operation with
zeroes.
For read operations, read chunk into req->buf first,
then copy relevant data into the buffers specified
by the read operations.
These operations are all functional, but have room
for future improvement. For example, this patch
will issue a separate backing read/write operations
for each backing block in the chunk - this could be
optimized to coalesce operations where the backing
blocks are contiguous.
While here, clean up freeing bufspace in one of
the error paths - this needs to be freed using
spdk_dma_free instead.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I6dbf4fc9a8fdf0f5424b1f1f9178c79891c96d0d
Reviewed-on: https://review.gerrithub.io/434116
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Focus of this patch is adding the foundations of manipulating
the chunk_map and backing_block bit arrays and persisting
chunk maps and the logical map to the pm_file.
No data is writting the backing device yet. That will come
in later patches. This also does not rebuild the bit arrays
from the pm_file during spdk_reduce_vol_load - that will also come
in a later patch.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ib5336dbe907c253e545704471de8c4e812bbc157
Reviewed-on: https://review.gerrithub.io/434115
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
ziperrno was left over from earlier versions of this
patch set that used "sbzip" instead of "reduce".
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I4159141b92727bb0a24db5f9aefcccaadc360d9e
Reviewed-on: https://review.gerrithub.io/436399
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Change-Id: I19bfc17636e0bf36d5a26e18ab2650ea6026f3a0
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/437043
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: Jim Harris <james.r.harris@intel.com>
Same as 29be88f (test/blob: always use SPDK_BS_PAGE_SIZE instead of
PAGE_SIZE), ARM system could not find defination of PAGE_SIZE when
building nvme_ns_ocssd_cmd_ut.c., we use OCSSD_SECTOR_SIZE instead of
Page_SIZE here, also use OCSSD_SECTOR_SIZE instead of "0x1000" for const
uint32_t sector_size.
Change-Id: Ib3062232e44b0be26ade7c64340918f2f23ada03
Signed-off-by: Liang Yan <lyan@suse.com>
Reviewed-on: https://review.gerrithub.io/430802
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Sasha Kotchubievsky <sashakot@mellanox.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
It is the initial patch to add the unit test for
NVMe-oF TCP transport.
Change-Id: Ia8c90a205fc78dd817f91675337ca81021211155
Signed-off-by: Yanbo Zhou <yanbo.zhou@intel.com>
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/426175
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
spdk.unittest.mk includes spdk.app.mk, but some unit test
Makefiles include both spdk.unittest.mk and spdk.app.mk,
meaning spdk.app.mk gets included twice. Fix that.
This hasn't been an issue because spdk.app.mk currently
only includes variables - but no rules.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I23d39e8084f79442fb06ae9b5a6a68d6134adff4
Reviewed-on: https://review.gerrithub.io/434281
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
For split SCSI read command, if there is any failure in the sequence of
it, the first error must be propagated to all subtasks of it.
For split SCSI write command, if there is any failure in the sequence
of it, the first error must be propagated to the primary subtask.
Before this patch,
for read task:
- any failure is propagated to already completed subtasks, but
is not propagated to any subtasks not completed yet, and
- if any failure occurs in non-primary subtasks, it is not propagated
to the primary subtask.
for write task:
- if the primary subtask completes after any failure of non-primary
subtasks, the failure will be overwritten by the success of the
primary task.
This patch fixes these issues.
Change-Id: I2d878798cbb40a8c5bd6a6fe5efb32b8de4a8ecd
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/436673
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This direct call of LUN reset has caused unexpected behavior
during heavy stress test.
Apply correct LUN reset handling for this case too.
Even if there are multiple LUNs, LUN resets are serialized in
the SCSI layer by previous patches and parallel calls of LUN resets
will not cause any issue.
Change-Id: I920e2864ef0fb41c6bd2b6153ef3caff6876e893
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/436115
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Discover commands previously blindly used the nvmf_req->data structure.
This only works if the entire command fits in a single contiguous
buffer. commit 1d9be84bfd changed the default buffer size such that
this would become a problem for as few as 8 subsystems.
Fixes github issue 525
This change may also help prevent data corruption as we were copying up
to nvmf_req->length data into the buffer. For requests with multiple
data buffers this can cause us to copy off the end of that buffer.
Change-Id: I788259da988b2458f57ee2795e1c5d3ced8803dd
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/435544
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
This patch is for ABORT TASK SET and LUN RESET.
iSCSI layer doesn't have pending queue and hence if the target task
is read task and is queued in queued_datain_tasks, it must be
aborted before submitting ABORT TASK to SCSI layer.
Aborting the target task may not complete by an iteration because
submitted read tasks are limited. Hence use poller to complete abortion
by repetition.
Change-Id: Idabf4931d751ee698e9809eafd5c151b979f048b
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/436077
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
By up to the previous patch in the patch series, unexpected behavior
due to write tasks in task management commands have been fixed.
But unexpected behavior due to read tasks in task management commands
have been still observed.
Remaining patches in the patch series will fix the unexpected behavior
due to read tasks in task management commands.
This patch is for ABORT TASK.
ABORT TASK is not supported in SCSI layer yet. But the initiator
doesn't care about the failure is due to not-supported or failure.
It must be avoided that the task management command returns SCSI Good
but some tasks are not aborted and return SCSI Good later. On the other
hand, it is acceptable that the task management command returns
failure but some tasks are partially aborted.
Hence this patch adds operation without checking the support status
in SCSI layer.
iSCSI layer doesn't have pending queue and hence if the target task
is read task and is queued in queued_datain_tasks, it must be
aborted before submitting ABORT TASK to SCSI layer.
Aborting the target task may not complete by an iteration because
submitted read tasks are limited. Hence use poller to complete abortion
by repetition.
Change-Id: I030a8b2f19c2f7c7d2f7b0b2c633579534db631b
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/436076
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
By subsequent patches for iSCSI, spdk_scsi_dev_queue_mgmt_task()
will not be called directly from the function that knows TMF code,
and currently setting TMF code to SCSI task is done in
spdk_scsi_dev_queue_mgmt_task().
Hence after subsequent patches for iSCSI, to hand off TMF code to
SCSI task, any dynamic context will be required.
To avoid the dynamic context, extract setting TMF code from
spdk_scsi_dev_queue_mgmt_task() and put appropriate place for
each call of spdk_scsi_dev_queue_mgmt_task().
Additionally, in spdk_abort_transfer_task_in_task_mgmt_resp(),
ref_task_tag is got from PDU but getting it from SCSI task is
much easier. Hence get ref_task_tag from SCSI task in the callback.
Change-Id: I7add9290598d2df7cfcf1506ec75d74c70c0f236
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/436643
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This patch tests the following:
- Reset task will not complete until prior IO tasks complete.
- IO tasks will suspend until prior reset tasks complete.
Change-Id: I8ae27874e7eb34f866bbd4e057ee5d8c27d040e8
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/435433
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
IO submissions to backends have to complete first in LUN reset.
Previous patch makes the waiting time limited.
LUN reset don't use timeout for now because even if LUN reset reports
timeout, error handling of the initiator will be escalated and the
initiator will issue stronger command and will wait eventually
until all outstanding tasks completed.
Besides, not using timeout in LUN reset will make implementation simple.
If timeout becomes necessary, I'll do it in the separate patch later.
Change-Id: Ie9e4502068c19b1727ea65dc773ddf08cedec7c4
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/434766
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Submission queue deleted by https://review.gerrithub.io/393911
is necessary to support proper LUN reset.
Hence revert the removal in this patch.
And using the submission queue, suspend IO task submission during
LUN reset to limit completion time of LUN reset.
Change-Id: I3cdc4f0165fe845637112c2900407d6b4a09df79
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/434765
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Task management functions don't require performance. Serializing
execution of task managment functions will sipmlify and stabilize
the logic and will be helpful for upcoming patches to support
other task management functions.
This patch introduces two queues, pending and submitted queue,
and serializes LUN reset exection and makes LUN hot removal
wait for LUN reset.
Besides, checking if LUN is NULL is moved to the upper layer as
same as IO task submission.
Change-Id: Ia0cf3f437a745ee70fc9b17744cc63c833690dda
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/434764
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
As a part of cleanup they're replaced by a device-agnostic
attach API, which is easier for us to manage.
Change-Id: Ia92bd8f4525712bd27ade16ead67435c5e0fbe7a
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/436479
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
As a part of cleanup they're replaced by a device-agnostic
attach API, which is easier for us to manage.
Change-Id: I7558590e41e5c580a130a6aba7ae4f7dcff58da8
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/436478
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Current SPDK SCSI supports only LUN RESET in task management function.
Upcoming patches will support other functions too but differentiate
the callback function about LUN reset from other task management
functions for now to avoid misunderstanding.
Change-Id: If8f00ce413fbcc54b12dd885cbf01597f83a2af9
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/434763
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This is a preparation to the next patch.
Change-Id: Ia7af66ba129a4666730f94be64d3699cded65e09
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/434762
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
As described in RFC3720, task management request will act on all
the commands from the same session having a CmdSN lower than the
task management CmdSN.
Current implementation clears all R2T tasks without checking CmdSN.
This patch changes the implementation to clear only R2T tasks
whose CmdSN is smaller than the task management request.
Additionally this patch adds to UT code to verify the change works
as expected.
Change-Id: I0b368cb13741bc05259924d27793438e9250b951
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/434761
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
For tests that aren't specifically testing the thread API,
leverage the ut_multithread framework to simplify them.
Change-Id: Ib772ac4ebd3179b71a51697edc50ad9b7b70536b
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/435938
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
libpmem only allows passing a size when CREATE flag
is set.
This requires some updates in the unit test stubs
for pmem_map_file as well. While here, do some
additional cleanup and add a g_volatile_pm_buf_len to
track the size of the allocated volatile pm buffer.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ib9fe58fd9946161dd20bb8391be2e9680705ab22
Reviewed-on: https://review.gerrithub.io/435945
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Includes tests for the parse_sgl function.
Change-Id: I83a854598c7320b31b75a4fa5ebbfe66cb708b6d
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/429070
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
This will be the logical block size presented by the
compressed volume to differ from the backing device's
block size.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ie4ef06e131d8e101a0c9ced228c56a02fcbfb7af
Reviewed-on: https://review.gerrithub.io/434113
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I90e7d698cae7577736319e38f089e3b759c9beef
Reviewed-on: https://review.gerrithub.io/435343
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
"trace" is for event tracing. SPDK used to use this
term for logging - we've moved some APIs to use "log"
but more needs to be changed. So start that now.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ib732c57d01602e56f37e9deed7135840a7c005be
Reviewed-on: https://review.gerrithub.io/435342
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
NVMf target can't support extended LBA format for now, so print a
error log for those NVMe backend devices with extended LBA format.
Fix the issue #497.
Change-Id: Idda76ba934dd0eb45f92ae22b0b71398b3ae69dd
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/432799
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Considering it's the part base object that's now accessible
in its remove callback, we can simplify the part API by making
it accept the part base object directly.
Change-Id: I87c3278929a063c115828d02e0def7fa536e6682
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/434835
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Unit tests shouldn't link in DPDK env libraries - they
should all use the test_env.c
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ib08ae33224ed9a12fa1ff255c4769a005fc10369
Reviewed-on: https://review.gerrithub.io/434259
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
For some old Linux Guest kernels, the new NVMe 1.3 feature: shadow
doorbell buffer is not enabled, while here, make a dummy BAR region
inside slave target, when Guest submits a new request, the doorbell
value will be write to the shared memory between Guest and vhost
target, so that the existing vhost target can support both new
Linux Guest kernel(newer than 4.12) and old Guest kernel.
Also, the shared BAR space can be used in future which we can move
ADMIN queue processing into SPDK vhost target, with this feature,
the QEMU driver will become very small and easy for upstreaming.
Change-Id: I9463e9f13421368f43bfe4076facddd119f4552e
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/419157
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Initial implmentation had a 1:1 session to crypto op ratio. After
working with a similar concept in CompressDev checked to see if
this was required and apparently it is not.
Saves a decent number of API calls per crypto op and in the poller.
Also saves on mempool usage. Performance improvement measurement
is WIP.
Change-Id: I73f2355e720a16fd46bc4a02657419f779f07cbb
Signed-off-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/433726
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Fiona Trahe <fiona.trahe@intel.com>
This will make future changes to the threading model
easier.
Change-Id: I4abfe0850648a0f7449804066d4a02823393e4a6
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/433680
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This will make the conversion to the new threading library
easier later on.
Change-Id: Ie7358e89128f28080a986e12cd014de7df4752e4
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/433679
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
In addition UT were changed to set_thread() before registering
io device.
Change-Id: I959dbc800db9c1f50564274a73d71e05e843d8c9
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/433611
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
A pm file consists of:
* volume metadata
* logical map
* chunk maps
Define all three of these in struct spdk_reduce_vol.
Also define -1ULL to denote an empty entry in the logical
map or a chunk map - and initialize the logical map
and chunk maps entirely with this value when initializing
a new compressed volume.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I9aae23c73d2fbbdc72050ab103fe9e686907eb40
Reviewed-on: https://review.gerrithub.io/433490
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Currently the API requires the caller to open the
pmem file and pass the mapped buffer pointer, length
and pmem flag to libreduce using the spdk_reduce_pm_file
structure.
Let's have the library just do the pmem_map_file() instead.
Users then just pass the path to create the pmem file. The
library will still use the spdk_reduce_pm_file structure
internally - so move its definition out of the public header
and into reduce.c.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I81fcbfdfbb3211dca016d6aa422cf2e1ab16d84d
Reviewed-on: https://review.gerrithub.io/432593
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
It is the first patch to follow the NVMe over fabrics
spec and implmenent the NVMe/TCP transport. It can be
divided into work in the host and target sides:
Host side: Add the TCP/IP transport in nvme lib (lib/nvme).
Target side: Add the TCP/IP transport in nvmf lib (lib/nvmf).
Change-Id: Idc4f93750df676354f6c2ea8ecdb234e3638fd44
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/425191
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Currently there are no timeout mechanism for Admin commands
when initialization, the NVMe driver may enter infinite loop.
While here, add a new parameter to the controller initialization
options, NVMe controller will report an error when timeout
happens during initialization.
Change-Id: Id0c6b6fa15abe5227b486bee95c8e02914b0d358
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/424622
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Change-Id: I36eed5fdd593eb3df15859a3e29c600be0913899
Reviewed-on: https://review.gerrithub.io/432559
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
No longer override the send_msg implementation.
This updates ut_multithread.c, which has minor ripple
effects into bdev_ut.c. But that unit test is otherwise
unchanged.
Change-Id: I2fd30a1010bdff0a810d376d985ab1b8a2b22fb9
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/424262
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Ensure in init_failure() that even after we generate
a UUID, the init still fails since the backing_dev
function pointers are still NULL.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I3287da8626152649416b4aaf4285eb9f84f79200
Reviewed-on: https://review.gerrithub.io/433084
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Write pm filepath first to offset 4K. Then write
the super block to offset 0. This ensures the backing
device isn't really valid until both are written
(avoiding the case where the super block got written
but not the path.)
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I55508aa67b4179e658827c982cd955d009a3f321
Reviewed-on: https://review.gerrithub.io/432505
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: <spdk-ci.pdl@broadcom.com>
While here, add basic implementations for these function ptrs
in the unit tests.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I54e18a3b331777602fed29382b95b449005efcce
Reviewed-on: https://review.gerrithub.io/432503
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
There are now other unit tests which exercise the success path.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I601395aff7d3c5e88bdcf2986a25e71b0e2f5d39
Reviewed-on: https://review.gerrithub.io/432500
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Change-Id: Idcfb328a26fc85f3c144da9c27086a71c00d0d46
Reviewed-on: https://review.gerrithub.io/427830
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
This option is deprecated. Also, rename the rpc and configuration
options for setting the opts to reflect that they now only set the max
number of subsystems
Change-Id: Iaabcbf33dd0a0dc489d81233fda74e9e7f3e0d2e
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/430161
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Add the following to struct spdk_reduce_pm_file:
* path of the pm file
* pointer to the mmapped pm region
* whether the mmaped address is pmem or not
Now use pmem_persist or pmem_msync to persist volume
parameters. Note that we do *not* persist the pm filename
to the pm file - the pm filename will be written to the
backing disk in a later patch.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I1d873ae086dc20600798a49b98e1bf490ac750f3
Reviewed-on: https://review.gerrithub.io/430648
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
--with-reduce required to build reduce. This depends on
libpmem being installed.
We still need to work out details in pkgdep.sh and
vm_setup.sh. Some distributions like Ubuntu still
require configuring extra package repositories to
get libpmem packages.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I4e056ce1da9a1fecb4458f8f5e7ff5d61c422533
Reviewed-on: https://review.gerrithub.io/430646
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
A compressed volume will require both a persistent
memory region/file for per-chunk metadata and a
backing device to store the compressed blocks.
Add functions here to calculate the sizes of these
based on the desired size of the compressed volume,
its chunk size and the size of each backing block.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I9203479e2a268c3ab0e2b0e06e348285e9d1cd13
Reviewed-on: https://review.gerrithub.io/430387
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
reduce will be a block compression algorithm designed
specifically for SPDK. It is called "reduce" because
it reduces the data size on disk.
This patch just adds the shell of a library, include
files and unit tests. This will be fleshed out in
the rest of the patches in this series.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I88e238af64142a7c0e50ab7b447280026b55581f
Reviewed-on: https://review.gerrithub.io/430386
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Since a PDU can be sentout in many times, so this
function is called multiple times, so move the
caculation function in spdk_iscsi_conn_write_pdu
Change-Id: Ib4da4a74c17709d98e4b01c2e76428021afea947
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.gerrithub.io/429931
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
In cases we probe without a specific trid, the underlying
rte_bus_probe() in spdk_pci_enumerate() might fail to
initialize some devices, but still return with code 0,
That's technically correct, as we asked just to probe
devices on the bus and that's what it did. Some devices might
have been initialized, others not. In secondary process we
blindly assumed all devices were probed successfully, which
might have eventually led to assert failures, as current
process was not on the ctrlr->active_procs list.
To fix it, just add an additional check before attaching
the controller in secondary process.
Change-Id: If015b1e562052a9189ed1a48091b209bd2dd5f2a
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/431727
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
We only detached the PCI device on the controller destruction,
which happens just once - in the primary process, but secondary
process needs the PCI detach as well.
Requesting to hotremove the NVMe PCIe controller in secondary
process is broken, because DPDK will still keep the device
reference and won't allow SPDK to hotplug it again.
Fix this by detaching the local PCI device whenever removing
a secondary process from spdk_nvme_ctrlr. This does require
an additional transport check in the generic NVMe layer, but
I found it an overkill to create a multi-process transport
abstraction just for this case.
Change-Id: I812dc1c878ade5b149556806228a2afcb49f0b17
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/431487
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Previously there was no consideration for IO that were outstanding
to the crypto device when handling a reset. This patch makes sure
that those IO are completed with FAIL status prior to completing
the reset that we pass down the stack.
It does so by sending down the reset first and in the completion
using spdk_for_each_channel and the poller to quiesce each channel
allowing the crypto side to complete all IOs before we finally
complete the reset IO after the last channel is quiesced.
Resets are tracked on a per bdev basis.
Addresses github issue #449.
Change-Id: Iadb07bada1fcaad33d9f224a60d983a7eb835236
Signed-off-by: paul luse <paul.e.luse@intel.com>
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/428552
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
By running random, 64KB, 70% write, and 30% read workload and periodical
TCP disconnection from the initiator, segmentation fault have occurred.
By investigation, when the last write task completed and dequeued
the primary write task in spdk_iscsi_task_cpl(), the primary write task
was already dequeued in spdk_clear_all_transfer_task().
This patch dequeues all primary write tasks after all outstanding
tasks complete.
After starting to exit the connection, the connection will not receive
any command. Hence waiting for completion of all pending tasks first will
avoid double dequeue safely.
Change-Id: I8e2b6b756be3a6e096675acc81cdb93fb6a7e350
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/430397
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
We need additional mocks for structures that were added
in the most recent DPDK release to make our unit tests
build properly.
Change-Id: If6e471528d4cb1dce24fb5d910e9e545fcdaa0f6
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/429987
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
If JSON structure is not known it is hard to use it with current API. To
address this gap lets add API that will give more flexibility the user.
This patch also adds proper unit tests.
Change-Id: I82eb8c8d4a562ee4c9eb5b72c69fe36004dc576e
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/424009
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
In the case that the subsystem in the related poll group has
NULL IO channel assigned due to some problem like out of resource,
for example, the NVMe SSD hardware itself has limited number
of IO qpairs. The subsystems in the particular poll group
could have zero valid channels. In this case, the creation of
assoicated poll group will fail and when adding the new qpair
to the specified poll group, needs to have a check and pick the
available poll group.
Change-Id: Iedee2a6375e48eb7bf899cfb0542c565c7ebd231
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.gerrithub.io/423646
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I189ad8889c74937bf43bcf2c3029416ddb94976d
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/425705
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: GangCao <gang.cao@intel.com>
Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Change-Id: I875cc9d6a6bd1e9e9ac25ca9103a2070226ac236
Reviewed-on: https://review.gerrithub.io/428877
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This function doesn't return error code
Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Change-Id: I67a8fa7393990470e509baa8934e78bc6f6a6c9e
Reviewed-on: https://review.gerrithub.io/429441
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Change-Id: I90f68e9dbe00854f569f6574b9c89508f89caa43
Reviewed-on: https://review.gerrithub.io/428876
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Purpose: We need to get the port info in other applications
(e.g., NVMe-oF TCP/IP transport)
Change-Id: I3a4636e764e44425436bb064cb0062c6f3e44035
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/428313
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Large read I/O will be typical in some use cases such as
web stream services. On the other hand, large write I/O
may not be typical but will be sufficiently probable.
Currently when large I/O is submitted to the RAID bdev,
the I/O will be divided by the strip size of it and then
divided I/Os are submitted sequentially.
This patch tries to improve the performance of the RAID bdev
in large I/Os. Besides, when the RAID bdev supports higher
levels of RAID (such as RAID5), it should issue multiple
I/Os to multiple base bdevs by batch fasion in the parity
update. Having experience in batched I/O will be helpful
in the future case too.
In this patch, submit split I/Os by batch until all child IOVs
are consumed or all data are submitted. If all child IOVs are
consumed before all data are submitted, wait until all batched
split I/Os complete and then submit again.
In this patch, test code is added too.
Change-Id: If6cd81cc0c306e3875a93c39dbe4288723b78937
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/424770
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Coverage report indicated these were not currently
covered.
Change-Id: Ic67f1d6321c39c59ede8a379051e238d3b0fecb6
Signed-off-by: Mike Altman <maltman@scalecomputing.com>
Reviewed-on: https://review.gerrithub.io/428447
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Normally, there will be only one separator in transport id,
for example, either ':' or '='. But the users may input
this: trtype=PCIe traddr=0000:81:00.0.
Thus, there will be two diffrent separator '=' and ':',
and our function doest not handle this case correctly.
And this patch can fix this issue, and also update the
test case.
Change-Id: Ic3f10dc1e37c66647fede37c5cf9523fc2652677
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.gerrithub.io/428307
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
spdk_bit_array_find_first_set() returns UINT32_MAX if no
set bits are found. But spdk_bit_array_find_first_clear()
would return the size of the bit array instead in this case.
(Note: the comments say size of the bit array + 1 which was
incorrect)
So this patch makes spdk_bit_array_find_first_clear()
consistent with spdk_bit_array_find_first_set() and returns
UINT32_MAX if no cleared bit is found.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I44c1e674149f8c2e87122800d5db45a2851f0bef
Reviewed-on: https://review.gerrithub.io/428225
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Splitting IO in the bdev layer is a little complex and increasing
the coverage in unit tests will be good.
Change-Id: I9d42cedc795b2804a472bdc310e936135c424fd3
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/427948
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Expected values are managed by linked list now. Hence we can set
up all expected values before starting tested split I/Os.
This change will make the logic clearer.
Change-Id: I9226619468220132aebd55a5da36490a145c128a
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/427947
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Change-Id: I994b5d46faffd34430cb39e66225929c4cba90ba
Reviewed-on: https://review.gerrithub.io/414935
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Connections to a subsystem may only originate on listeners
that have been explicitly allowed for the given subsystem.
Change-Id: Ib575a5803cc7af75c15d1a2c76b9866594e7a1db
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/425024
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Each file that need to check SPDK_CONFIG_* options need to include
spdk/config.h explicitly.
Change-Id: If9f2a91ac4c2b1a300dcf88ec3e2a12714ad344a
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/427221
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
We need to include it in devel package.
Change-Id: I823200632e8bcb9fdb86c8cb5fbf3a651a710b78
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/426839
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
This patch is to introduce the specific QoS related structure
and the enumeration for types of QoS rate limits. Later new
types of QoS rate limits can be supported easily.
Change-Id: Idb8d2e7627fd145bf2b0ddb296c968b6b068f48c
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.gerrithub.io/424459
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Current test code works only for sequential split I/Os. The next
patch will support submitting batchedsplit I/Os.
Prepare test code as a separate patch to make review easier.
Change-Id: I05b732c9a656ecab74a2594bf50ddf42eb41584e
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/425877
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
When a bdev IO is split, if iovec size in a strip is more than 32,
the IO will fail.
Remove the limitation by spliting the split IO further.
Change-Id: I962ad86dfe63ea1fcd86ffa52ead7452fb80e53d
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/425876
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
For single base bdev case, IO is changed not to be split. However
UT code was not changed together.
Change-Id: I81e66b256892b7bee7a42c72bb0bce6d3320b617
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/425534
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Initial support for softare AESNI_MB DPDK driver only.
Have tested (both aesni and QAT seprately and concurrrently) on underlying NVMe devices
with bdevio and a bdevperf script that runs IOs from 512B to 128K each with Q depths from
1 to 512 in powers of 2 for 30 seconds each run.
QAT can be included in the code (but not makefile) and marked as experimental
until we are ready to test in CI. It works well on 2 systems but is a big PITA to get
the hardware setup and configured for use with DPDK (IMHO).
Change-Id: If518c3df8e74e00efa18afdf194824c5e69778fc
Signed-off-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/403107
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
- Add independent functions to create transport with specific opts
and add to target while maintaining backward compatibility with
current apps and rpc configuration that still use the add listener
method to create a transport.
- Add new rpc function to create transport and add to target.
+ Update json reporting to include new rpc function.
+ Update python scripts to support new rpc function.
+ New nvmf test script (cr_trprt.sh) to test new rpc function.
Change-Id: I12d0a42e34c9edff757755f18a78b722d5e1523e
Signed-off-by: John Barnard <john.barnard@broadcom.com>
Reviewed-on: https://review.gerrithub.io/423590
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This patch just adds the call, but doesn't change behaviour of blobstore.
io unit size remains same as page size.
Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Change-Id: Idcd1b7d5126fc7cacf12c996188bd41e2c9a744d
Reviewed-on: https://review.gerrithub.io/425355
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
The bdevio tests will now work on RAID volumes, so enable
RAID testing there too.
It is probably safe now to just build RAID by default, and
not require --with-raid - but let's do some more testing
with vhost and NVMe-oF against it first before we do so.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I15697f035cb688574a7ecb8be24d0c84fc622d83
Reviewed-on: https://review.gerrithub.io/423408
Reviewed-by: Kunal Sablok <kunal.sablok@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Change the behavior of construct_raid_bdev RPC to be consistent with
other bdev modules.
Create a config of raid bdev first. Then create a raid bdev based
on the config.
If both succeed, the config and the raid bdev are not removed even if
any failure occur later in construct_raid_bdev RPC. Otherwise, both
are removed and return failure.
During iteration of adding base bdevs in construct_raid_bdev RPC,
- skip any nonexistent bdev and move to the next base bdev.
- if adding any base bdev fails, move to the next bdev.
- if adding base bdevs don't fail but any base bdev doesn't exist,
the raid bdev is in the configuring state and return success.
- if adding base bdev fails, the raid bdev is in the configuring state
and return failure.
- if all adding base bdevs succeed, configure the raid bdev. If configuring
the raid bdev fails, move the raid bdev to the offline state and return
failure. If configuring the raid bdev succeed, return success.
check_and_remove_raid_bdev() becomes unused in raid_bdev_rpc.c but
is still necessary in UT. Hence move this function to UT.
In UT, finding a raid bdev config by name becomes necessary. Hence
factor out the iteration to a function and use the function in UT.
Change-Id: Ifa36967bdc987d97030e3a4e36684cb37b329d4e
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/423622
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Kunal Sablok <kunal.sablok@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
When construct_raid_bdev RPC is callled, even if all adding base bdevs
to it fail, a raid bdev with no base bdev should exist and be in
the configuring state.
Extract raid_bdev_create() from raid_bdev_add_base_bdev() and put it
before starting iteration of raid_bdev_add_base_bdev().
This change cam be applied to .INI config file too.
Change-Id: Ia71aca0bbdb33049fe5bd9b8fd8163c99c032401
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/424148
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Kunal Sablok <kunal.sablok@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Cleverly, UT for raid bdev sets number of base bdevs by using
the random number generator.
When the random number generator outputs 1, test code especially
for single base bdev case failed.
When removing raid_bdev_send_passthru from library, the corresponding
test code was not removed.
This patch removes the test code.
Change-Id: I23be88358fe364bc757979a26398b80339e19910
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/424159
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Kunal Sablok <kunal.sablok@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Currently in the function nvme_ctrlr_start() the initialization
process is executed as a whole, in the case there are many controllers
in one system, which means other controllers must call the function
one by one. While here, we add several states here, which can
help refactoring the initialization process.
Change-Id: I209cf964bbf6e151823a7ecdc6a3f6e6e69df297
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/424157
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
There is a race condition with the following sequence:
spdk_bdev_open()
spdk_bdev_unregister() <-- starts deferred message
spdk_bdev_close()
deferred message runs, crashes
Change-Id: I81fbced0849949cfb2dae5a7cc6f60c9685a8885
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/424739
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Latest scan-build finds a legitimate issue here - although
it's not clear why it was only found when testing against
a seemingly unrelated patch...
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ic981f6f403c638b0b2d454c0be9f7fbe145171d9
Reviewed-on: https://review.gerrithub.io/424887
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
spdk_bdev_close() must be called on the same thread as
spdk_bdev_open(). Further, the remove callback on the
descriptor will also be run on the same thread as
spdk_bdev_open().
Change-Id: I949d6dd67de1e63d39f06944d473e4aa7134111b
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/424738
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: GangCao <gang.cao@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Change-Id: Ib5e977b0bad15af7a2a71000c1fc4861b5b5b0af
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/424465
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This was added a long time back for tracking an rte_mbuf
whose buffer was a different rte_mbuf - all related to
a userspace TCP stack that is no longer in development.
The concept isn't useful now, so remove it to reduce
the complexity of the tracing code.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I310e492eba7f55df242bb29d82fb19f6daee1f51
Reviewed-on: https://review.gerrithub.io/424565
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
This issue is related to Clang assuming that global variables can be
modified from their original value.
Change-Id: I576a4af7e379a951a9f971850391c7e17145df87
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/424717
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This is a string name used for debugging only.
Change-Id: I9827f0e6c83be7bc13951c7b5f0951ce6c2a1ece
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/424127
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
This is in response to a scan-build error introduced with clang 6.0
Change-Id: Iee5a2538ec9a6575e5b3087bf43b8ded5d099fe7
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/424576
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
The type of malloc used in some of the NVMe functions changed and has
re-introduced some scan-build failures.
Change-Id: I2055260c256429a1a2c6404a6d87605807d4170e
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/424573
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This is a variant of spdk_trace_record which takes a tsc
parameter. This allows callers who already have the
current tsc to pass it in as a parameter, saving an extra
rdtsc in the trace library.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I074f4f9fdbe649ad6717f932982a574bc61f2997
Reviewed-on: https://review.gerrithub.io/424277
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This reduces overhead of spdk_trace_record calls when
tracing is not enabled.
While here, remove a couple of unit test stubs for
spdk_trace_record that weren't needed anymore.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I2cbb91e7d3311d95444d663f466d846676d2dcce
Reviewed-on: https://review.gerrithub.io/424276
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This makes it easier to debug failures.
Change-Id: Ic1099ec08308d5e50dd9fe03798dc0873579f2cc
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/423961
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
I had to make small modifications to one test in the dev_ut.c file to
prevent a memory leak error. However, changing the index in this test
from -1 to 0 has no effect on the coverage.
Change-Id: Ia8a475aaa41118f072a5d9006bab9d9c84b4a10a
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/424126
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
thread_ut.c: ensure that the channel context is non-null
util: get rid of a couple of uninitialized value errors.
Change-Id: Idbaad4efea14acd7ae8f98b60aa514e45213a134
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/424124
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Set the bdev->optimal_io_boundary to the strip size, and
set split_on_optimal_io_boundary = true. This will ensure
that all I/O submitted to the raid module do not cross
a strip boundary, meaning it does not need to be split
across multiple member disks.
This is a step towards removing the iovcnt == 1
limitation. Further improvements and simplifications
will be made in future patches before removing this
restriction.
Unit tests need to be adjusted here to not span
boundaries either.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I08943805def673288f552a1b7662a4fbe16f25eb
Reviewed-on: https://review.gerrithub.io/423323
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Splitting a 1TB unmap into individual 64KB unmap commands
(for a RAID volume with 64KB strip size) would be awful -
the RAID module can be much smarter about this.
So back out the changes for splitting I/O without payload.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I24fe6d911f4e3c9db4b2cb5d66c7236a5596e0d9
Reviewed-on: https://review.gerrithub.io/424103
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Valgrind found this on my system. The unit test itself
was passing correctly (we wrote uninitialized data and then
successfully read back the same uninitialized data) - but
Valgrind doesn't like code that makes any decisions based
on reading uninitialized memory.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I762f5dd0cb368328bd5eba9cecf7db404b457fad
Reviewed-on: https://review.gerrithub.io/424109
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
One was an API issue, another was a dereference of a null pointer that
resulted from a possible call to a stubbed function in the unittest
common code, and the last was an erroneous use-after-free due to
scan-build apparently not understanding TAILQ_REMOVE.
Change-Id: I9111817e9e990ff6e388351c08f34c2e945df62d
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/423959
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Fixes a use after free error in bdev/bdev_ut
Fixes an uninitialized argument value error in mt/bdev/bdev_ut
Adds extra checks in the spdk_bdev_part_construct code
Fixes a dereference of null pointer error in bdev/vbdev_lvol_ut
Change-Id: I1eaa6ebfec0a36a0d910504c70b16ccc651de22a
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/423923
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This removes assumptions about hte lifecycle of these variables relative
to the parent bdev and allows us to avoid ambiguity about who
is responsible for freeing these buffers.
Change-Id: Ia996653562d532fa1501faf21d3fdff85033ab33
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/424105
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
When a raid bdev is constructed by JSON-RPC construct_raid_bdev,
information about config and slot can be passed to
raid_bdev_add_base_device() and raid_bdev_can_claim() doesn't have
to be called.
Hence extract raid_bdev_can_claim_bdev() from raid_bdev_add_base_device()
and put it to raid_bdev_examine().
Change-Id: I92e02bf3661cb97b691246f32198ba946810d96c
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/423618
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Kunal Sablok <kunal.sablok@intel.com>
raid_base_bdev_config is suffciently descriptive as name and so
"name" can be used as the name of the base bdev.
Other structs have used "name" as the name of bdev.
Change-Id: I090ef541a84fd14d9ec3bfebbdc96ae649709551
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/423614
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Kunal Sablok <kunal.sablok@intel.com>
This patch adds UT for IOV split operation.
Next patch will refactor split of vectored read/write a little.
Change-Id: I77fbaf042fd894b23ad5ae846f54c28498e8e62b
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/422768
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
For secure iSCSI targets, dynamic reconfiguration of CHAP secrets is
must to have.
Currently CHAP secrets are loaded for every CHAP authentication
operation. The current implementation will not work correctly
when CHAP secrets in the file are changed dynamically.
If SPDK loads CHAP secrets from the file only at boot and they can be
configured by RPCs, user can change CHAP secrets safely during run time.
Even if there are any users who expect dynamic reconfiguration of
CHAP secrets based on the current implementation, if we provide
this better alternative based on RPCs, they will be able to continue to
satisfy their requirement.
This patch change the current implementation so that SPDK loads
CHAP secrets from the file once at boot and uses them in memory
hereafter.
Besides, use fixed size buffers to hold CHAP secrets.
Previously dynamically allocated buffers by strdup() had been used,
but it required many nomem checks and should be avoided.
Other iSCSI targets/initiators have used fixed size buffers and SPDK
follows others.
Set the size of buffers for both user names and secrets to 256 (the
last byte is for NULL termination). 256 is sufficiently large
compared with others.
CHANGELOG will be updated in the separate patch because new RPCs will
be added instead.
Change-Id: I499e792817c2ed01c3d970bbd3d34a6b1fccf65b
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/421463
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
- Move most of the target opts from nvmf_tgt to nvmf_transport.
- Update transport create functions to pass in transport opts.
- When transport opts are NULL in transport create function, use
target opts. (for backward compatiblity)
- Part 1 of 2 patches. Part 2 (to follow after part 1 accepted)
will allow independent creation of transport with specific opts
while maintaining backward compatibility with current apps and
rpc configuration that still use the add listener method to
create a transport.
Change-Id: I0e27447c4a98e0b6a6c590541404b4e4be879b47
Signed-off-by: John Barnard <john.barnard@broadcom.com>
Reviewed-on: https://review.gerrithub.io/423329
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
These tests don't currently require the env mockup,
but they will in a future patch.
Change-Id: Ibf311f198a291f825a83e7b89708cd2442f35f7a
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/423183
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
According to the purpose of the lists, state_link and global_link
will be enough to understand the purpose well.
Besides, in raid_bdev_remove_base_bdev(), if any entry is not removed
during iteration, TAILQ_FOREACH_SAFE() is not necessary. Hence
change TAILQ_FOREACH_SAFE to TAILQ_FOREACH for this case.
Change-Id: I3022c58faf96721df9241e07dbb5a06d7de89e70
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/423056
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Kunal Sablok <kunal.sablok@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
struct raid_bdev has a pointer raid_bdev_config to its configuration,
but raid_bdev is duplicated between raid_bdev and raid_bdev_config
and just config may be enough as a name.
Change-Id: I77f6a69b913cd540c22f05803c486a4caf103f24
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/422923
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Kunal Sablok <kunal.sablok@intel.com>
base_bdevs_io_channel is good but base_channel may be enough and
fit other bdev modules.
Change-Id: I67a1d224f1ef4ca1fc048b4325333f2552a37150
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/422921
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Kunal Sablok <kunal.sablok@intel.com>
raid_base_bdev_info is sufficiently descriptive as name and so
names of member variables in raid_base_bdev_info can be simplified.
Change-Id: I759a758ec0f3ac21de767ad379a902f1b04cc66d
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/422919
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Kunal Sablok <kunal.sablok@intel.com>
A number of modules (RAID, logical volumes) have logical
"stripes" that require splitting an I/O into several
child I/O. For example, on a RAID-0 with 128KB strip size,
an I/O that spans a 128KB boundary will require sending
one I/O for the portion that comes before the boundary to
one member disk, and another I/O for the portion that comes
after the boundary to another member disk. Logical volumes
are similar - data is allocated in clusters, so an I/O that
spans a cluster boundary may need to be split since the
clusters may not be contiguous on disk.
Putting the splitting logic in the common bdev layer ensures
bdev module authors don't have to always do this themselves.
This is especially helpful for cases like splitting an I/O
described by many iovs - we can simplify this a lot by
handling it in the common bdev layer.
Note that currently we will only submit one child I/O
at a time. This could be improved later to submit multiple
child I/O in parallel, but the complexity in the iov splitting
code also increases a lot.
Note: Some Intel NVMe SSDs have a similar characteristic.
We will not use this bdev stripe feature for NVMe though -
we want to primarily use the splitting functionality inside
of the NVMe driver itself to ensure it remains fully
functional. Many SPDK users use the NVMe driver without
the bdev layer.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ife804ecc56f6b2b55345a0d0ae9fda9e68632b3b
Reviewed-on: https://review.gerrithub.io/423024
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This patch series will refactor/improve CHAP authentication and consolidate
it into iscsi_subsystem.c.
To get better reviews, this patch just moves spdk_iscsi_chap_get_authinfo()
from iscsi.c to iscsi_subsystem.c.
Change-Id: I953f5c851bfe67dc02f6f82966132b4216e79228
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/422764
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
A new unit test in an upcoming patch will initialize the bdev
layer separately, so modify the existing tests now so that
they properly cleanup the bdev layer after the unit test
that initialized it is complete.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ie00837a90391ca4d0dd578e978fb26483fe5c0cf
Reviewed-on: https://review.gerrithub.io/423023
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
New function was added in bdev layer to allow
handling spdk_bdev_io buffer exhaustion.
This patch adds that functionality to nvmf bdev.
Change-Id: I5b915abcccec930c3eb684d95bc64482e7887af5
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/418937
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
So we don't need to allocate memory (maybe failed) just for free other
memory.
Change-Id: I2c83f6acc2aa6ed79455bff90f952a2e70b44d59
Signed-off-by: wuzhouhui <wuzhouhui@kingsoft.com>
Reviewed-on: https://review.gerrithub.io/422203
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
The intents of these arrays was to keep track in the
bdev layer of all base<->virtual bdev relationships -
i.e. which member disk bdevs make up a RAID bdev,
which logical volume bdevs are associated with a
bdev that contains an lvolstore, etc.
Currently none of this is used however. And trying
to keep track in the bdev layer instead of asking
the bdev modules for the relationships has a number
of complications. Early one, we tried to do this
with TAILQs - but that doesn't work since this can't
be done with a single TAILQ_ENTRY in the bdev
structures. So we moved to arrays - that works a bit
better, but then the pointer arrays have to be
realloc'd which isn't ideal.
The biggest problem though with these arrays is that
they held bdev pointers - not bdev descriptor pointers.
It's not really valid to access bdevs without a
descriptor - the descriptors are what make sure active
references are accounted for when a bdev is hotplugged.
Of course the bdev layer knows when a bdev is getting
removed and could go and do the updates to these
arrays separately - but that just seems very convoluted.
So for now just remove these arrays completely. If
there is a future need for the bdev layer to
understand relationships between bdevs, we can add
module APIs so that the generic layer can ask
the modules about the relationships.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I99ef1068240bff1262f64f234260cf2fb44df51d
Reviewed-on: https://review.gerrithub.io/420932
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: GangCao <gang.cao@intel.com>
When an SPDK application shuts down, the bdev layer will
automatically unregister all of the bdevs to ensure they
are properly quiesced and cleaned up.
Some modules may want to perform different operations when
a bdev is destructed during normal runtime vs. shutdown.
For example, for lvol, when the last lvol is cleaned up,
it should unload the lvolstore, release and close the bdev
that contains the lvolstore. You never want to do this
during normal runtime though - it is perfectly valid to
have an lvolstore that contains no lvols. RAID and future
bdev modules such as multipath have similar use cases.
So add a new bdev module callback named "fini_start".
If a module specifies a function pointer for this callback,
the bdev layer will call it before it starts the bdev
unregistrations.
This enables some future patches to the bdev layer such
that it will always unregister block devices that are not
claimed (i.e. logical volumes) before block devices that
are claimed (i.e. the bdev containing an lvolstore).
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I6e87f5c2b27f16731ea5def858f26e882a29495a
Reviewed-on: https://review.gerrithub.io/421175
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Decrementing struct spdk_jsonrpc_server_conn::outstanding_requests
should be atomic since this variable are accesed from multiple threads.
Istead of that just route the request back to the server thread with
nothing to send.
As we are here change spdk_jsonrpc_server_send_response() to take only
struct spdk_jsonrpc_request parameter.
Change-Id: I9b856e7d530355cea43a29f58f4f9405e7e35fc2
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/422124
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
In RDMA, qpairs can't be removed from poll groups because
the poll group defines the completion queue. So don't
allow this operation anymore, even if it were theoretically
possible on other transports.
Change-Id: I69a3d1b336decd2d25e43ddea94f8b2095ef662f
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/421174
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Memory for socket structure is allocated in the selected net
framework but freed in the libsocket on the higher level.
This patch moves memory free to the net framework implementation.
Change-Id: Ia3d4e1553a858a38beb390986e9af105778c12c7
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-on: https://review.gerrithub.io/421587
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Unify the regular mocks and the pointer mocks. Simplify
several of the #defines.
Change-Id: Ica8c69dbb70a685a55b5961b73fd7872f451c305
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/418884
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Claim the block device when adding it to a new Namespace,
and prevent the block device to be added twice for other
modules and Namespaces. Also remove the test that using
same block device over different Namespaces.
Fix issue #371.
Change-Id: Ib7ce18e9fde4a15c0f19ce9e28e69145e54570e0
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/420472
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
During IO submission, raid_bdev can be get by bdev_io->bdev->ctxt.
Hence holding raid_bdev in raid_bdev_io_channel is duplicated.
Change-Id: I722432718aca8c5846541816b6ecca56821d77f6
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/421182
Reviewed-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Kunal Sablok <kunal.sablok@intel.com>
Reviewed-by: GangCao <gang.cao@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Locating spdk_bdev structure to the beginning of raid_bdev structure
will simplify the hierarchy and match other bdev modules.
Change-Id: I1bfbf773bc96a4f144e6bff772ade05bb42762e9
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/420818
Reviewed-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Kunal Sablok <kunal.sablok@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
The poll group pause, resume, remove, and add functions are only called
from the subsystem_state_change_on_pg function. Previously, they would
return immediately and the state change would move on to the next
channel. However, some of these functions (specifically remove) kick off
asynchronous APIs and we should not iterate past them until those
asynchronous operations complete.
Change-Id: I78804273b39f2d171ba26ac4478ad515356833f3
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/419289
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Use strdup for all strings allocated by JSON decoder and free them on
all paths.
Change-Id: Ia30d32c2fd6bd79f62588ca19bd3bb093d38a502
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/420325
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Kunal Sablok <kunal.sablok@intel.com>
For front-ends like iSCSI (and NVMe-oF in the future)
which want the backend to specify the data buffer, the
RAID module doesn't copy the pointer to the allocated
buffer from the child IO back to the parent IO. It
really can't copy the pointer - the child IO owns it
and will free it.
So the RAID module needs to allocate the buffer first
and then pass it down.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I3b1eceac9b1cdd26130e59e1d400c9869a19f881
Reviewed-on: https://review.gerrithub.io/420677
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Previously cluster sizes only set as default were tested,
this patch adds test for load when different cluster_size is used.
Change-Id: I585adff25c81de809435ef235314cd4b8b116261
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/420607
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
At times, it may be necessary to free requests without completing them.
For example, when freeing a qpair, one needs to free the AER sent from
the host before deleting the qpair. It is important not to send a
completion for the AER because:
1. According to the spec, this will trigger the host to send another AER
2. No Asynchronous Events have occured, so we should not complete the
AER.
Change-Id: I92e163f0fed0ee2bc942569a647cb3c1967edec9
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/419732
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Since destroy_lvol_bdev was implemented, it is now more convinient
to call destroy/unload for lvol store explicitly.
Rather than using lvs struct to pass a request checked on each
lvol close/destroy.
Change-Id: I56ee626e96f8752909d1584a20fe3345c5607fdc
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/420285
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
It is not an error if bdev is smaller than cluster size so convert this
error to infolog. This fixes false error message dring examine process.
To return proper error message when creating blobstore using RPC the
_spdk_bs_alloc was adjusted to return errocode that is propagated up and
converted to "No space left on device".
Fixes#316
Change-Id: Ic9803720a55125fcfa34263346f2d9e1aae03a53
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/420054
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
This patch fixes lvol delete behaviour.
First, we look if there are any dependencies that disallow lvol deletion.
If there are any (i.e. dependent clones) we fail.
Otherwise we delete lvol and unregister associated bdev.
destroy_bdev no longer deletes lvol.
Fixes#345
Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I99e6abded2ed3ae2742103f81fc7eb937ad1cab4
Reviewed-on: https://review.gerrithub.io/407402
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Previously on all ut_lvol_* tests, creation of lvs was mocked.
There is no reason to keep it that way instead of creating
actual lvs using same code that is unit tested.
Change-Id: Id0a7816da0c195d596a6da516471b3f46bb4e475
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/419562
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
This variable was unnecessary since one within lvs->bs_dev
can be used to verify functionality and within stubs.
Change-Id: I43e12236e08f70381e0da5730b760db4ec3c577b
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/419561
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Those did not affect the tests, thus can be removed.
Change-Id: Ic1a23d4a40a0b5af1a2aff770e6eb4f0614c8f04
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/419560
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Added proper paths for examine:
- if lvs loaded, do spdk_lvs_unload
- if lvol created, call spdk_lvol_close instead of spdk_blob_close
Change-Id: I77a2905417d1c53f36b96da3fbea2679d9a2f56d
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/419558
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
RPC method (nvme-passthrough-cmd) needs base64 to
transfer binary buffer of nvme-cmd through Json text.
Change-Id: I50b9d0efca97da45d6d4fd2a1b53b74811947191
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/417805
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
This patch prevents to copy cluster data when there is not
backing blob to improve cluster allocation performance
in thin provisioned blobs.
Change-Id: Ie766d2e5274daa74c2b13b2198a20205e3417467
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-on: https://review.gerrithub.io/417938
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Before this patch when serializing extents,
unallocated clusters were treated as separate lba.
This caused metadata to grow without need.
Change-Id: I5d66466dda5f5e6d4d53f4ed5bd0bac18c74be96
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/419180
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
When hot removal of a LUN is started, callback is called for each
iSCSI connection which accesses the LUN. Callback checks all transfer
tasks complete and then close the LUN.
If the connection clears all transfer tasks before getting all responses
to them from the initiator, the initiator continues to retry data write.
Hence the connection have to wait until all transfer tasks complete.
Change-Id: Iad9063673cfedbd78758890d55a4254512e4fca4
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/417199
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Use a descriptor got by opening the LUN to allocate IO channel of the LUN.
This requires opening the LUN is done before allocating IO channel of the
LUN.
Use the descriptor to free IO channel of the LUN too.
Additionally, assert is added to the close LUN function to check if
IO channel is freed before the last close LUN function is called.
Change-Id: Iafb2f9ce790fff25801ea45b3286f3e26943807b
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/417807
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
