Note that this also works around a false positive in
gcc-11 of type -Wstringop-overread.
Fixes issue #2391.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ib5301b9ef9fa3ead2a1a2318655533a8cfba03fe
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11709
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
When generating the discovery log page, add entries for
the discovery subsystem, skipping the listener associated
with the command generating the log page.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Id6a14a7d5cdce483f8f3c2eff1b4ededd40bc029
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11542
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Currently we accept connections to the discovery
subsystem on any listener that has been added to
any subsystem (not just the discovery subsystem).
This is not proper behavior, especially for TCP. TCP
defines port 8009 (not 4420) as the discovery port,
so the current behavior means that if NVM subsystems
are listening on port 4420, then the discovery
subsystem by default is listening there too.
For now, continue to allow connections, but print
a warning message when someone connects to the
discovery subsystem on a listener trid that wasn't
previously added.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I734bc49d1a21b2edfb675aef4b8551e2d0ccd4d7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11539
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
According to NVMe over Fabrics spec number of SGLs supported by the
controller is reported in MSDBD. But it is also implicitly limited by
command capsule size (IOCCSZ) since SGL are passed in capsule.
This patch adjusts max_sges to capsule size if required. Adjustment to
MSDBD is also moved to transport layer because it is fabrics specific
parameter and is not valid for PCIe transport.
Signed-off-by: Evgeniy Kochetov <evgeniik@nvidia.com>
Change-Id: I44918eb949345c61242ca50a524d21d04b6ac058
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11669
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This confirms that the error path can return more efficient
without memcpy such as xattr->name.
Signed-off-by: Dong Yi <dongx.yi@intel.com>
Change-Id: Ic2ed28121ed76eda9d7b24ed6c4c95b0588817de
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11654
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
One simple fix for nvmf_vfio_user_qpair_abort_request().
Current implementation mixed up request of abort cmd and the request
to abort, which cause problems.
Signed-off-by: Rui Chang <rui.chang@arm.com>
Change-Id: Ia0db9aa738e372789fc502ef877fd1c841c0a2e3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11711
Reviewed-by: John Levon <levon@movementarian.org>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
The patch
nvme: Set dnr to zero for nvme_qpair_abort_reqs()
1b3172f726
did the change stated in the title.
However,
Revert "nvme/rdma: Correct qpair disconnect process"
c8f986c7ee
destroyed it for RDMA transport.
Additionally, we had still set DNR to 1 in nvme_qpair_init().
This patch fixes both.
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: Iee60ac24aa7e04cce0f394014c9d9afc9d2b56ec
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11644
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
When function returned from the error handling the mempool on
'sess' was not released which lead to a memory leak.
Fixes issue #2393.
Signed-off-by: Weiguo Li <liwg06@foxmail.com>
Change-Id: Ida3651e9369fb5c4948969480d398a723b2cb6a2
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11714
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
We do the null check for 'fc_req', but already dereferenced it
before the check. Swap their position to avoid null dereference.
Fixes issue #2395.
Signed-off-by: Weiguo Li <liwg06@foxmail.com>
Change-Id: I33b9e6b51b54f6ada9c072cf7ab0acda2622472f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11721
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
A common pattern is:
if (foo->thread == spdk_get_thread())
cb(arg);
else
spdk_thread_send_msg(foo->thread, cb, arg);
for cases where it's important the callback runs on a particular thread,
but it doesn't matter if it's synchronous or asynchronous.
Add a new API to support this pattern, and convert over the current
instances.
Signed-off-by: John Levon <john.levon@nutanix.com>
Change-Id: Idfbf77c02c9321c52e07181ffd8b0c437e1ab335
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11503
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
We can ask libvfio-user for the listening socket fd, and register that
for SPDK interrupt handling.
Signed-off-by: John Levon <john.levon@nutanix.com>
Change-Id: I8d0ba7a86403f2d0170b9359480f1fefc1036557
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10721
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
The accept poller only needs to run when vfu_attach_ctx() makes sense:
in other words, when we don't have a controller created.
Signed-off-by: John Levon <john.levon@nutanix.com>
Change-Id: Icef4e6184c9ae6d7951d015530a05132c4ba6994
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10720
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Now vfio-user owns its the accept poller itself, there's no reason to
loop across all endpoints: instead, the lifetime of the accept poller is
better matched by creating it in the ->listen() transport callback.
Signed-off-by: John Levon <john.levon@nutanix.com>
Change-Id: Ia92e29b1cee263f1461f640cfdd27cdb674848fb
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10719
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
For the benefit of forthcoming vfio-user changes, register the poll
group poller prior to calling the transport create callback, and pass in
a pointer to the poll group itself.
Signed-off-by: John Levon <john.levon@nutanix.com>
Change-Id: Idbc24126c9d46f8162e4ded07c5a0ecf074fc7dd
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10718
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Json writes this scheduler name using pointer, but it sometimes be null.
This case it exists, so add check for it.
Fixes issue #2384
Signed-off-by: Dong Yi <dongx.yi@intel.com>
Change-Id: I7c989c72cfbd53ea6b02d86457b29440484e5a37
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11677
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
The above values were added with shared CQ feature, and they
are left when restore the CQs in destination VM.
Change-Id: Ib1f28dad833da31e571eb2e2f0b5c81f0bf05a3b
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11419
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
The CSTS register in source VM was left to migrate.
Fix#2362.
Change-Id: Ieef028578d2897b27a6ff594b16801462eb1b75e
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11418
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Thanos Makatos <thanos.makatos@nutanix.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
The only release to not unlink hugepages after mmaping
them is for multiprocess. But if shm_id is not
specified, then we aren't using multiprocess. This
ensures that all hugepages get released when the
process exits, even if there is memory in those
hugepages that was not freed during process shutdown.
Make sure we don't enable both huge-unlink and
single-file-segments at the same time though, DPDK doesn't
support that.
Note that even when using multi-process, if hugepages
aren't released, they aren't really leaked. DPDK will
clean them up next time the application runs.
Fixes issue #2267.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I017bd4f7ed9cf6aaa141879539b099fb48f357f4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10991
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Move some of this code into separate functions. There
is no change in functionality here - this just helps
reduce the size and complexity of an upcoming patch.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I72243f17312e66bb6ef2168b9b78076c9eb2e6f3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11531
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
The dif_enable is not cleared in _nvmf_rdma_request_free. When the
rdma_req is used again, the dif_enable is true while
dif.dif_ctx->block_size is zero. As a result, an infinite loop occurs in
nvmf_rdma_fill_wr_sgl.
Fix issue: #2380
Signed-off-by: Chunsong Feng <fengchunsong@huawei.com>
Change-Id: Ic179855f7b257e39ed4a5f6705fbc9dea64210ae
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11646
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Since there is a pthread_mutex_unlock() in normal condition, another
pthread_mutex_unlock() in the "tmp != NULL" branch should be removed,
otherwise will cause a double unlock.
Fixes issue #2378.
Signed-off-by: Weiguo Li <liwg06@foxmail.com>
Change-Id: I6c80a9527dd60e0b7c1d3c54b6da371b31118f02
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11642
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
bs_back_dev_lba_to_io_unit() and bs_num_pages_to_cluster_boundary() are
unused inline functions. The last consumer (by the earlier _spdk_* name)
was removed in commit 6609b776.
Change-Id: Ib1babfed8002fb44451b337aa0db66c15a6805d2
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11561
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
bs_sequence_to_batch_completion has been unused since the removal of
other unused code in commit ba870c2e99.
Change-Id: Ifb60c65c1c68d1855b49eda4c57d99e983bca5ec
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11560
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
rte_dmadev was introduced in DPDK 21.11, and rte_vhost
is now dependent on it. So link rte_dmadev if we find
it and if CONFIG_VHOST is enabled.
Fixes issue #2374.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iccbf7cb897f51cbc9d545274d4d00a442b2fd353
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11594
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
For reference, this was deprecated in Aug 2020, commit
ID 511fe1553.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ic0a16af9113b8b136271a2ce6a071bbc379261c4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11545
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
This was a parameter on the nvmf_create_transport
RPC, and was replaced with max_io_qpairs_per_ctrlr to
reduce confusion on whether this number included the
admin queue or not.
nvmf_vhost test was using this deprecated parameter.
Change it to use -m (max_io_qpairs_per_ctrlr)
instead. '-p 4' would have been evaluated as 1 admin
queue + 3 I/O queues, but it's likely the intent
was for 4 I/O queues. This is a perfect example of
why this parameter was deprecated.
For reference, this was deprecated in June 2020,
commit 1551197db.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I4364fc0a76c9993b376932b6eea243d7cefca9cd
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11543
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
To supplement useful information in bdev trace record,
offset and len of each I/O have been added.
Signed-off-by: zhaoshushu.zss <zhaoshushu.zss@alibaba-inc.com>
Change-Id: I3e776144d16cb9eda2a9fb72b83d423ac3050f0d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11504
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
With async connect, we need to avoid the case
where the initiator is sending the icreq, and
meanwhile the application submits enough I/O
such that the request objects are exhausted, leaving
none for the FABRICS/CONNECT command that we need
to send after the icreq is done.
So allocate an extra request, and then use it
when sending the FABRICS/CONNECT command, rather
than trying to pull one from the qpair's STAILQ.
Fixes issue #2371.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: If42a3fbb3fd9d863ee48cf5cae75a9ba1754c349
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11515
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Group fields such that those not used in the I/O path
are at the end of the structure.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I43eca1faacd29a5bf34be6ee644191d865cd42a9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11514
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
This macro will be used in an upcoming patch
that needs to construct an nvme_request structure
outside of the standard nvme_allocate() routines.
Examined x86 optimized assembly with this patch,
and there is no change.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I0f6b8500e06b56edc33f437f351536cf857d13d3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11513
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Split part of IO completion into 2 functions.
This patch reduces changes in the next patch
which will reuse some part of spdk_bdev_io_complete
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Change-Id: Iaeac81aa5208b4ca303f60410b6a54f8df13b069
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11519
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Move part of function _bdev_io_set_buf which sets
metadata pointer to another function _bdev_io_set_md_buf
Next patches will make copying of bounce buffer async,
metadata will be copied when data copy completes.
This patch makes next change simpler
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Change-Id: Iced45393f43f9c5a4818e4e9eadb3351583e0c00
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11518
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Although there are no use cases right now where a batch can have
mixed op types, there may be in the future and rather than have
one blow up because ops have different reserved fields and its
not valid to submit an op with a non-zero reserved field, go
ahead and zero these out like we do with descriptors in the non
batch case.
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I6d1bb416dc84aa1f76407c76aedf0768dd003218
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11325
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
We always set num_batches to the same value we set num_descriptors
so just make it explicit. Makes it easier to experiment with
different values when performance testing.
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I648262001772c791a032d6cab38dc3b03c1d55c1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11354
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This has changed to control the number of read buffers allocated to the
group, but it is only valid to set this register if the device has
indicated it supports it. Further, the default value is what we want
anyway, so we can skip setting it altogether.
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: Ic54672ea6cb16acc7613860e36d9f7033048bd98
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11484
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
All of the other structs and unions spell out register, so match the
style.
Change-Id: Ie502e80206305037d1518a1db590d89b7479abb4
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11433
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
This is called GENSTS in the spec, so match that name.
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: I0e8f917e13908f3920ab297e14cb3adee856eaa5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11432
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
SPDK can submit more commands to remote NVMf target than allowed by
negotiated queue size. SPDK submits up to SQSIZE commands, but only
SQSIZE-1 are allowed.
Here is a relevant quote from NVMe over Fabrics rev.1.1a ch.2.4.1
“Submission Queue Flow Control Negotiation”:
If SQ flow control is disabled, then the host should limit the number
of outstanding commands for a queue pair to be less than the size of
the Submission Queue. If the controller detects that the number of
outstanding commands for a queue pair is greater than or equal to the
size of the Submission Queue, then the controller shall:
a) stop processing commands and set the Controller Fatal
Status (CSTS.CFS) bit to ‘1’ (refer to section 10.5 in the NVMe Base
specification); and
b) terminate the NVMe Transport connection and end the association
between the host and the controller.
Signed-off-by: Evgeniy Kochetov <evgeniik@nvidia.com>
Change-Id: Ifbcf5d51911fc4ddcea1f7cde3135571648606f3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11413
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
According to NVMe over Fabrics specification (rev.1.1a) HSQSIZE sent
in RDMA_CM_REQUEST private data (ch.7.3.6.4) shall be the same as
SQSIZE later sent in Connect command (ch.3.3).
SPDK NVMe RDMA initiator adjusts SQSIZE to CRQSIZE received from
target in RDMA_CM_ACCEPT private data. Target is allowed to send
CRQSIZE < HSQSIZE if RNR retries are used. So, it is possible that
SQSIZE sent by SPDK will be lower than previously sent HSQSIZE. There
are targets validating this match and they reject connection from
SPDK.
Linux kernel NVMe initiator doesn't perform such adjustments and
connects well to such targets.
This patch aligns SPDK behavior with specification and Linux kernel
implementation.
Signed-off-by: Evgeniy Kochetov <evgeniik@nvidia.com>
Change-Id: I01968d1c07d284396fa5939932d85841351d7a45
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11350
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
The controller data structure may be freed before subsystem resume done
callback, we can take endpoint as the input parameter to avoid this issue.
AddressSanitizer: heap-use-after-free on address 0x625000046100 at pc 0x00000082818f bp 0x7fff7b09bd10 sp 0x7fff7b09bd00
READ of size 8 at 0x625000046100 thread T0 (reactor_0)
#0 0x82818e in vfio_user_dev_quiesce_resume_done /spdk/lib/nvmf/vfio_user.c:2147
#1 0x782cc0 in subsystem_state_change_done /spdk/lib/nvmf/subsystem.c:634
#2 0xad047b in _call_completion /spdk/lib/thread/thread.c:2344
#3 0xabc48d in msg_queue_run_batch /spdk/lib/thread/thread.c:710
#4 0xac0670 in thread_poll /spdk/lib/thread/thread.c:926
#5 0xac0ead in spdk_thread_poll /spdk/lib/thread/thread.c:986
#6 0x9a5b4f in _reactor_run /spdk/lib/event/reactor.c:920
#7 0x9a6442 in reactor_run /spdk/lib/event/reactor.c:958
#8 0x9a717c in spdk_reactors_start /spdk/lib/event/reactor.c:1060
#9 0x99884a in spdk_app_start /spdk/lib/event/app.c:643
#10 0x407e82 in main /spdk/app/nvmf_tgt/nvmf_main.c:75
#11 0x7f822095ff42 in __libc_start_main (/lib64/libc.so.6+0x23f42)
#12 0x407abd in _start (/spdk/build/bin/nvmf_tgt+0x407abd)
0x625000046100 is located 0 bytes inside of 8320-byte region [0x625000046100,0x625000048180)
freed by thread T0 (reactor_0) here:
#0 0x7f82219ff91f in __interceptor_free (/lib64/libasan.so.5+0x10d91f)
#1 0x837059 in _free_ctrlr /spdk/lib/nvmf/vfio_user.c:2976
#2 0x837327 in free_ctrlr /spdk/lib/nvmf/vfio_user.c:2996
#3 0x843541 in nvmf_vfio_user_close_qpair /spdk/lib/nvmf/vfio_user.c:3742
#4 0x7d1d91 in nvmf_transport_qpair_fini /spdk/lib/nvmf/transport.c:604
#5 0x7ad922 in _nvmf_qpair_destroy /spdk/lib/nvmf/nvmf.c:1055
#6 0x761362 in nvmf_qpair_request_cleanup /spdk/lib/nvmf/ctrlr.c:4026
#7 0x761906 in spdk_nvmf_request_free /spdk/lib/nvmf/ctrlr.c:4041
#8 0x75a931 in nvmf_qpair_free_aer /spdk/lib/nvmf/ctrlr.c:3576
#9 0x7ae626 in spdk_nvmf_qpair_disconnect /spdk/lib/nvmf/nvmf.c:1127
#10 0x83db36 in _vfio_user_qpair_disconnect /spdk/lib/nvmf/vfio_user.c:3433
#11 0xabc48d in msg_queue_run_batch /spdk/lib/thread/thread.c:710
#12 0xac0670 in thread_poll /spdk/lib/thread/thread.c:926
#13 0xac0ead in spdk_thread_poll /spdk/lib/thread/thread.c:986
#14 0x9a5b4f in _reactor_run /spdk/lib/event/reactor.c:920
#15 0x9a6442 in reactor_run /spdk/lib/event/reactor.c:958
#16 0x9a717c in spdk_reactors_start /spdk/lib/event/reactor.c:1060
#17 0x99884a in spdk_app_start /spdk/lib/event/app.c:643
#18 0x407e82 in main /spdk/app/nvmf_tgt/nvmf_main.c:75
#19 0x7f822095ff42 in __libc_start_main (/lib64/libc.so.6+0x23f42)
previously allocated by thread T0 (reactor_0) here:
#0 0x7f82219fff16 in __interceptor_calloc (/lib64/libasan.so.5+0x10df16)
#1 0x837413 in nvmf_vfio_user_create_ctrlr /spdk/lib/nvmf/vfio_user.c:3010
#2 0x83bc68 in nvmf_vfio_user_accept /spdk/lib/nvmf/vfio_user.c:3313
#3 0xabfbd8 in thread_execute_timed_poller /spdk/lib/thread/thread.c:872
#4 0xac0c75 in thread_poll /spdk/lib/thread/thread.c:960
#5 0xac0ead in spdk_thread_poll /spdk/lib/thread/thread.c:986
#6 0x9a5b4f in _reactor_run /spdk/lib/event/reactor.c:920
#7 0x9a6442 in reactor_run /spdk/lib/event/reactor.c:958
#8 0x9a717c in spdk_reactors_start /spdk/lib/event/reactor.c:1060
#9 0x99884a in spdk_app_start /spdk/lib/event/app.c:643
#10 0x407e82 in main /spdk/app/nvmf_tgt/nvmf_main.c:75
#11 0x7f822095ff42 in __libc_start_main (/lib64/libc.so.6+0x23f42)
SUMMARY: AddressSanitizer: heap-use-after-free /spdk/lib/nvmf/vfio_user.c:2147 in vfio_user_dev_quiesce_resume_done
Change-Id: Icf5e5b360b9107a3c5eb960ae59b7fe10ace1c66
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11420
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: John Levon <levon@movementarian.org>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
There are errors occur that uninitialised value created by a stack allocation when running unittest_accel and unittest_nvme_rdma with valgrind.
Change-Id: I4b48b472cc7c189cbcaf8ca772830a23118e7e17
Signed-off-by: Jaylyn Ren <jaylyn.ren@arm.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10559
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Next patches enable memory domains async pull/push
functionality.
Previously buffer from internal bdev pool was passed
as func argument to bdev_io_get_buf_complete. Now
since bdev_io_get_buf_complete can be called
asynchronously, this buffer is stored in
bdev_io->internal.buf
Also move bdev_io_get_buf_complete up in file
to minimize changes in next patches
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Change-Id: I6e9d3b35dc85e0e88703dd24a4b4da837adc5b74
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11165
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
if ctrlr->listener was NULL, nvmf_ctrlr_get_ana_state() returned
inaccessible even if ana_reporting was disabled. Then the corresponding
initiator received unexpected ANA error and could not process it
appropriately.
Change nvmf_ctrlr_get_ana_state() to return optimized always if
ana_reporting is disabled.
Additionally, check if ctrlr->listener is not NULL before calling
SPDK_DTRACE_PROBE3().
Fixes#2335
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: Ib2376694cf89d85ec5687fba7e87439f494f30b0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11402
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
it shall be executed on ctrlr's thread not subsystem's
Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: I58c60525191085d3d6a583862ba5d71ea90940c7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11105
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
