Leverage SO_INCOMING_CPU to get the CPU affinity of connections
(sockets). And allocate the connections to specific poll groups,
which aims to utilize cache locality.
From our test:
6 P4600 NVMe on target,target uses 8 cores, NIC irqs are bound to
these 8 cores, and initiator side uses 24 and 32 cores,
we can get 11%~17% randwrite performance boost for posix, and 8%~12%
for uring.
Change-Id: I011e0a21502c85adcccd4a14fbe9838b43f54976
Signed-off-by: Richael Zhuang <richael.zhuang@arm.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5748
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Use the same style compared the code in posix_sock_close.
Thus if we cannot close sock->fd, i.e., we leak the fd,
but we can still free the memory related with uring sock.
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: Id2f0e8a2c7065f100c2b009e76a49b528fd221b6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6539
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
The statement causes this issue is:
assert(group_impl->num_removed_socks < MAX_EVENTS_PER_POLL);
The call trace is:
The previous solution is:
commitid with: e71e81b631
But with this solution, it will always add the sock
into the removed_socks list even if it is not under polling
context by sock_group_impl_poll_count. So it will exceed the size of
removed_socks array if sock_group_impl_poll_count function will not be
called. And we should not use a large array, because it is just a workaround,
it just hides the bug.
So our current solution is:
1 Remove the code in sock layer, i.e., rollback the commit
e71e81b631. This patch is
not the right fix. The sock->cb_fn's NULL pointer case is
caused by the cb_fn of write operation (if the
spdk_sock_group_remove_sock is inside the cb_fn). And it is not
caused by the epoll related cache issue described in commit
"e7181.." commit, but caused by the following situation:
(1)The socket's cb_fn is set to NULL which is caused by
spdk_sock_group_remove_sock by the socket itself
inside a call back function from a write operation.
(2) And the socket is already in the pending_recv list. It is
not caused by the epoll event issue, e.g., socket A changes Socket B's
cb_fn. By the way, A socket A should never remove a socket B from a polling group.
If it really does it, it should use spdk_thread_sendmsg to make sure
it happens in the next round.
2 Add the code check in each posix, uring implementation module.
If sock->cb_fn is NULL, we will not return the socket to the active socks list.
And this is enough to address the issue.
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I79187f2f1301c819c46a5c3bdd84372f75534f2f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6472
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Default to using epoll unless __FreeBSD__ is defined. Add macros SPDK_KEVENT
and SPDK_EPOLL to indicate whether epoll or kevent is present. The macros
match the naming convention for SPDK_ZEROCOPY which controls zero copy
in a similar way.
Signed-off-by: Nick Connolly <nick.connolly@mayadata.io>
Change-Id: I4c46fb94b254cb075427bfe07a8085887254c45a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6466
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
When socket is being created and zcopy is disabled
by the config, we can return from posix_sock_alloc
function before we try to set quick_ack
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Change-Id: I6670b8337e70ec12b18a5e6753674fbef9e95648
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6382
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
A single sock connection can call posix_sock_flush,
and this sock may not belong to a polling group.
So add the check in sock_check_zcopy to avoid such issue.
Fixes#1788
Change-Id: Id0a2f80ad0f3cdb7fc736a3be3211e49513751b1
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6334
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
To allow SO_MINOR updates on LTS for the whole year it is supported,
the major version for all components needs to be increased.
This is to prevent scenario where two versions exists with matching
versions, but conflicting ABI.
Ex. Next SPDK release adds an API call increasing the minor version,
then LTS needs just a subset of those additions.
Increasing major so version after LTS, allows the quarterly releases
to update versions as needed. Yet allowing LTS to increase minor
version separately.
Disabled test for increasing SO version without ABI change, as
that is goal of this patch. This check shall be removed with SPDK 21.04
release.
This patch:
- increases SO_VER by 1 for all components
- resets SO_MINOR to 0 for all components
- removes suppressions for ABI tests
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I44d01154430a074103bd21c7084f44932e81fe72
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6167
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
When zero copy is enabled in initiator, there could
be the case that a socket connection does not belong
to a polling group, i.e, the application does not use
socket polling group. Then we should actively call
_sock_check_zcopy in posix_sock_flush function when zero
copy policy is enabled.
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: Idceaa7557eb265daa878db40c922494c3de35ea8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5423
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Community-CI: Mellanox Build Bot
In NVMF TCP initiator when zero copy is disabled,
all requests are completed when we receive EPOLLIN event
for socket, add socket to pending_recv list and call socket's
callback which calls qpair_process_completions. As part of
completions processing on NVME level we receive the number
of completions and resubmit the same number of queued requests.
When zero copy is enabled, some transport requests can be
completed when we receive and process EPOLLERR event, it
happens out of qpair_process_completions context. So part
of requests can be completed, transport level contains
free requests but NVME layer don't have info about it
until it calls qpair_process_completions. And there is
a chance that on posix level when we poll sockets we
receive only EPOLLERR flag without EPOLLIN. In this
case we can complete several requests but don't call
qpair_process_completion so we don't resubmit queued
requests. It may lead to a hang in the end of test run
when there are no mo requests to be completed on transport
level (no EPOLLIN event) and we receive EPOLERR only,
so we can't resubmit queured requests.
This patch fixes this problem, it add a socket to
group's pending_recv list if we received EPOLERR
event and completed at least 1 socket request.
So socket's callback can be called even without
EPOLLIN event.
Fixes issue #1685
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Change-Id: I21d5c2fe6eb0787aab9531925a7f0e2fe18bafaa
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6162
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
The purpose is to reduce the duplicated functions
in posix and uring implmentation.
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: Ia0568b2490d362e7e78fa59b3ca88a60313ba0bd
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5284
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Since spdk_sock_group_poll should do level trigger to
the callback func registered in spdk_sock_group_add_sock,
if there is data in pipe, but not event occurs, poll func
should still reap sock who has data in pipe.
Change-Id: If3a983f80fd04708e45ad0398c7d34018ec52bc7
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5072
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
By design the opts for each implementation
can not match spdk_sock_impl_opts.
During get_opts for specific implementation
only used fields are filled.
Yet iterating over all spdk_sock_impl_opts fields
would yeild garbage values for unset fields.
This is the case right now when doing save_config RPC
with uring enabled. A garbe value for enable_zerocopy_send
is returned.
sock.c:829:62: runtime error: load of value 165, which is not a valid value for type '_Bool'
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ie0512a7dffc36c8ff89256d08f8a2f4fefcf9e83
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4699
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
In NVME TCP initiator zero copy is enabled for IO qpairs
and disabled for admin qpairs
Change-Id: Ibdf521dccde9b95ec5dd15a5eb2baed8fcf8b88e
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Signed-off-by: Or Gerlitz <ogerlitz@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4211
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
A preparation step for enabling zero copy in NVMEoF TCP initiator.
This option will be used to disable zero copy
for admin qpair. This is needed since the admin
qpair's socket is not connected to socket poll group
and we can't receive buffer reclaim notification.
Change-Id: Ibfbb8a156aafcd7ba8975a50f790da7fbd37d96f
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Signed-off-by: Or Gerlitz <ogerlitz@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4210
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
When net.core.optmem_max is not set high enough, a call to sendmsg()
might fail with ENOBUFS. Currently this is treated as an error.
When we have no more buffer space left, we should continue to process
any completions and by doing so, free up the auxiliary buffers we ran out
of.
With this change I was able to run perf against the spdk target with a
purposely set to a low, value of optmem_max, where previously it would
fail.
This fixes github issue #1592
Signed-off-by: Jeffry Molanus <jeffry.molanus@gmail.com>
Change-Id: Ieeeed4fbecd827d0da815456b57fbe81495fe54d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4129
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
This patch is used to enable placement_id getting
in sock layer and also add the rpc support.
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I70de57b0ed392a0aefce9d3ff1f61ef924015a87
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4146
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
The type of sendmsg_idx is uint32_t, so the maximal
is 2^32 -1, so it could be overflow and get 0, so
we should fix it.
PS: I think that our code may have potential defect.
In my experiment, I try to init sendmsg_idx with 2^32 -1,
so the first req->internal.offset = 2^32 - 1.
But for the ee_info and ee_data in "struct sock_extended_err"
got from _sock_check_zcopy is all 0 in the target side.
So it means that the this req will never be completed.
With the increase of sendmsg_idx (the type is
uint32_t), sendmsg_idx will finally goto 2^32 - 1, so I
think it will still kick the issue I described.
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: Ic9aaf629d73d5b7e2c81800a4f7f92c728adbc34
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3948
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
This patch removes implementation of VPP socket abstraction
along with ways to compile it.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I089f7703cfc4fb517f8f80f4368e544bced549b6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3734
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Add a helper function iscsi_parse_redirect_addr() to validate the
passed IP address-port pair.
iSCSI login redirection will support only numeric IP address and
TCP port, and add AI_NUMERICSERV and AI_NUMERICHOST.
This function is almost same as nvme_tcp_parse_addr() and
nvme_rdma_parse_addr().
Besides, update error log in posix_sock_create() to use
gai_strerror(). gai_strerror() will provide more accurate
information as done by nvme_tcp_parse_addr() and nvme_rdma_parse_addr().
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I65c6de81a64dcb26551ce796172d0458e1c298a7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3357
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
TCP delayed ACK can be disabled or enabled by enabling or disabling
quick ACK, respectively.
The recently added spdk_sock_impl_opts is helpful for sock library
to control quick ACK.
Hence this patch adds and uses an option enable_quickack. The option
is effective only for the POSIX sock module.
We have spdk_sock_opts now too but spdk_sock_impl_opts will be better
for this case.
This option is not supported on FreeBSD. FreeBSD users can set the
option globally via sysctl if desired.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ic89620267acce5872dc8ecaf7a99bb70ae97e993
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3603
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
The pipe buffer has obvious performance influence on arm64. The
following is my test result with 1core, we can also enable it
on arm64 currently like the posix socket. And later we can find
the optimal pipe size that won't cause a degradation for large
payloads.
randwrite randread
512 byte 61% 97%
4096 byte 84% 16%
16384 byte -13% -17%
Signed-off-by: Richael Zhuang <richael.zhuang@arm.com>
Change-Id: Ib4df60751c5e06ef9bd7fc7bb7efafa5ad4de211
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3329
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Zero copy send can cause performance degradation with small
payloads. This patch adds an option to disable it if required. By
default zero copy is enabled.
Signed-off-by: Evgeniy Kochetov <evgeniik@mellanox.com>
Change-Id: I14f2b21ad375e770cb08f850360898bac675b351
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3344
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Receive pipe reduces number of system calls and gives significant
performance improvement with kernel TCP stack and relatively small IO
sizes. With user space TCP/IP implementations there are no system
calls and double buffering introduced by pipe has negative impact on
performance. Receive pipe remains enabled by default.
Signed-off-by: Evgeniy Kochetov <evgeniik@mellanox.com>
Change-Id: Ic5ddee42293df2c233ba7ffbe6662de7917ac586
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3343
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Call recv to trigger busy polling even when no socket is active. when
epoll_wait returns zero, the first socket in poll group is used to
trigger busy polling in kernel stack and potentially reap incoming data
Change-Id: I15f04cb4a2c7b382dd07391eda69678fd7919790
Signed-off-by: Sudheer Mogilappagari <sudheer.mogilappagari@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3180
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
For some exceptional cases (e.g.,
https://github.com/spdk/spdk/issues/1486),
we may detect POLLERR or other events. So for those events,
we can just ingore it, but not use SPDK_UNREACHABLE.
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I073575408783ff75e50b40d45ddf09388a2cab96
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3262
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Poller should return status > 0 when it did some work
(CPU was used for some time) marking its call as busy
CPU time.
Active pollers should return BUSY status only if they
did any meangful work besides checking some conditions
(e.g. processing requests, do some complicated operations).
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: Id4636a0997489b129cecfe785592cc97b50992ba
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2164
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
If the socks parameter(passed in uring_sock_group_impl_poll) is NULL,
we do not need to handle the sock_flush and prep the pollin task,
otherwise it will cause the assert issue when we reap the task when
we handle the nvmf_shutdown_tc3 issue.
Because in uring_sock_group_impl_remove_sock, we finally
set sock->group = NULL. Without this patch,
when we call uring_sock_group_impl_poll in this function,
pollin_task or write_task are prepared, then in the next round,
we will reap those tasks again.
PS: Error info can be found in
https://ci.spdk.io/results/autotest-per-patch/builds/19186/archive/nvmf-tcp-vg-autotest/build.log
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I7e6deaa05e958b52e71e0bbf0ccdd20e35583685
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3031
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
According to the description of io_uring_queue_exit:
Tear down function for io_uring. Unmaps all setup shared ring buffers
and closes the low-level io_uring file descriptor returned by the kernel.
So we should remove the close operatoin on ring fd.
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I910c6e8acd935925b7985c2aa750df385004eb55
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2922
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Thus, we can make sure that when read data is larger than
the pipe size, it will not read the data into the pipe.
Change-Id: I87f3b03fd9b81eb693e9eae0fea9eef7d1b9eaa8
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2450
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
When the initiator terminates the connection suddently, we
found the keep alive timeout issue, e.g.,
nvmf_ctrlr_keep_alive_poll: *NOTICE*:
Disconnecting host from subsystem nqn.2016-06.io.spdk:cnode1 due to keep alive timeout.
The root cause is that we did not closes the connection on our target side on time
whening using uring sock, and this patch can fix the issue.
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I295f58bbdbae0ac3f5308f6eadef6a75c5ad07d8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2544
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
This reverts commit 79215d80c1.
Reason: Find the bugs while using batched. For example, if
we fetch 3 CQEs, A, B, C and put it in a cqes[] array.
Then we leverage io uring cqe seen to handle, Then we handle the
CQE A, then invokes the call back related with A. In A's call back,
it may also call the reap function (sock_uring_group_reap),
then the CQEs will be handled again. Thus the CQEs B and C can be already
handled. Then we will handle B or C again, then it triggers the error.
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I67ece3b24e677b88d66d08722b00539693b42e1e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2543
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Also, while we are here, consolidate setting SO_SUFFIX to one spot.
Previously, it was possible for a library to slip through
without an SO version.
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I4db5fa5839502d266c6259892e5719b05134518c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2361
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
The pipe buffer gives about 19% randwrite and 56% randread
performance boost on arm64 from my test. We can also enable it on
arm64.
Signed-off-by: Richael Zhuang <richael.zhuang@arm.com>
Change-Id: Iff4c6eaffb4ec5ae8fe02b35f72aed7f6b272bb5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2255
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Purpose: To disabled the zero copy support if IP is configured on a loopback
device.
To address issue: https://github.com/spdk/spdk/issues/1340
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: Ia337f108851653c60d259328b6fefd4ba1e3d654
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2372
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Purpose: This can improve the performance.
Change-Id: I3f5526ab8716cb0771b5e193afa9a0dbffec5cc0
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1929
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
We should add the cancel task feature in uring if
sock is removed from the group.
This issue is detected after the group polling
feature is added in NVMe tcp initiatator side. Before
the patch, SPDK NVMe initiator side will not use
group polling feature. After the group polling
feature added in the iniatiator side, we should add this
patch.
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I012b403a9f57a5a8ee96c83471b775f0d99b9eb6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2049
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Purpose: prepare for the next patch
Change-Id: Iae6a3d7a2c90a78c7dd1e510e718e4e1c07b3f32
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2048
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Port the similiar code from posix sock module if a sock
is removed and added to a new group.
Change-Id: I306bed3c90f79ee06a363fbab02c776e756cda2f
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2047
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Purpose: This is used to make users can specify
some options on the socket, e.g., the different priority for the socket.
While creating sockets, the priority needs to be set before connect()
and listen system calls, so better to add one parameter in spdk_sock_opts
which can contain options (e.g., priority) in spdk_sock_listen_ext and
spdk_sock_connect_ext functions.
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Signed-off-by: Sudheer Mogilappagari <sudheer.mogilappagari@intel.com>
Change-Id: I406238e9da7abd69f937b7072535a19124ed0169
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1874
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
