ivampiresp/Spdk
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

sock/posix: Add sock to pending list on zcopy event

Browse Source 
In NVMF TCP initiator when zero copy is disabled,
all requests are completed when we receive EPOLLIN event
for socket, add socket to pending_recv list and call socket's
callback which calls qpair_process_completions. As part of
completions processing on NVME level we receive the number
of completions and resubmit the same number of queued requests.

When zero copy is enabled, some transport requests can be
completed when we receive and process EPOLLERR event, it
happens out of qpair_process_completions context. So part
of requests can be completed, transport level contains
free requests but NVME layer don't have info about it
until it calls qpair_process_completions. And there is
a chance that on posix level when we poll sockets we
receive only EPOLLERR flag without EPOLLIN. In this
case we can complete several requests but don't call
qpair_process_completion so we don't resubmit queued
requests. It may lead to a hang in the end of test run
when there are no mo requests to be completed on transport
level (no EPOLLIN event) and we receive EPOLERR only,
so we can't resubmit queured requests.

This patch fixes this problem, it add a socket to
group's pending_recv list if we received EPOLERR
event and completed at least 1 socket request.
So socket's callback can be called even without
EPOLLIN event.

Fixes issue #1685

Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Change-Id: I21d5c2fe6eb0787aab9531925a7f0e2fe18bafaa
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6162
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
This commit is contained in:
Alexey Marchuk 2021-01-29 09:47:13 +03:00 committed by Tomasz Zawadzki
parent 1293377629
commit 2ae4adc342
1 changed files with 7 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

7
module/sock/posix/posix.c
View File

@ -658,6 +658,7 @@ static int
_sock_check_zcopy(struct spdk_sock *sock)
{
struct spdk_posix_sock *psock = __posix_sock(sock);
struct spdk_posix_sock_group_impl *group = __posix_group_impl(sock->group_impl);
struct msghdr msgh = {};
uint8_t buf[sizeof(struct cmsghdr) + sizeof(struct sock_extended_err)];
ssize_t rc;
@ -721,6 +722,12 @@ _sock_check_zcopy(struct spdk_sock *sock)
}
}
/* If we reaped buffer reclaim notification and sock is not in pending_recv list yet,
* add it now. It allows to call socket callback and process completions */
if (found && !psock->pending_recv) {
psock->pending_recv = true;
TAILQ_INSERT_TAIL(&group->pending_recv, psock, link);
}
}
}