From f36c033c71b27aa8d479ce5985079ef862cdaa11 Mon Sep 17 00:00:00 2001 From: Richael Zhuang Date: Wed, 19 Jan 2022 14:23:26 +0800 Subject: [PATCH] uring: fix bug when inserting sock into pending_recv list There is io error when running NVMe over TCP fio test with uring socket. It's easy to reproduce the bug with the following configuration: target 1 core, 16NVMe SSD, 2 initiators each connects to 8 NVMe namespaces, each runs fio with numjobs=3. For if in each round, we inset the sock to the head of the pending_recv list, and then get max_events socks from head of the list to process, there is possibility that some socks are always not processed. Although there was a strategy to cycle the pending_recv list to make sure we poll things not in the same order. Such as a list: A B C D E F, if max_events is 3, then this strategy makes the list is rearranged to D E F A B C. But it will make this strategy not effective if using TAILQ_INSERT_HEAD(&group->pending_recv, sock...). Using TAILQ_INSERT_TAIL(&group->pending_recv, sock...) can fix it. Signed-off-by: Richael Zhuang Change-Id: I8429b8eee29a9f9f820ad291d1b65ce2c2be22ea Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11154 Tested-by: SPDK CI Jenkins Reviewed-by: Aleksey Marchuk Reviewed-by: Jim Harris Reviewed-by: Ben Walker Community-CI: Broadcom CI Community-CI: Mellanox Build Bot --- module/sock/uring/uring.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/module/sock/uring/uring.c b/module/sock/uring/uring.c index 80ac12db3..4d99d7490 100644 --- a/module/sock/uring/uring.c +++ b/module/sock/uring/uring.c @@ -1024,7 +1024,7 @@ sock_uring_group_reap(struct spdk_uring_sock_group_impl *group, int max, int max if (sock->base.cb_fn != NULL && sock->pending_recv == false) { sock->pending_recv = true; - TAILQ_INSERT_HEAD(&group->pending_recv, sock, link); + TAILQ_INSERT_TAIL(&group->pending_recv, sock, link); } } break;