From 59a1fbe937e62c589ea272867867d479e90d4151 Mon Sep 17 00:00:00 2001 From: Jim Harris Date: Wed, 22 May 2019 15:37:06 -0700 Subject: [PATCH] iscsi: fix hang on high QD large reads We throttle the number of data_in operations per connection. Currently after a read is completed, we try to send more data_in operations since one has just been completed. But we are trying to send more too early. The data_in_cnt doesn't actually get decremented until after the PDU is written on the socket. So this results in a case where data_in_cnt == 64, and all 64 read operations complete before any of those 64 are actually transmitted onto the TCP socket. There are no more read operations waiting, so we won't try to handle the data_in list again, and if none of these 64 resulted in a SCSI command completing, then the initiator may not send us any more read I/O which would have also kicked the data_in list. So the solution is to kick the data_in list after the PDU has been written - not after a read I/O is completed back from the SCSI layer. Signed-off-by: Jim Harris Change-Id: Ia01cf96e8eb6e08ddcaaeff449386e78de7c5bc5 Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455454 Tested-by: SPDK CI Jenkins Reviewed-by: Ben Walker Reviewed-by: Shuhei Matsumoto Reviewed-by: GangCao Reviewed-by: Changpeng Liu --- lib/iscsi/conn.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/lib/iscsi/conn.c b/lib/iscsi/conn.c index ae27c159c..6c1d8957e 100644 --- a/lib/iscsi/conn.c +++ b/lib/iscsi/conn.c @@ -314,6 +314,7 @@ spdk_iscsi_conn_free_pdu(struct spdk_iscsi_conn *conn, struct spdk_iscsi_pdu *pd conn->data_in_cnt--; spdk_iscsi_task_put(spdk_iscsi_task_get_primary(pdu->task)); } + spdk_iscsi_conn_handle_queued_datain_tasks(conn); } } else if (pdu->bhs.opcode == ISCSI_OP_SCSI_RSP && pdu->task->scsi.status != SPDK_SCSI_STATUS_GOOD) { @@ -1012,8 +1013,6 @@ process_read_task_completion(struct spdk_iscsi_conn *conn, spdk_iscsi_task_put(task); } process_completed_read_subtask_list(conn, primary); - - spdk_iscsi_conn_handle_queued_datain_tasks(conn); } void