nvme/perf: Fix the hang issue of nvme perf.

When using nvme perf program to test against NVME-oF target,
the nvme perf program will hang if we kill the NVMe-oF target.

For example, if we run the following command:

1 On the target side, start a SPDK NVMe-oF target;

2 On the initiator side, we run:
./build/examples/perf -r 'trtype:rdma adrfam:IPv4 traddr:192.168.7.55 trsvcid:4420' -q 128
-o 4096 -w randwrite -t 100

3 Then we kill the NVMe-oF target on the target, the nvme perf program
will hang.

For NVMe perf program, I think that we should check it in

Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: Ia864394acdb6e705484dd0db6f015b567eb527a7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3774
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

examples/nvme/perf/perf.c

@@ -243,6 +243,8 @@ static bool g_header_digest;
 static bool g_data_digest;
 static bool g_no_shn_notification;
 static bool g_mix_specified;
+/* The flag is used to exit the program while keep alive fails on the transport */
+static bool g_exit;
 /* Default to 10 seconds for the keep alive value. This value is arbitrary. */
 static uint32_t g_keep_alive_timeout_in_ms = 10000;
 
@@ -1253,7 +1255,7 @@ work_fn(void *arg)
 		ns_ctx = ns_ctx->next;
 	}
 
-	while (1) {
+	while (spdk_likely(!g_exit)) {
 		/*
 		 * Check for completed I/O for each controller. A new
 		 * I/O will be submitted in the io_complete callback
@@ -2176,6 +2178,7 @@ nvme_poll_ctrlrs(void *arg)
 {
 	struct ctrlr_entry *entry;
 	int oldstate;
+	int rc;
 
 	spdk_unaffinitize_thread();
 
@@ -2185,7 +2188,10 @@ nvme_poll_ctrlrs(void *arg)
 		entry = g_controllers;
 		while (entry) {
 			if (entry->trtype != SPDK_NVME_TRANSPORT_PCIE) {
-				spdk_nvme_ctrlr_process_admin_completions(entry->ctrlr);
+				rc = spdk_nvme_ctrlr_process_admin_completions(entry->ctrlr);
+				if (spdk_unlikely(rc < 0 && !g_exit)) {
+					g_exit = true;
+				}
 			}
 			entry = entry->next;
 		}