ivampiresp/Spdk
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

NVMe-oF Target: Add FC transport.

Browse Source 
- New files and updates to existing SPDK files to add the NVMf-FC transport.
  - Depends on an existing low level driver library. This driver is not part of SPDK repository.
  - Makefile updates to build FC transport (using CONFIG_FC)
  - Update configure script for FC build.
  - New FC unit test for FC-LS commands.
  - Update unittest.sh to run FC unit test (when built).

Signed-off-by: John Barnard <john.barnard@broadcom.com>
Signed-off-by: Anil Veerabhadrappa <anil.veerabhadrappa@broadcom.com>
Change-Id: If31d4d25feab76c2dbe90a7faf71d465c2c3a354
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/450077
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This commit is contained in:
Anil Veerabhadrappa 2019-04-22 18:28:02 +09:00 committed by Ben Walker
parent e10fc6ea00
commit ed56a3d482
24 changed files with 7944 additions and 234 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
CHANGELOG.md
View File

@ -22,6 +22,12 @@ and `spdk_dix_remap_ref_tag` have been added to remap DIF reference tag.
New APIs `spdk_dif_update_crc32c` and `spdk_dif_update_crc32c_stream` have been New APIs `spdk_dif_update_crc32c` and `spdk_dif_update_crc32c_stream` have been
added to compute CRC-32C checksum for extended LBA payload. added to compute CRC-32C checksum for extended LBA payload.
### NVME-oF Target (FC)
New Fibre Channel transport for NVMe over Fabrics target. Requires an FC HBA to use.
Also, requires a Fibre Channel HBA low level driver (lld) library. The driver library
and API header file path can be provided as an argument to "--with-fc".
### NVMe-oF Target ### NVMe-oF Target
Persistent reservation emulation has been added to the NVMe-oF target. Persistent reservation Persistent reservation emulation has been added to the NVMe-oF target. Persistent reservation

5
CONFIG
View File

@ -94,6 +94,11 @@ CONFIG_FIO_SOURCE_DIR=/usr/src/fio
CONFIG_RDMA=n CONFIG_RDMA=n
CONFIG_RDMA_SEND_WITH_INVAL=n CONFIG_RDMA_SEND_WITH_INVAL=n
# Enable FC support for the NVMf target.
# Requires FC low level driver (from FC vendor)
CONFIG_FC=n
CONFIG_FC_PATH=
# Build Ceph RBD support in bdev modules # Build Ceph RBD support in bdev modules
# Requires librbd development libraries # Requires librbd development libraries
CONFIG_RBD=n CONFIG_RBD=n

7
app/nvmf_tgt/Makefile
View File

@ -48,4 +48,11 @@ ifeq ($(OS),Linux)
SPDK_LIB_LIST += event_nbd nbd SPDK_LIB_LIST += event_nbd nbd
endif endif
ifeq ($(CONFIG_FC),y)
ifneq ($(strip $(CONFIG_FC_PATH)),)
SYS_LIBS += -L$(CONFIG_FC_PATH)
endif
SYS_LIBS += -lufc
endif
include $(SPDK_ROOT_DIR)/mk/spdk.app.mk include $(SPDK_ROOT_DIR)/mk/spdk.app.mk

7
app/spdk_tgt/Makefile
View File

@ -58,6 +58,13 @@ ifeq ($(OS),Linux)
SPDK_LIB_LIST += event_nbd nbd SPDK_LIB_LIST += event_nbd nbd
endif endif
ifeq ($(CONFIG_FC),y)
ifneq ($(strip $(CONFIG_FC_PATH)),)
SYS_LIBS += -L$(CONFIG_FC_PATH)
endif
SYS_LIBS += -lufc
endif
include $(SPDK_ROOT_DIR)/mk/spdk.app.mk include $(SPDK_ROOT_DIR)/mk/spdk.app.mk
install: $(APP) install: $(APP)

25
configure vendored
View File

@ -69,6 +69,10 @@ function usage()
echo " No path required." echo " No path required."
echo " rdma Build RDMA transport for NVMf target and initiator." echo " rdma Build RDMA transport for NVMf target and initiator."
echo " No path required." echo " No path required."
echo " fc Build FC transport for NVMf target."
echo " If an argument is provided, it is considered a directory containing"
echo " libufc.a and fc_lld.h. Otherwise the regular system paths will"
echo " be searched."
echo " shared Build spdk shared libraries." echo " shared Build spdk shared libraries."
echo " No path required." echo " No path required."
echo " iscsi-initiator Build with iscsi bdev module." echo " iscsi-initiator Build with iscsi bdev module."
@ -218,6 +222,18 @@ for i in "$@"; do
--without-rdma) --without-rdma)
CONFIG[RDMA]=n CONFIG[RDMA]=n
;; ;;
--with-fc=*)
CONFIG[FC]=y
CONFIG[FC_PATH]=$(readlink -f ${i#*=})
;;
--with-fc)
CONFIG[FC]=y
CONFIG[FC_PATH]=
;;
--without-fc)
CONFIG[FC]=n
CONFIG[FC_PATH]=
;;
--with-shared) --with-shared)
CONFIG[SHARED]=y CONFIG[SHARED]=y
;; ;;
@ -499,6 +515,15 @@ than or equal to 4.14 will see significantly reduced performance.
fi fi
fi fi
if [[ "${CONFIG[FC]}" = "y" ]]; then
if [[ -n "${CONFIG[FC_PATH]}" ]]; then
if [ ! -d "${CONFIG[FC_PATH]}" ]; then
echo "${CONFIG[FC_PATH]}: directory not found"
exit 1
fi
fi
fi
if [[ "${CONFIG[ISAL]}" = "y" ]] || [[ "${CONFIG[CRYPTO]}" = "y" ]]; then if [[ "${CONFIG[ISAL]}" = "y" ]] || [[ "${CONFIG[CRYPTO]}" = "y" ]]; then
ver=$(nasm -v | awk '{print $3}' | sed 's/[^0-9]*//g') ver=$(nasm -v | awk '{print $3}' | sed 's/[^0-9]*//g')
if [[ "${ver:0:1}" -le "2" ]] && [[ "${ver:0:3}" -le "213" ]] && [[ "${ver:0:5}" -lt "21303" ]]; then if [[ "${ver:0:1}" -le "2" ]] && [[ "${ver:0:3}" -le "213" ]] && [[ "${ver:0:5}" -lt "21303" ]]; then

25
doc/nvmf.md
View File

@ -125,6 +125,31 @@ Using .ini style configuration files for configuration of the NVMe-oF target is
be replaced with JSON based RPCs. .ini style configuration files can be converted to json format by way be replaced with JSON based RPCs. .ini style configuration files can be converted to json format by way
of the new script `scripts/config_converter.py`. of the new script `scripts/config_converter.py`.
## FC transport support {#nvmf_fc_transport}
To build nvmf_tgt with the FC transport, there is an additional FC LLD (Low Level Driver) code dependency.
Please contact your FC vendor for instructions to obtain FC driver module.
### Broadcom FC LLD code
FC LLD driver for Broadcom FC NVMe capable adapters can be obtained from,
https://github.com/ecdufcdrvr/bcmufctdrvr.
### Fetch FC LLD module and then build SPDK with FC enabled:
After cloning SPDK repo and initialize submodules, FC LLD library is built which then can be linked with
the fc transport.
~~~{.sh}
git clone https://github.com/spdk/spdk spdk
git clone https://github.com/ecdufcdrvr/bcmufctdrvr fc
cd spdk
git submodule update --init
cd ../fc
make DPDK_DIR=../spdk/dpdk/build SPDK_DIR=../spdk
cd ../spdk
./configure --with-fc=../fc/build
make
~~~
### Using RPCs {#nvmf_config_rpc} ### Using RPCs {#nvmf_config_rpc}
Start the nvmf_tgt application with elevated privileges. Once the target is started, Start the nvmf_tgt application with elevated privileges. Once the target is started,

16
etc/spdk/nvmf.conf.in
View File

@ -146,6 +146,22 @@
# Set whether to use the C2H Success optimization, only used for TCP transport. # Set whether to use the C2H Success optimization, only used for TCP transport.
# C2HSuccess true # C2HSuccess true
# Define FC transport
#[Transport]
# Set FC transport type.
#Type FC
# Set the maximum number of submission and completion queues per session.
# Setting this to '8', for example, allows for 8 submission and 8 completion queues
# per session.
#MaxQueuesPerSession 5
# Set the maximum number of outstanding I/O per queue.
#MaxQueueDepth 128
# Set the maximum I/O size. Must be a multiple of 4096.
#MaxIOSize 65536
[Nvme] [Nvme]
# NVMe Device Whitelist # NVMe Device Whitelist
# Users may specify which NVMe devices to claim by their transport id. # Users may specify which NVMe devices to claim by their transport id.

12
include/spdk/nvmf_fc_spec.h
View File

@ -1,7 +1,7 @@
/* /*
* BSD LICENSE * BSD LICENSE
* *
* Copyright (c) 2018 Broadcom. All Rights Reserved. * Copyright (c) 2018-2019 Broadcom. All Rights Reserved.
* The term "Broadcom" refers to Broadcom Inc. and/or its subsidiaries. * The term "Broadcom" refers to Broadcom Inc. and/or its subsidiaries.
* *
* Redistribution and use in source and binary forms, with or without * Redistribution and use in source and binary forms, with or without
@ -400,4 +400,14 @@ struct spdk_nvmf_fc_ls_rjt {
}; };
SPDK_STATIC_ASSERT(sizeof(struct spdk_nvmf_fc_ls_rjt) == 40, "size_mismatch"); SPDK_STATIC_ASSERT(sizeof(struct spdk_nvmf_fc_ls_rjt) == 40, "size_mismatch");
/*
* FC World Wide Name
*/
struct spdk_nvmf_fc_wwn {
union {
uint64_t wwn; /* World Wide Names consist of eight bytes */
uint8_t octets[sizeof(uint64_t)];
} u;
};
#endif #endif

8
lib/nvmf/Makefile
View File

@ -59,4 +59,12 @@ endif
endif endif
endif endif
ifeq ($(CONFIG_FC),y)
C_SRCS += fc.c fc_ls.c
CFLAGS += -I$(CURDIR)
ifneq ($(strip $(CONFIG_FC_PATH)),)
CFLAGS += -I$(CONFIG_FC_PATH)
endif
endif
include $(SPDK_ROOT_DIR)/mk/spdk.lib.mk include $(SPDK_ROOT_DIR)/mk/spdk.lib.mk

3986
lib/nvmf/fc.c Normal file
View File

File diff suppressed because it is too large Load Diff

1681
lib/nvmf/fc_ls.c Normal file
View File

File diff suppressed because it is too large Load Diff

601
lib/nvmf/nvmf_fc.h
View File

@ -1,7 +1,7 @@
/* /*
* BSD LICENSE * BSD LICENSE
* *
* Copyright (c) 2018 Broadcom. All Rights Reserved. * Copyright (c) 2018-2019 Broadcom. All Rights Reserved.
* The term "Broadcom" refers to Broadcom Inc. and/or its subsidiaries. * The term "Broadcom" refers to Broadcom Inc. and/or its subsidiaries.
* *
* Redistribution and use in source and binary forms, with or without * Redistribution and use in source and binary forms, with or without
@ -34,6 +34,7 @@
#ifndef __NVMF_FC_H__ #ifndef __NVMF_FC_H__
#define __NVMF_FC_H__ #define __NVMF_FC_H__
#include "spdk/nvme.h"
#include "spdk/nvmf.h" #include "spdk/nvmf.h"
#include "spdk/assert.h" #include "spdk/assert.h"
#include "spdk/nvme_spec.h" #include "spdk/nvme_spec.h"
@ -43,6 +44,16 @@
#include "nvmf_internal.h" #include "nvmf_internal.h"
#define SPDK_NVMF_FC_TR_ADDR_LEN 64 #define SPDK_NVMF_FC_TR_ADDR_LEN 64
#define NVMF_FC_INVALID_CONN_ID UINT64_MAX
#define SPDK_FC_HW_DUMP_REASON_STR_MAX_SIZE 256
#define SPDK_MAX_NUM_OF_FC_PORTS 32
#define SPDK_NVMF_PORT_ID_MAX_LEN 32
/*
* FC HWQP pointer
*/
typedef void *spdk_nvmf_fc_lld_hwqp_t;
/* /*
* FC HW port states. * FC HW port states.
@ -59,7 +70,7 @@ enum spdk_fc_hwqp_state {
}; };
/* /*
* NVMF BCM FC Object state * NVMF FC Object state
* Add all the generic states of the object here. * Add all the generic states of the object here.
* Specific object states can be added separately * Specific object states can be added separately
*/ */
@ -86,25 +97,11 @@ enum spdk_nvmf_fc_request_state {
SPDK_NVMF_FC_REQ_SUCCESS, SPDK_NVMF_FC_REQ_SUCCESS,
SPDK_NVMF_FC_REQ_FAILED, SPDK_NVMF_FC_REQ_FAILED,
SPDK_NVMF_FC_REQ_ABORTED, SPDK_NVMF_FC_REQ_ABORTED,
SPDK_NVMF_FC_REQ_BDEV_ABORTED,
SPDK_NVMF_FC_REQ_PENDING, SPDK_NVMF_FC_REQ_PENDING,
SPDK_NVMF_FC_REQ_MAX_STATE, SPDK_NVMF_FC_REQ_MAX_STATE,
}; };
/*
* FC HWQP pointer
*/
typedef void *spdk_nvmf_fc_lld_hwqp_t;
/*
* FC World Wide Name
*/
struct spdk_nvmf_fc_wwn {
union {
uint64_t wwn; /* World Wide Names consist of eight bytes */
uint8_t octets[sizeof(uint64_t)];
} u;
};
/* /*
* Generic DMA buffer descriptor * Generic DMA buffer descriptor
*/ */
@ -142,7 +139,7 @@ struct spdk_nvmf_fc_abts_ctx {
* NVME FC transport errors * NVME FC transport errors
*/ */
struct spdk_nvmf_fc_errors { struct spdk_nvmf_fc_errors {
uint32_t no_xri; uint32_t no_xchg;
uint32_t nport_invalid; uint32_t nport_invalid;
uint32_t unknown_frame; uint32_t unknown_frame;
uint32_t wqe_cmplt_err; uint32_t wqe_cmplt_err;
@ -154,9 +151,7 @@ struct spdk_nvmf_fc_errors {
uint32_t invalid_cq_type; uint32_t invalid_cq_type;
uint32_t invalid_cq_id; uint32_t invalid_cq_id;
uint32_t fc_req_buf_err; uint32_t fc_req_buf_err;
uint32_t aq_buf_alloc_err; uint32_t buf_alloc_err;
uint32_t write_buf_alloc_err;
uint32_t read_buf_alloc_err;
uint32_t unexpected_err; uint32_t unexpected_err;
uint32_t nvme_cmd_iu_err; uint32_t nvme_cmd_iu_err;
uint32_t nvme_cmd_xfer_err; uint32_t nvme_cmd_xfer_err;
@ -173,10 +168,11 @@ struct spdk_nvmf_fc_errors {
/* /*
* Send Single Request/Response Sequence. * Send Single Request/Response Sequence.
*/ */
struct spdk_nvmf_fc_send_srsr { struct spdk_nvmf_fc_srsr_bufs {
struct spdk_nvmf_fc_buffer_desc rqst; void *rqst;
struct spdk_nvmf_fc_buffer_desc rsp; size_t rqst_len;
struct spdk_nvmf_fc_buffer_desc sgl; /* Note: Len = (2 * bcm_sge_t) */ void *rsp;
size_t rsp_len;
uint16_t rpi; uint16_t rpi;
}; };
@ -210,6 +206,7 @@ struct spdk_nvmf_fc_nport {
*/ */
struct spdk_nvmf_fc_conn { struct spdk_nvmf_fc_conn {
struct spdk_nvmf_qpair qpair; struct spdk_nvmf_qpair qpair;
struct spdk_nvme_transport_id trid;
uint64_t conn_id; uint64_t conn_id;
struct spdk_nvmf_fc_hwqp *hwqp; struct spdk_nvmf_fc_hwqp *hwqp;
@ -229,12 +226,11 @@ struct spdk_nvmf_fc_conn {
/* number of read/write requests that are outstanding */ /* number of read/write requests that are outstanding */
uint16_t cur_fc_rw_depth; uint16_t cur_fc_rw_depth;
/* requests that are waiting to obtain xri/buffer */ /* requests that are waiting to obtain xchg/buffer */
TAILQ_HEAD(, spdk_nvmf_fc_request) pending_queue; TAILQ_HEAD(, spdk_nvmf_fc_request) pending_queue;
struct spdk_nvmf_fc_association *fc_assoc; struct spdk_nvmf_fc_association *fc_assoc;
/* additional FC info here - TBD */
uint16_t rpi; uint16_t rpi;
/* for association's connection list */ /* for association's connection list */
@ -245,67 +241,70 @@ struct spdk_nvmf_fc_conn {
/* for hwqp's connection list */ /* for hwqp's connection list */
TAILQ_ENTRY(spdk_nvmf_fc_conn) link; TAILQ_ENTRY(spdk_nvmf_fc_conn) link;
/* New QP create context. */
struct nvmf_fc_ls_op_ctx *create_opd;
}; };
/* /*
* Structure for maintaining the XRI's * Structure for maintaining the FC exchanges
*/ */
struct spdk_nvmf_fc_xri { struct spdk_nvmf_fc_xchg {
uint32_t xri; /* The actual xri value */ uint32_t xchg_id; /* The actual xchg identifier */
/* Internal */ /* Internal */
TAILQ_ENTRY(spdk_nvmf_fc_xri) link; TAILQ_ENTRY(spdk_nvmf_fc_xchg) link;
bool is_active; bool active;
bool aborted;
bool send_abts; /* Valid if is_aborted is set. */
}; };
struct spdk_nvmf_fc_poll_group; /*
* FC poll group structure
*/
struct spdk_nvmf_fc_poll_group {
struct spdk_nvmf_transport_poll_group tp_poll_group;
struct spdk_nvmf_poll_group *poll_group;
struct spdk_nvmf_tgt *nvmf_tgt;
struct spdk_nvmf_fc_transport *fc_transport;
uint32_t hwqp_count; /* number of hwqp's assigned to this pg */
TAILQ_HEAD(, spdk_nvmf_fc_hwqp) hwqp_list;
TAILQ_ENTRY(spdk_nvmf_fc_poll_group) link;
};
/* /*
* HWQP poller structure passed from Master thread * HWQP poller structure passed from Master thread
*/ */
struct spdk_nvmf_fc_hwqp { struct spdk_nvmf_fc_hwqp {
enum spdk_fc_hwqp_state state; /* queue state (for poller) */
uint32_t lcore_id; /* core hwqp is running on (for tracing purposes only) */ uint32_t lcore_id; /* core hwqp is running on (for tracing purposes only) */
struct spdk_thread *thread; /* thread hwqp is running on */ struct spdk_thread *thread; /* thread hwqp is running on */
uint32_t hwqp_id; /* A unique id (per physical port) for a hwqp */ uint32_t hwqp_id; /* A unique id (per physical port) for a hwqp */
uint32_t rq_size; /* receive queue size */ uint32_t rq_size; /* receive queue size */
spdk_nvmf_fc_lld_hwqp_t queues; /* vendor HW queue set */ spdk_nvmf_fc_lld_hwqp_t queues; /* vendor HW queue set */
struct spdk_nvmf_fc_port *fc_port; /* HW port structure for these queues */ struct spdk_nvmf_fc_port *fc_port; /* HW port structure for these queues */
struct spdk_nvmf_fc_poll_group *poll_group; struct spdk_nvmf_fc_poll_group *fc_poll_group;
void *context; /* Vendor Context */
/* qpair (fc_connection) list */
TAILQ_HEAD(, spdk_nvmf_fc_conn) connection_list; TAILQ_HEAD(, spdk_nvmf_fc_conn) connection_list;
uint32_t num_conns; /* number of connections to queue */ uint32_t num_conns; /* number of connections to queue */
uint16_t cid_cnt; /* used to generate unique conn. id for RQ */
uint32_t free_q_slots; /* free q slots available for connections */
enum spdk_fc_hwqp_state state; /* Poller state (e.g. online, offline) */
/* Internal */ struct spdk_nvmf_fc_request *fc_reqs_buf;
struct spdk_mempool *fc_request_pool; TAILQ_HEAD(, spdk_nvmf_fc_request) free_reqs;
TAILQ_HEAD(, spdk_nvmf_fc_request) in_use_reqs; TAILQ_HEAD(, spdk_nvmf_fc_request) in_use_reqs;
TAILQ_HEAD(, spdk_nvmf_fc_xri) pending_xri_list;
struct spdk_nvmf_fc_errors counters; struct spdk_nvmf_fc_errors counters;
uint32_t send_frame_xri;
uint8_t send_frame_seqid;
/* Pending LS request waiting for XRI. */ /* Pending LS request waiting for FC resource */
TAILQ_HEAD(, spdk_nvmf_fc_ls_rqst) ls_pending_queue; TAILQ_HEAD(, spdk_nvmf_fc_ls_rqst) ls_pending_queue;
/* Sync req list */ /* Sync req list */
TAILQ_HEAD(, spdk_nvmf_fc_poller_api_queue_sync_args) sync_cbs; TAILQ_HEAD(, spdk_nvmf_fc_poller_api_queue_sync_args) sync_cbs;
TAILQ_ENTRY(spdk_nvmf_fc_hwqp) link; TAILQ_ENTRY(spdk_nvmf_fc_hwqp) link;
};
struct spdk_nvmf_fc_ls_rsrc_pool { void *context; /* Vendor specific context data */
void *assocs_mptr;
uint32_t assocs_count;
TAILQ_HEAD(, spdk_nvmf_fc_association) assoc_free_list;
void *conns_mptr;
uint32_t conns_count;
TAILQ_HEAD(, spdk_nvmf_fc_conn) fc_conn_free_list;
}; };
/* /*
@ -314,11 +313,10 @@ struct spdk_nvmf_fc_ls_rsrc_pool {
struct spdk_nvmf_fc_port { struct spdk_nvmf_fc_port {
uint8_t port_hdl; uint8_t port_hdl;
enum spdk_fc_port_state hw_port_status; enum spdk_fc_port_state hw_port_status;
uint32_t xri_base;
uint32_t xri_count;
uint16_t fcp_rq_id; uint16_t fcp_rq_id;
struct spdk_ring *xri_ring;
struct spdk_nvmf_fc_hwqp ls_queue; struct spdk_nvmf_fc_hwqp ls_queue;
new_qpair_fn new_qp_cb;
uint32_t num_io_queues; uint32_t num_io_queues;
struct spdk_nvmf_fc_hwqp *io_queues; struct spdk_nvmf_fc_hwqp *io_queues;
/* /*
@ -328,8 +326,7 @@ struct spdk_nvmf_fc_port {
int num_nports; int num_nports;
TAILQ_ENTRY(spdk_nvmf_fc_port) link; TAILQ_ENTRY(spdk_nvmf_fc_port) link;
struct spdk_nvmf_fc_ls_rsrc_pool ls_rsrc_pool; struct spdk_mempool *io_resource_pool; /* Pools to store bdev_io's for this port */
struct spdk_mempool *io_rsrc_pool; /* Pools to store bdev_io's for this port */
void *port_ctx; void *port_ctx;
}; };
@ -342,7 +339,7 @@ struct spdk_nvmf_fc_request {
uint32_t poller_lcore; /* for tracing purposes only */ uint32_t poller_lcore; /* for tracing purposes only */
struct spdk_thread *poller_thread; struct spdk_thread *poller_thread;
uint16_t buf_index; uint16_t buf_index;
struct spdk_nvmf_fc_xri *xri; struct spdk_nvmf_fc_xchg *xchg;
uint16_t oxid; uint16_t oxid;
uint16_t rpi; uint16_t rpi;
struct spdk_nvmf_fc_conn *fc_conn; struct spdk_nvmf_fc_conn *fc_conn;
@ -353,6 +350,8 @@ struct spdk_nvmf_fc_request {
uint32_t magic; uint32_t magic;
uint32_t s_id; uint32_t s_id;
uint32_t d_id; uint32_t d_id;
void *buffers[SPDK_NVMF_MAX_SGL_ENTRIES];
bool data_from_pool;
TAILQ_ENTRY(spdk_nvmf_fc_request) link; TAILQ_ENTRY(spdk_nvmf_fc_request) link;
TAILQ_ENTRY(spdk_nvmf_fc_request) pending_link; TAILQ_ENTRY(spdk_nvmf_fc_request) pending_link;
TAILQ_HEAD(, spdk_nvmf_fc_caller_ctx) abort_cbs; TAILQ_HEAD(, spdk_nvmf_fc_caller_ctx) abort_cbs;
@ -361,6 +360,7 @@ struct spdk_nvmf_fc_request {
SPDK_STATIC_ASSERT(!offsetof(struct spdk_nvmf_fc_request, req), SPDK_STATIC_ASSERT(!offsetof(struct spdk_nvmf_fc_request, req),
"FC request and NVMF request address don't match."); "FC request and NVMF request address don't match.");
/* /*
* NVMF FC Association * NVMF FC Association
*/ */
@ -370,7 +370,6 @@ struct spdk_nvmf_fc_association {
struct spdk_nvmf_fc_nport *tgtport; struct spdk_nvmf_fc_nport *tgtport;
struct spdk_nvmf_fc_remote_port_info *rport; struct spdk_nvmf_fc_remote_port_info *rport;
struct spdk_nvmf_subsystem *subsystem; struct spdk_nvmf_subsystem *subsystem;
struct spdk_nvmf_host *host;
enum spdk_nvmf_fc_object_state assoc_state; enum spdk_nvmf_fc_object_state assoc_state;
char host_id[FCNVME_ASSOC_HOSTID_LEN]; char host_id[FCNVME_ASSOC_HOSTID_LEN];
@ -392,8 +391,8 @@ struct spdk_nvmf_fc_association {
void *ls_del_op_ctx; /* delete assoc. callback list */ void *ls_del_op_ctx; /* delete assoc. callback list */
/* req/resp buffers used to send disconnect to initiator */ /* disconnect cmd buffers (sent to initiator) */
struct spdk_nvmf_fc_send_srsr snd_disconn_bufs; struct spdk_nvmf_fc_srsr_bufs *snd_disconn_bufs;
}; };
/* /*
@ -430,10 +429,13 @@ enum spdk_nvmf_fc_poller_api {
SPDK_NVMF_FC_POLLER_API_QUIESCE_QUEUE, SPDK_NVMF_FC_POLLER_API_QUIESCE_QUEUE,
SPDK_NVMF_FC_POLLER_API_ACTIVATE_QUEUE, SPDK_NVMF_FC_POLLER_API_ACTIVATE_QUEUE,
SPDK_NVMF_FC_POLLER_API_ABTS_RECEIVED, SPDK_NVMF_FC_POLLER_API_ABTS_RECEIVED,
SPDK_NVMF_FC_POLLER_API_REQ_ABORT_COMPLETE,
SPDK_NVMF_FC_POLLER_API_ADAPTER_EVENT, SPDK_NVMF_FC_POLLER_API_ADAPTER_EVENT,
SPDK_NVMF_FC_POLLER_API_AEN, SPDK_NVMF_FC_POLLER_API_AEN,
SPDK_NVMF_FC_POLLER_API_QUEUE_SYNC, SPDK_NVMF_FC_POLLER_API_QUEUE_SYNC,
SPDK_NVMF_FC_POLLER_API_QUEUE_SYNC_DONE, SPDK_NVMF_FC_POLLER_API_QUEUE_SYNC_DONE,
SPDK_NVMF_FC_POLLER_API_ADD_HWQP,
SPDK_NVMF_FC_POLLER_API_REMOVE_HWQP,
}; };
/* /*
@ -445,6 +447,7 @@ typedef void (*spdk_nvmf_fc_poller_api_cb)(void *cb_data, enum spdk_nvmf_fc_poll
* Poller API callback data * Poller API callback data
*/ */
struct spdk_nvmf_fc_poller_api_cb_info { struct spdk_nvmf_fc_poller_api_cb_info {
struct spdk_thread *cb_thread;
spdk_nvmf_fc_poller_api_cb cb_func; spdk_nvmf_fc_poller_api_cb cb_func;
void *cb_data; void *cb_data;
enum spdk_nvmf_fc_poller_api_ret ret; enum spdk_nvmf_fc_poller_api_ret ret;
@ -465,6 +468,7 @@ struct spdk_nvmf_fc_poller_api_del_connection_args {
bool send_abts; bool send_abts;
/* internal */ /* internal */
int fc_request_cnt; int fc_request_cnt;
bool backend_initiated;
}; };
struct spdk_nvmf_fc_poller_api_quiesce_queue_args { struct spdk_nvmf_fc_poller_api_quiesce_queue_args {
@ -486,6 +490,7 @@ struct spdk_nvmf_fc_poller_api_abts_recvd_args {
struct spdk_nvmf_fc_poller_api_queue_sync_done_args { struct spdk_nvmf_fc_poller_api_queue_sync_done_args {
struct spdk_nvmf_fc_hwqp *hwqp; struct spdk_nvmf_fc_hwqp *hwqp;
struct spdk_nvmf_fc_poller_api_cb_info cb_info;
uint64_t tag; uint64_t tag;
}; };
@ -498,7 +503,7 @@ struct spdk_nvmf_fc_ls_rqst {
uint32_t rqst_len; uint32_t rqst_len;
uint32_t rsp_len; uint32_t rsp_len;
uint32_t rpi; uint32_t rpi;
struct spdk_nvmf_fc_xri *xri; struct spdk_nvmf_fc_xchg *xchg;
uint16_t oxid; uint16_t oxid;
void *private_data; /* for LLD only (LS does not touch) */ void *private_data; /* for LLD only (LS does not touch) */
TAILQ_ENTRY(spdk_nvmf_fc_ls_rqst) ls_pending_link; TAILQ_ENTRY(spdk_nvmf_fc_ls_rqst) ls_pending_link;
@ -525,6 +530,48 @@ struct __attribute__((__packed__)) spdk_nvmf_fc_rq_buf_ls_request {
SPDK_STATIC_ASSERT(sizeof(struct spdk_nvmf_fc_rq_buf_ls_request) == SPDK_STATIC_ASSERT(sizeof(struct spdk_nvmf_fc_rq_buf_ls_request) ==
FCNVME_MAX_LS_BUFFER_SIZE, "LS RQ Buffer overflow"); FCNVME_MAX_LS_BUFFER_SIZE, "LS RQ Buffer overflow");
/* Poller API structures (arguments and callback data */
typedef void (*spdk_nvmf_fc_del_assoc_cb)(void *arg, uint32_t err);
struct spdk_nvmf_fc_ls_add_conn_api_data {
struct spdk_nvmf_fc_poller_api_add_connection_args args;
struct spdk_nvmf_fc_ls_rqst *ls_rqst;
struct spdk_nvmf_fc_association *assoc;
bool aq_conn; /* true if adding connection for new association */
};
/* Disconnect (connection) request functions */
struct spdk_nvmf_fc_ls_del_conn_api_data {
struct spdk_nvmf_fc_poller_api_del_connection_args args;
struct spdk_nvmf_fc_ls_rqst *ls_rqst;
struct spdk_nvmf_fc_association *assoc;
bool aq_conn; /* true if deleting AQ connection */
};
/* used by LS disconnect association cmd handling */
struct spdk_nvmf_fc_ls_disconn_assoc_api_data {
struct spdk_nvmf_fc_nport *tgtport;
struct spdk_nvmf_fc_ls_rqst *ls_rqst;
};
/* used by delete association call */
struct spdk_nvmf_fc_delete_assoc_api_data {
struct spdk_nvmf_fc_poller_api_del_connection_args args;
struct spdk_nvmf_fc_association *assoc;
bool from_ls_rqst; /* true = request came for LS */
spdk_nvmf_fc_del_assoc_cb del_assoc_cb;
void *del_assoc_cb_data;
};
struct nvmf_fc_ls_op_ctx {
union {
struct spdk_nvmf_fc_ls_add_conn_api_data add_conn;
struct spdk_nvmf_fc_ls_del_conn_api_data del_conn;
struct spdk_nvmf_fc_ls_disconn_assoc_api_data disconn_assoc;
struct spdk_nvmf_fc_delete_assoc_api_data del_assoc;
} u;
struct nvmf_fc_ls_op_ctx *next_op_ctx;
};
struct spdk_nvmf_fc_poller_api_queue_sync_args { struct spdk_nvmf_fc_poller_api_queue_sync_args {
uint64_t u_id; uint64_t u_id;
@ -535,6 +582,185 @@ struct spdk_nvmf_fc_poller_api_queue_sync_args {
TAILQ_ENTRY(spdk_nvmf_fc_poller_api_queue_sync_args) link; TAILQ_ENTRY(spdk_nvmf_fc_poller_api_queue_sync_args) link;
}; };
/**
* Following defines and structures are used to pass messages between master thread
* and FCT driver.
*/
enum spdk_fc_event {
SPDK_FC_HW_PORT_INIT,
SPDK_FC_HW_PORT_ONLINE,
SPDK_FC_HW_PORT_OFFLINE,
SPDK_FC_HW_PORT_RESET,
SPDK_FC_NPORT_CREATE,
SPDK_FC_NPORT_DELETE,
SPDK_FC_IT_ADD, /* PRLI */
SPDK_FC_IT_DELETE, /* PRLI */
SPDK_FC_ABTS_RECV,
SPDK_FC_LINK_BREAK,
SPDK_FC_HW_PORT_DUMP,
SPDK_FC_UNRECOVERABLE_ERR,
SPDK_FC_EVENT_MAX,
};
/**
* Arguments for to dump assoc id
*/
struct spdk_nvmf_fc_dump_assoc_id_args {
uint8_t pport_handle;
uint16_t nport_handle;
uint32_t assoc_id;
};
/**
* Arguments for HW port init event.
*/
struct spdk_nvmf_fc_hw_port_init_args {
uint32_t ls_queue_size;
spdk_nvmf_fc_lld_hwqp_t ls_queue;
uint32_t io_queue_size;
uint32_t io_queue_cnt;
spdk_nvmf_fc_lld_hwqp_t *io_queues;
void *cb_ctx;
void *port_ctx;
uint8_t port_handle;
uint8_t nvme_aq_index; /* io_queue used for nvme admin queue */
uint16_t fcp_rq_id; /* Base rq ID of SCSI queue */
};
/**
* Arguments for HW port link break event.
*/
struct spdk_nvmf_hw_port_link_break_args {
uint8_t port_handle;
void *cb_ctx;
};
/**
* Arguments for HW port online event.
*/
struct spdk_nvmf_fc_hw_port_online_args {
uint8_t port_handle;
void *cb_ctx;
};
/**
* Arguments for HW port offline event.
*/
struct spdk_nvmf_fc_hw_port_offline_args {
uint8_t port_handle;
void *cb_ctx;
};
/**
* Arguments for n-port add event.
*/
struct spdk_nvmf_fc_nport_create_args {
uint8_t port_handle;
uint16_t nport_handle;
struct spdk_uuid container_uuid; /* UUID of the nports container */
struct spdk_uuid nport_uuid; /* Unique UUID for the nport */
uint32_t d_id;
struct spdk_nvmf_fc_wwn fc_nodename;
struct spdk_nvmf_fc_wwn fc_portname;
uint32_t subsys_id; /* Subsystemid */
char port_id[SPDK_NVMF_PORT_ID_MAX_LEN];
void *cb_ctx;
};
/**
* Arguments for n-port delete event.
*/
struct spdk_nvmf_fc_nport_delete_args {
uint8_t port_handle;
uint32_t nport_handle;
uint32_t subsys_id; /* Subsystem id */
void *cb_ctx;
};
/**
* Arguments for I_T add event.
*/
struct spdk_nvmf_fc_hw_i_t_add_args {
uint8_t port_handle;
uint32_t nport_handle;
uint16_t itn_handle;
uint32_t rpi;
uint32_t s_id;
uint32_t initiator_prli_info;
uint32_t target_prli_info; /* populated by the SPDK master */
struct spdk_nvmf_fc_wwn fc_nodename;
struct spdk_nvmf_fc_wwn fc_portname;
void *cb_ctx;
};
/**
* Arguments for I_T delete event.
*/
struct spdk_nvmf_fc_hw_i_t_delete_args {
uint8_t port_handle;
uint32_t nport_handle;
uint16_t itn_handle; /* Only used by FC LLD driver; unused in SPDK */
uint32_t rpi;
uint32_t s_id;
void *cb_ctx;
};
/**
* Arguments for ABTS event.
*/
struct spdk_nvmf_fc_abts_args {
uint8_t port_handle;
uint32_t nport_handle;
uint32_t rpi;
uint16_t oxid, rxid;
void *cb_ctx;
};
/**
* Arguments for link break event.
*/
struct spdk_nvmf_fc_link_break_args {
uint8_t port_handle;
};
/**
* Arguments for port reset event.
*/
struct spdk_nvmf_fc_hw_port_reset_args {
uint8_t port_handle;
bool dump_queues;
char reason[SPDK_FC_HW_DUMP_REASON_STR_MAX_SIZE];
uint32_t **dump_buf;
void *cb_ctx;
};
/**
* Arguments for unrecoverable error event
*/
struct spdk_nvmf_fc_unrecoverable_error_event_args {
};
/**
* Callback function to the FCT driver.
*/
typedef void (*spdk_nvmf_fc_callback)(uint8_t port_handle,
enum spdk_fc_event event_type,
void *arg, int err);
/**
* Enqueue an FCT event to master thread
*
* \param event_type Type of the event.
* \param args Pointer to the argument structure.
* \param cb_func Callback function into fc driver.
*
* \return 0 on success, non-zero on failure.
*/
int
spdk_nvmf_fc_master_enqueue_event(enum spdk_fc_event event_type,
void *args,
spdk_nvmf_fc_callback cb_func);
/* /*
* dump info * dump info
*/ */
@ -578,79 +804,16 @@ struct spdk_nvmf_fc_caller_ctx {
}; };
/* /*
* Low level FC driver function table (functions provided by vendor FC device driver) * NVMF FC Exchange Info (for debug)
*/ */
struct spdk_nvmf_fc_ll_drvr_ops { struct spdk_nvmf_fc_xchg_info {
uint32_t xchg_base;
/* initialize the low level driver */ uint32_t xchg_total_count;
int (*lld_init)(void); uint32_t xchg_avail_count;
uint32_t send_frame_xchg_id;
/* low level driver finish */ uint8_t send_frame_seqid;
void (*lld_fini)(void);
/* initialize hw queues */
int (*init_q)(struct spdk_nvmf_fc_hwqp *hwqp);
void (*reinit_q)(spdk_nvmf_fc_lld_hwqp_t queues_prev,
spdk_nvmf_fc_lld_hwqp_t queues_curr);
/* initialize hw queue buffers */
int (*init_q_buffers)(struct spdk_nvmf_fc_hwqp *hwqp);
/* poll the hw queues for requests */
uint32_t (*poll_queue)(struct spdk_nvmf_fc_hwqp *hwqp);
/* receive data (for data-in requests) */
int (*recv_data)(struct spdk_nvmf_fc_request *fc_req);
/* send data (for data-out requests) */
int (*send_data)(struct spdk_nvmf_fc_request *fc_req);
/* release hw queust buffer */
void (*q_buffer_release)(struct spdk_nvmf_fc_hwqp *hwqp, uint16_t buff_idx);
/* transmist nvme response */
int (*xmt_rsp)(struct spdk_nvmf_fc_request *fc_req, uint8_t *ersp_buf, uint32_t ersp_len);
/* transmist LS response */
int (*xmt_ls_rsp)(struct spdk_nvmf_fc_nport *tgtport, struct spdk_nvmf_fc_ls_rqst *ls_rqst);
/* issue abts */
int (*issue_abort)(struct spdk_nvmf_fc_hwqp *hwqp, struct spdk_nvmf_fc_xri *xri,
bool send_abts, spdk_nvmf_fc_caller_cb cb, void *cb_args);
/* transmit abts response */
int (*xmt_bls_rsp)(struct spdk_nvmf_fc_hwqp *hwqp, uint16_t ox_id, uint16_t rx_id, uint16_t rpi,
bool rjt, uint8_t rjt_exp, spdk_nvmf_fc_caller_cb cb, void *cb_args);
/* transmit single request - single response */
int (*xmt_srsr_req)(struct spdk_nvmf_fc_hwqp *hwqp, struct spdk_nvmf_fc_send_srsr *srsr,
spdk_nvmf_fc_caller_cb cb, void *cb_args);
/* issue queue marker (abts processing) */
int (*issue_q_marker)(struct spdk_nvmf_fc_hwqp *hwqp, uint64_t u_id, uint16_t skip_rq);
/* assign a new connection to a hwqp (return connection ID) */
struct spdk_nvmf_fc_hwqp *(*assign_conn_to_hwqp)(
struct spdk_nvmf_fc_hwqp *queues, uint32_t num_queues,
uint64_t *conn_id, uint32_t sq_size, bool for_aq);
/* get the hwqp from the given connection id */
struct spdk_nvmf_fc_hwqp *(*get_hwqp_from_conn_id)(struct spdk_nvmf_fc_hwqp *hwqp,
uint32_t num_queues, uint64_t conn_id);
/* release connection ID (done with using it) */
void (*release_conn)(struct spdk_nvmf_fc_hwqp *hwqp, uint64_t conn_id, uint32_t sq_size);
/* dump all queue info into dump_info */
void (*dump_all_queues)(struct spdk_nvmf_fc_hwqp *ls_queues,
struct spdk_nvmf_fc_hwqp *io_queues,
uint32_t num_queues,
struct spdk_nvmf_fc_queue_dump_info *dump_info);
}; };
extern struct spdk_nvmf_fc_ll_drvr_ops spdk_nvmf_fc_lld_ops;
/* /*
* NVMF FC inline and function prototypes * NVMF FC inline and function prototypes
*/ */
@ -688,41 +851,39 @@ spdk_nvmf_fc_req_in_xfer(struct spdk_nvmf_fc_request *fc_req)
} }
} }
typedef void (*spdk_nvmf_fc_del_assoc_cb)(void *arg, uint32_t err); static inline void
int spdk_nvmf_fc_delete_association(struct spdk_nvmf_fc_nport *tgtport, spdk_nvmf_fc_create_trid(struct spdk_nvme_transport_id *trid, uint64_t n_wwn, uint64_t p_wwn)
uint64_t assoc_id, bool send_abts, {
spdk_nvmf_fc_del_assoc_cb del_assoc_cb, trid->trtype = (enum spdk_nvme_transport_type) SPDK_NVMF_TRTYPE_FC;
void *cb_data); trid->adrfam = SPDK_NVMF_ADRFAM_FC;
snprintf(trid->trsvcid, sizeof(trid->trsvcid), "none");
snprintf(trid->traddr, sizeof(trid->traddr), "nn-0x%lx:pn-0x%lx", n_wwn, p_wwn);
}
void spdk_nvmf_fc_ls_init(struct spdk_nvmf_fc_port *fc_port); void spdk_nvmf_fc_ls_init(struct spdk_nvmf_fc_port *fc_port);
void spdk_nvmf_fc_ls_fini(struct spdk_nvmf_fc_port *fc_port); void spdk_nvmf_fc_ls_fini(struct spdk_nvmf_fc_port *fc_port);
struct spdk_nvmf_fc_port *spdk_nvmf_fc_port_list_get(uint8_t port_hdl); void spdk_nvmf_fc_handle_ls_rqst(struct spdk_nvmf_fc_ls_rqst *ls_rqst);
void nvmf_fc_ls_add_conn_failure(
struct spdk_nvmf_fc_association *assoc,
struct spdk_nvmf_fc_ls_rqst *ls_rqst,
struct spdk_nvmf_fc_conn *fc_conn,
bool aq_conn);
int spdk_nvmf_fc_nport_set_state(struct spdk_nvmf_fc_nport *nport, int spdk_nvmf_fc_xmt_ls_rsp(struct spdk_nvmf_fc_nport *tgtport,
enum spdk_nvmf_fc_object_state state); struct spdk_nvmf_fc_ls_rqst *ls_rqst);
int spdk_nvmf_fc_assoc_set_state(struct spdk_nvmf_fc_association *assoc, void spdk_nvmf_fc_init_hwqp(struct spdk_nvmf_fc_port *fc_port, struct spdk_nvmf_fc_hwqp *hwqp);
enum spdk_nvmf_fc_object_state state);
bool spdk_nvmf_fc_nport_add_rem_port(struct spdk_nvmf_fc_nport *nport,
struct spdk_nvmf_fc_remote_port_info *rem_port);
bool spdk_nvmf_fc_nport_remove_rem_port(struct spdk_nvmf_fc_nport *nport,
struct spdk_nvmf_fc_remote_port_info *rem_port);
void spdk_nvmf_fc_init_poller_queues(struct spdk_nvmf_fc_hwqp *hwqp); void spdk_nvmf_fc_init_poller_queues(struct spdk_nvmf_fc_hwqp *hwqp);
void spdk_nvmf_fc_reinit_poller_queues(struct spdk_nvmf_fc_hwqp *hwqp, struct spdk_nvmf_fc_conn *spdk_nvmf_fc_hwqp_find_fc_conn(struct spdk_nvmf_fc_hwqp *hwqp,
void *queues_curr); uint64_t conn_id);
void spdk_nvmf_fc_init_poller(struct spdk_nvmf_fc_port *fc_port, void spdk_nvmf_fc_hwqp_reinit_poller_queues(struct spdk_nvmf_fc_hwqp *hwqp, void *queues_curr);
struct spdk_nvmf_fc_hwqp *hwqp);
void spdk_nvmf_fc_add_hwqp_to_poller(struct spdk_nvmf_fc_hwqp *hwqp, bool admin_q); struct spdk_nvmf_fc_port *spdk_nvmf_fc_port_lookup(uint8_t port_hdl);
void spdk_nvmf_fc_remove_hwqp_from_poller(struct spdk_nvmf_fc_hwqp *hwqp);
bool spdk_nvmf_fc_port_is_offline(struct spdk_nvmf_fc_port *fc_port); bool spdk_nvmf_fc_port_is_offline(struct spdk_nvmf_fc_port *fc_port);
@ -732,81 +893,72 @@ bool spdk_nvmf_fc_port_is_online(struct spdk_nvmf_fc_port *fc_port);
int spdk_nvmf_fc_port_set_online(struct spdk_nvmf_fc_port *fc_port); int spdk_nvmf_fc_port_set_online(struct spdk_nvmf_fc_port *fc_port);
int spdk_nvmf_fc_hwqp_port_set_online(struct spdk_nvmf_fc_hwqp *hwqp);
int spdk_nvmf_fc_hwqp_port_set_offline(struct spdk_nvmf_fc_hwqp *hwqp);
int spdk_nvmf_fc_rport_set_state(struct spdk_nvmf_fc_remote_port_info *rport, int spdk_nvmf_fc_rport_set_state(struct spdk_nvmf_fc_remote_port_info *rport,
enum spdk_nvmf_fc_object_state state); enum spdk_nvmf_fc_object_state state);
void spdk_nvmf_fc_port_list_add(struct spdk_nvmf_fc_port *fc_port); void spdk_nvmf_fc_port_add(struct spdk_nvmf_fc_port *fc_port);
struct spdk_nvmf_fc_nport *spdk_nvmf_fc_nport_get(uint8_t port_hdl, uint16_t nport_hdl);
int spdk_nvmf_fc_port_add_nport(struct spdk_nvmf_fc_port *fc_port, int spdk_nvmf_fc_port_add_nport(struct spdk_nvmf_fc_port *fc_port,
struct spdk_nvmf_fc_nport *nport); struct spdk_nvmf_fc_nport *nport);
uint32_t spdk_nvmf_fc_nport_get_association_count(struct spdk_nvmf_fc_nport *nport);
int spdk_nvmf_fc_port_remove_nport(struct spdk_nvmf_fc_port *fc_port, int spdk_nvmf_fc_port_remove_nport(struct spdk_nvmf_fc_port *fc_port,
struct spdk_nvmf_fc_nport *nport); struct spdk_nvmf_fc_nport *nport);
struct spdk_nvmf_fc_nport *spdk_nvmf_fc_nport_find(uint8_t port_hdl, uint16_t nport_hdl);
int spdk_nvmf_fc_nport_set_state(struct spdk_nvmf_fc_nport *nport,
enum spdk_nvmf_fc_object_state state);
bool spdk_nvmf_fc_nport_add_rem_port(struct spdk_nvmf_fc_nport *nport,
struct spdk_nvmf_fc_remote_port_info *rem_port);
bool spdk_nvmf_fc_nport_remove_rem_port(struct spdk_nvmf_fc_nport *nport,
struct spdk_nvmf_fc_remote_port_info *rem_port);
bool spdk_nvmf_fc_nport_has_no_rport(struct spdk_nvmf_fc_nport *nport);
int spdk_nvmf_fc_assoc_set_state(struct spdk_nvmf_fc_association *assoc,
enum spdk_nvmf_fc_object_state state);
int spdk_nvmf_fc_delete_association(struct spdk_nvmf_fc_nport *tgtport,
uint64_t assoc_id, bool send_abts, bool backend_initiated,
spdk_nvmf_fc_del_assoc_cb del_assoc_cb,
void *cb_data);
bool spdk_nvmf_ctrlr_is_on_nport(uint8_t port_hdl, uint16_t nport_hdl,
struct spdk_nvmf_ctrlr *ctrlr);
void spdk_nvmf_fc_assign_queue_to_master_thread(struct spdk_nvmf_fc_hwqp *hwqp);
void spdk_nvmf_fc_poll_group_add_hwqp(struct spdk_nvmf_fc_hwqp *hwqp);
void spdk_nvmf_fc_poll_group_remove_hwqp(struct spdk_nvmf_fc_hwqp *hwqp);
int spdk_nvmf_fc_hwqp_set_online(struct spdk_nvmf_fc_hwqp *hwqp);
int spdk_nvmf_fc_hwqp_set_offline(struct spdk_nvmf_fc_hwqp *hwqp);
uint32_t spdk_nvmf_fc_get_prli_service_params(void); uint32_t spdk_nvmf_fc_get_prli_service_params(void);
bool spdk_nvmf_fc_nport_is_rport_empty(struct spdk_nvmf_fc_nport *nport); void spdk_nvmf_fc_handle_abts_frame(struct spdk_nvmf_fc_nport *nport, uint16_t rpi, uint16_t oxid,
void spdk_nvmf_fc_handle_abts_frame(struct spdk_nvmf_fc_nport *nport,
uint16_t rpi, uint16_t oxid,
uint16_t rxid); uint16_t rxid);
void spdk_nvmf_fc_request_abort(struct spdk_nvmf_fc_request *fc_req, bool send_abts,
spdk_nvmf_fc_caller_cb cb, void *cb_args);
int spdk_nvmf_fc_xmt_srsr_req(struct spdk_nvmf_fc_hwqp *hwqp,
struct spdk_nvmf_fc_srsr_bufs *srsr_bufs,
spdk_nvmf_fc_caller_cb cb, void *cb_args);
struct spdk_nvmf_tgt *spdk_nvmf_fc_get_tgt(void);
void spdk_nvmf_fc_dump_all_queues(struct spdk_nvmf_fc_port *fc_port, void spdk_nvmf_fc_dump_all_queues(struct spdk_nvmf_fc_port *fc_port,
struct spdk_nvmf_fc_queue_dump_info *dump_info); struct spdk_nvmf_fc_queue_dump_info *dump_info);
void spdk_nvmf_fc_handle_ls_rqst(struct spdk_nvmf_fc_ls_rqst *ls_rqst);
int spdk_nvmf_fc_xmt_ls_rsp(struct spdk_nvmf_fc_nport *tgtport,
struct spdk_nvmf_fc_ls_rqst *ls_rqst);
struct spdk_nvmf_fc_nport *spdk_nvmf_bcm_req_fc_nport_get(struct spdk_nvmf_request *req);
struct spdk_nvmf_fc_association *spdk_nvmf_fc_get_ctrlr_assoc(struct spdk_nvmf_ctrlr *ctrlr);
bool spdk_nvmf_fc_nport_is_association_empty(struct spdk_nvmf_fc_nport *nport);
int spdk_nvmf_fc_xmt_srsr_req(struct spdk_nvmf_fc_hwqp *hwqp,
struct spdk_nvmf_fc_send_srsr *srsr,
spdk_nvmf_fc_caller_cb cb, void *cb_args);
uint32_t spdk_nvmf_fc_get_num_nport_ctrlrs_in_subsystem(uint8_t port_hdl, uint16_t nport_hdl,
struct spdk_nvmf_subsystem *subsys);
bool spdk_nvmf_fc_is_spdk_ctrlr_on_nport(uint8_t port_hdl, uint16_t nport_hdl,
struct spdk_nvmf_ctrlr *ctrlr);
int spdk_nvmf_fc_get_ctrlr_init_traddr(char *traddr, struct spdk_nvmf_ctrlr *ctrlr);
uint32_t spdk_nvmf_fc_get_hwqp_id(struct spdk_nvmf_request *req);
void spdk_nvmf_fc_req_abort(struct spdk_nvmf_fc_request *fc_req,
bool send_abts, spdk_nvmf_fc_caller_cb cb,
void *cb_args);
int spdk_nvmf_fc_add_port_listen(void *arg1, void *arg2);
int spdk_nvmf_fc_remove_port_listen(void *arg1, void *arg2);
void spdk_nvmf_fc_subsys_connect_cb(void *cb_ctx,
struct spdk_nvmf_request *req);
void spdk_nvmf_fc_subsys_disconnect_cb(void *cb_ctx,
struct spdk_nvmf_qpair *qpair);
uint32_t spdk_nvmf_fc_get_master_lcore(void);
struct spdk_thread *spdk_nvmf_fc_get_master_thread(void); struct spdk_thread *spdk_nvmf_fc_get_master_thread(void);
/* /*
* These functions are used by low level FC driver * These functions are called by low level FC driver
*/ */
static inline struct spdk_nvmf_fc_conn * static inline struct spdk_nvmf_fc_conn *
@ -841,31 +993,26 @@ enum spdk_nvmf_fc_poller_api_ret spdk_nvmf_fc_poller_api_func(
enum spdk_nvmf_fc_poller_api api, enum spdk_nvmf_fc_poller_api api,
void *api_args); void *api_args);
int spdk_nvmf_fc_process_frame(struct spdk_nvmf_fc_hwqp *hwqp, uint32_t buff_idx, int spdk_nvmf_fc_hwqp_process_frame(struct spdk_nvmf_fc_hwqp *hwqp, uint32_t buff_idx,
struct spdk_nvmf_fc_frame_hdr *frame, struct spdk_nvmf_fc_frame_hdr *frame,
struct spdk_nvmf_fc_buffer_desc *buffer, uint32_t plen); struct spdk_nvmf_fc_buffer_desc *buffer, uint32_t plen);
void spdk_nvmf_fc_process_pending_req(struct spdk_nvmf_fc_hwqp *hwqp); void spdk_nvmf_fc_hwqp_process_pending_reqs(struct spdk_nvmf_fc_hwqp *hwqp);
void spdk_nvmf_fc_process_pending_ls_rqst(struct spdk_nvmf_fc_hwqp *hwqp); void spdk_nvmf_fc_hwqp_process_pending_ls_rqsts(struct spdk_nvmf_fc_hwqp *hwqp);
void spdk_nvmf_fc_req_set_state(struct spdk_nvmf_fc_request *fc_req, void spdk_nvmf_fc_request_set_state(struct spdk_nvmf_fc_request *fc_req,
enum spdk_nvmf_fc_request_state state); enum spdk_nvmf_fc_request_state state);
void spdk_nvmf_fc_free_req(struct spdk_nvmf_fc_request *fc_req); char *spdk_nvmf_fc_request_get_state_str(int state);
void spdk_nvmf_fc_req_abort_complete(void *arg1); void spdk_nvmf_fc_request_free(struct spdk_nvmf_fc_request *fc_req);
void spdk_nvmf_fc_request_abort_complete(void *arg1);
bool spdk_nvmf_fc_send_ersp_required(struct spdk_nvmf_fc_request *fc_req, bool spdk_nvmf_fc_send_ersp_required(struct spdk_nvmf_fc_request *fc_req,
uint32_t rsp_cnt, uint32_t xfer_len); uint32_t rsp_cnt, uint32_t xfer_len);
struct spdk_nvmf_fc_xri *spdk_nvmf_fc_get_xri(struct spdk_nvmf_fc_hwqp *hwqp);
int spdk_nvmf_fc_put_xri(struct spdk_nvmf_fc_hwqp *hwqp,
struct spdk_nvmf_fc_xri *xri);
void spdk_nvmf_fc_release_xri(struct spdk_nvmf_fc_hwqp *hwqp,
struct spdk_nvmf_fc_xri *xri, bool xb, bool abts);
int spdk_nvmf_fc_handle_rsp(struct spdk_nvmf_fc_request *req); int spdk_nvmf_fc_handle_rsp(struct spdk_nvmf_fc_request *req);
#endif #endif

5
lib/nvmf/transport.c
View File

@ -47,6 +47,9 @@ static const struct spdk_nvmf_transport_ops *const g_transport_ops[] = {
&spdk_nvmf_transport_rdma, &spdk_nvmf_transport_rdma,
#endif #endif
&spdk_nvmf_transport_tcp, &spdk_nvmf_transport_tcp,
#ifdef SPDK_CONFIG_FC
&spdk_nvmf_transport_fc,
#endif
}; };
#define NUM_TRANSPORTS (SPDK_COUNTOF(g_transport_ops)) #define NUM_TRANSPORTS (SPDK_COUNTOF(g_transport_ops))
@ -87,7 +90,7 @@ spdk_nvmf_transport_create(enum spdk_nvme_transport_type type,
ops = spdk_nvmf_get_transport_ops(type); ops = spdk_nvmf_get_transport_ops(type);
if (!ops) { if (!ops) {
SPDK_ERRLOG("Transport type %s unavailable.\n", SPDK_ERRLOG("Transport type '%s' unavailable.\n",
spdk_nvme_transport_id_trtype_str(type)); spdk_nvme_transport_id_trtype_str(type));
return NULL; return NULL;
} }

1
lib/nvmf/transport.h
View File

@ -225,5 +225,6 @@ bool spdk_nvmf_transport_opts_init(enum spdk_nvme_transport_type type,
extern const struct spdk_nvmf_transport_ops spdk_nvmf_transport_rdma; extern const struct spdk_nvmf_transport_ops spdk_nvmf_transport_rdma;
extern const struct spdk_nvmf_transport_ops spdk_nvmf_transport_tcp; extern const struct spdk_nvmf_transport_ops spdk_nvmf_transport_tcp;
extern const struct spdk_nvmf_transport_ops spdk_nvmf_transport_fc;
#endif /* SPDK_NVMF_TRANSPORT_H */ #endif /* SPDK_NVMF_TRANSPORT_H */

7
test/app/bdev_svc/Makefile
View File

@ -48,4 +48,11 @@ ifeq ($(OS),Linux)
SPDK_LIB_LIST += event_nbd nbd SPDK_LIB_LIST += event_nbd nbd
endif endif
ifeq ($(CONFIG_FC),y)
ifneq ($(strip $(CONFIG_FC_PATH)),)
SYS_LIBS += -L$(CONFIG_FC_PATH)
endif
SYS_LIBS += -lufc
endif
include $(SPDK_ROOT_DIR)/mk/spdk.app.mk include $(SPDK_ROOT_DIR)/mk/spdk.app.mk

2
test/common/lib/test_env.c
View File

@ -46,7 +46,7 @@ DEFINE_STUB(spdk_pci_ioat_get_driver, struct spdk_pci_driver *, (void), NULL)
DEFINE_STUB(spdk_pci_virtio_get_driver, struct spdk_pci_driver *, (void), NULL) DEFINE_STUB(spdk_pci_virtio_get_driver, struct spdk_pci_driver *, (void), NULL)
DEFINE_STUB(spdk_env_get_first_core, uint32_t, (void), 0); DEFINE_STUB(spdk_env_get_first_core, uint32_t, (void), 0);
DEFINE_STUB(spdk_env_get_next_core, uint32_t, (uint32_t prev_core), 0); DEFINE_STUB(spdk_env_get_next_core, uint32_t, (uint32_t prev_core), 0);
DEFINE_STUB(spdk_env_get_last_core, uint32_t, (void), 0); DEFINE_STUB(spdk_env_get_last_core, uint32_t, (void), 1);
DEFINE_STUB(spdk_env_get_current_core, uint32_t, (void), 0); DEFINE_STUB(spdk_env_get_current_core, uint32_t, (void), 0);
DEFINE_STUB(spdk_env_get_socket_id, uint32_t, (uint32_t core), 0); DEFINE_STUB(spdk_env_get_socket_id, uint32_t, (uint32_t core), 0);

2
test/unit/lib/nvmf/Makefile
View File

@ -38,6 +38,8 @@ DIRS-y = tcp.c ctrlr.c subsystem.c ctrlr_discovery.c ctrlr_bdev.c
DIRS-$(CONFIG_RDMA) += rdma.c DIRS-$(CONFIG_RDMA) += rdma.c
DIRS-$(CONFIG_FC) += fc.c fc_ls.c
.PHONY: all clean $(DIRS-y) .PHONY: all clean $(DIRS-y)
all: $(DIRS-y) all: $(DIRS-y)

1
test/unit/lib/nvmf/fc.c/.gitignore vendored Normal file
View File

@ -0,0 +1 @@
fc_ut

58
test/unit/lib/nvmf/fc.c/Makefile Normal file
View File

@ -0,0 +1,58 @@
#
# BSD LICENSE
#
# Copyright (c) 2018 Broadcom. All Rights Reserved.
# The term "Broadcom" refers to Broadcom Limited and/or its subsidiaries.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions
# are met:
#
# * Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# * Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in
# the documentation and/or other materials provided with the
# distribution.
# * Neither the name of Intel Corporation nor the names of its
# contributors may be used to endorse or promote products derived
# from this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#
SPDK_ROOT_DIR := $(abspath $(CURDIR)/../../../../../)
include $(SPDK_ROOT_DIR)/mk/config.mk
CFLAGS += -I$(SPDK_ROOT_DIR)/test/common/lib -I$(SPDK_ROOT_DIR)/lib \
-I$(SPDK_ROOT_DIR)/lib/nvmf
ifneq ($(strip $(CONFIG_FC_PATH)),)
CFLAGS += -I$(CONFIG_FC_PATH)
endif
TEST_FILE = fc_ut.c
include $(SPDK_ROOT_DIR)/mk/spdk.unittest.mk
# Disable clang warning: taking address of packed member of class or structure may result in an unaligned pointer value [-Werror,-Waddress-of-packed-member]
ifeq ($(CC),clang)
CLANG_VERSION := $(shell $(CC) -v 2>&1 | \
sed -n "s/.*version \([0-9]*\.[0-9]*\).*/\1/p")
CLANG_MAJOR_VERSION := $(shell echo $(CLANG_VERSION) | cut -f1 -d.)
ifeq ($(shell test $(CLANG_MAJOR_VERSION) -ge 4 && echo 1), 1)
CFLAGS += -Wno-address-of-packed-member
endif
endif

536
test/unit/lib/nvmf/fc.c/fc_ut.c Normal file
View File

@ -0,0 +1,536 @@
/*
* BSD LICENSE
*
* Copyright (c) 2018-2019 Broadcom. All Rights Reserved.
* The term "Broadcom" refers to Broadcom Inc. and/or its subsidiaries.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
*
* * Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in
* the documentation and/or other materials provided with the
* distribution.
* * Neither the name of Intel Corporation nor the names of its
* contributors may be used to endorse or promote products derived
* from this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
* "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
* LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
* A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
* OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
* SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
* LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
* DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
* THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/* NVMF FC Transport Unit Test */
#include "spdk/env.h"
#include "spdk_cunit.h"
#include "spdk/nvmf.h"
#include "spdk_internal/event.h"
#include "spdk/endian.h"
#include "spdk/trace.h"
#include "spdk_internal/log.h"
#include "ut_multithread.c"
#include "transport.h"
#include "nvmf_internal.h"
#include "nvmf_fc.h"
#include "json/json_util.c"
#include "json/json_write.c"
#include "nvmf/nvmf.c"
#include "nvmf/transport.c"
#include "nvmf/subsystem.c"
#include "nvmf/fc.c"
#include "nvmf/fc_ls.c"
/*
* SPDK Stuff
*/
#ifdef SPDK_CONFIG_RDMA
const struct spdk_nvmf_transport_ops spdk_nvmf_transport_rdma = {
.type = SPDK_NVME_TRANSPORT_RDMA,
.opts_init = NULL,
.create = NULL,
.destroy = NULL,
.listen = NULL,
.stop_listen = NULL,
.accept = NULL,
.listener_discover = NULL,
.poll_group_create = NULL,
.poll_group_destroy = NULL,
.poll_group_add = NULL,
.poll_group_poll = NULL,
.req_free = NULL,
.req_complete = NULL,
.qpair_fini = NULL,
.qpair_is_idle = NULL,
.qpair_get_peer_trid = NULL,
.qpair_get_local_trid = NULL,
.qpair_get_listen_trid = NULL,
};
#endif
const struct spdk_nvmf_transport_ops spdk_nvmf_transport_tcp = {
.type = SPDK_NVME_TRANSPORT_TCP,
};
struct spdk_trace_histories *g_trace_histories;
DEFINE_STUB_V(_spdk_trace_record, (uint64_t tsc, uint16_t tpoint_id, uint16_t poller_id,
uint32_t size, uint64_t object_id, uint64_t arg1));
DEFINE_STUB(spdk_env_get_core_count, uint32_t, (void), 4);
DEFINE_STUB(spdk_nvme_transport_id_compare, int,
(const struct spdk_nvme_transport_id *trid1,
const struct spdk_nvme_transport_id *trid2), 0);
DEFINE_STUB_V(spdk_trace_register_object, (uint8_t type, char id_prefix));
DEFINE_STUB_V(spdk_trace_register_description,
(const char *name, uint16_t tpoint_id, uint8_t owner_type,
uint8_t object_type, uint8_t new_object, uint8_t arg1_type,
const char *arg1_name));
DEFINE_STUB_V(spdk_trace_add_register_fn, (struct spdk_trace_register_fn *reg_fn));
DEFINE_STUB(spdk_bdev_get_name, const char *, (const struct spdk_bdev *bdev), "fc_ut_test");
DEFINE_STUB_V(spdk_nvmf_ctrlr_destruct, (struct spdk_nvmf_ctrlr *ctrlr));
DEFINE_STUB_V(spdk_nvmf_qpair_free_aer, (struct spdk_nvmf_qpair *qpair));
DEFINE_STUB(spdk_bdev_get_io_channel, struct spdk_io_channel *, (struct spdk_bdev_desc *desc),
NULL);
DEFINE_STUB_V(spdk_nvmf_request_exec, (struct spdk_nvmf_request *req));
DEFINE_STUB_V(spdk_nvmf_ctrlr_ns_changed, (struct spdk_nvmf_ctrlr *ctrlr, uint32_t nsid));
DEFINE_STUB(spdk_bdev_open, int, (struct spdk_bdev *bdev, bool write,
spdk_bdev_remove_cb_t remove_cb,
void *remove_ctx, struct spdk_bdev_desc **desc), 0);
DEFINE_STUB_V(spdk_bdev_close, (struct spdk_bdev_desc *desc));
DEFINE_STUB(spdk_bdev_module_claim_bdev, int,
(struct spdk_bdev *bdev, struct spdk_bdev_desc *desc,
struct spdk_bdev_module *module), 0);
DEFINE_STUB_V(spdk_bdev_module_release_bdev, (struct spdk_bdev *bdev));
DEFINE_STUB(spdk_bdev_get_block_size, uint32_t, (const struct spdk_bdev *bdev), 512);
const char *
spdk_nvme_transport_id_trtype_str(enum spdk_nvme_transport_type trtype)
{
switch (trtype) {
case SPDK_NVME_TRANSPORT_PCIE:
return "PCIe";
case SPDK_NVME_TRANSPORT_RDMA:
return "RDMA";
case SPDK_NVME_TRANSPORT_FC:
return "FC";
default:
return NULL;
}
}
const char *
spdk_nvme_transport_id_adrfam_str(enum spdk_nvmf_adrfam adrfam)
{
switch (adrfam) {
case SPDK_NVMF_ADRFAM_IPV4:
return "IPv4";
case SPDK_NVMF_ADRFAM_IPV6:
return "IPv6";
case SPDK_NVMF_ADRFAM_IB:
return "IB";
case SPDK_NVMF_ADRFAM_FC:
return "FC";
default:
return NULL;
}
}
const struct spdk_uuid *
spdk_bdev_get_uuid(const struct spdk_bdev *bdev)
{
return &bdev->uuid;
}
static bool g_lld_init_called = false;
int
nvmf_fc_lld_init(void)
{
g_lld_init_called = true;
return 0;
}
static bool g_lld_fini_called = false;
void
nvmf_fc_lld_fini(void)
{
g_lld_fini_called = true;
}
DEFINE_STUB_V(nvmf_fc_lld_start, (void));
DEFINE_STUB(nvmf_fc_init_q, int, (struct spdk_nvmf_fc_hwqp *hwqp), 0);
DEFINE_STUB_V(nvmf_fc_reinit_q, (void *queues_prev, void *queues_curr));
DEFINE_STUB(nvmf_fc_init_rqpair_buffers, int, (struct spdk_nvmf_fc_hwqp *hwqp), 0);
DEFINE_STUB(nvmf_fc_set_q_online_state, int, (struct spdk_nvmf_fc_hwqp *hwqp, bool online), 0);
DEFINE_STUB(nvmf_fc_put_xchg, int, (struct spdk_nvmf_fc_hwqp *hwqp, struct spdk_nvmf_fc_xchg *xri),
0);
DEFINE_STUB(nvmf_fc_recv_data, int, (struct spdk_nvmf_fc_request *fc_req), 0);
DEFINE_STUB(nvmf_fc_send_data, int, (struct spdk_nvmf_fc_request *fc_req), 0);
DEFINE_STUB_V(nvmf_fc_rqpair_buffer_release, (struct spdk_nvmf_fc_hwqp *hwqp, uint16_t buff_idx));
DEFINE_STUB(nvmf_fc_xmt_rsp, int, (struct spdk_nvmf_fc_request *fc_req, uint8_t *ersp_buf,
uint32_t ersp_len), 0);
DEFINE_STUB(nvmf_fc_xmt_ls_rsp, int, (struct spdk_nvmf_fc_nport *tgtport,
struct spdk_nvmf_fc_ls_rqst *ls_rqst), 0);
DEFINE_STUB(nvmf_fc_issue_abort, int, (struct spdk_nvmf_fc_hwqp *hwqp,
struct spdk_nvmf_fc_xchg *xri,
spdk_nvmf_fc_caller_cb cb, void *cb_args), 0);
DEFINE_STUB(nvmf_fc_xmt_bls_rsp, int, (struct spdk_nvmf_fc_hwqp *hwqp,
uint16_t ox_id, uint16_t rx_id,
uint16_t rpi, bool rjt, uint8_t rjt_exp,
spdk_nvmf_fc_caller_cb cb, void *cb_args), 0);
DEFINE_STUB(nvmf_fc_alloc_srsr_bufs, struct spdk_nvmf_fc_srsr_bufs *, (size_t rqst_len,
size_t rsp_len), NULL);
DEFINE_STUB_V(nvmf_fc_free_srsr_bufs, (struct spdk_nvmf_fc_srsr_bufs *srsr_bufs));
DEFINE_STUB(nvmf_fc_xmt_srsr_req, int, (struct spdk_nvmf_fc_hwqp *hwqp,
struct spdk_nvmf_fc_srsr_bufs *xmt_srsr_bufs,
spdk_nvmf_fc_caller_cb cb, void *cb_args), 0);
DEFINE_STUB(nvmf_fc_q_sync_available, bool, (void), true);
DEFINE_STUB(nvmf_fc_issue_q_sync, int, (struct spdk_nvmf_fc_hwqp *hwqp, uint64_t u_id,
uint16_t skip_rq), 0);
DEFINE_STUB(nvmf_fc_assign_conn_to_hwqp, bool, (struct spdk_nvmf_fc_hwqp *hwqp,
uint64_t *conn_id, uint32_t sq_size), true);
DEFINE_STUB(nvmf_fc_get_hwqp_from_conn_id, struct spdk_nvmf_fc_hwqp *,
(struct spdk_nvmf_fc_hwqp *queues,
uint32_t num_queues, uint64_t conn_id), NULL);
DEFINE_STUB_V(nvmf_fc_release_conn, (struct spdk_nvmf_fc_hwqp *hwqp, uint64_t conn_id,
uint32_t sq_size));
DEFINE_STUB_V(nvmf_fc_dump_all_queues, (struct spdk_nvmf_fc_hwqp *ls_queue,
struct spdk_nvmf_fc_hwqp *io_queues,
uint32_t num_io_queues,
struct spdk_nvmf_fc_queue_dump_info *dump_info));
DEFINE_STUB_V(nvmf_fc_get_xri_info, (struct spdk_nvmf_fc_hwqp *hwqp,
struct spdk_nvmf_fc_xchg_info *info));
DEFINE_STUB(nvmf_fc_get_rsvd_thread, struct spdk_thread *, (void), NULL);
uint32_t
nvmf_fc_process_queue(struct spdk_nvmf_fc_hwqp *hwqp)
{
hwqp->lcore_id++;
return 0; /* always return 0 or else it will poll forever */
}
struct spdk_nvmf_fc_xchg *
nvmf_fc_get_xri(struct spdk_nvmf_fc_hwqp *hwqp)
{
static struct spdk_nvmf_fc_xchg xchg;
xchg.xchg_id = 1;
return &xchg;
}
#define MAX_FC_UT_POLL_THREADS 8
static struct spdk_nvmf_poll_group *g_poll_groups[MAX_FC_UT_POLL_THREADS] = {0};
#define MAX_FC_UT_HWQPS MAX_FC_UT_POLL_THREADS
static struct spdk_nvmf_tgt *g_nvmf_tgt = NULL;
static struct spdk_nvmf_transport *g_nvmf_tprt = NULL;
uint8_t g_fc_port_handle = 0xff;
struct spdk_nvmf_fc_hwqp lld_q[MAX_FC_UT_HWQPS];
static void
_add_transport_done(void *arg, int status)
{
CU_ASSERT(status == 0);
}
static void
_add_transport_done_dup_err(void *arg, int status)
{
CU_ASSERT(status == -EEXIST);
}
static void
create_transport_test(void)
{
const struct spdk_nvmf_transport_ops *ops = NULL;
struct spdk_nvmf_transport_opts opts = { 0 };
allocate_threads(8);
set_thread(0);
g_nvmf_tgt = spdk_nvmf_tgt_create(2);
SPDK_CU_ASSERT_FATAL(g_nvmf_tgt != NULL);
ops = spdk_nvmf_get_transport_ops((enum spdk_nvme_transport_type) SPDK_NVMF_TRTYPE_FC);
SPDK_CU_ASSERT_FATAL(ops != NULL);
ops->opts_init(&opts);
g_lld_init_called = false;
g_nvmf_tprt = spdk_nvmf_transport_create((enum spdk_nvme_transport_type) SPDK_NVMF_TRTYPE_FC,
&opts);
SPDK_CU_ASSERT_FATAL(g_nvmf_tprt != NULL);
CU_ASSERT(g_lld_init_called == true);
CU_ASSERT(opts.max_queue_depth == g_nvmf_tprt->opts.max_queue_depth);
CU_ASSERT(opts.max_qpairs_per_ctrlr == g_nvmf_tprt->opts.max_qpairs_per_ctrlr);
CU_ASSERT(opts.in_capsule_data_size == g_nvmf_tprt->opts.in_capsule_data_size);
CU_ASSERT(opts.max_io_size == g_nvmf_tprt->opts.max_io_size);
CU_ASSERT(opts.io_unit_size == g_nvmf_tprt->opts.io_unit_size);
CU_ASSERT(opts.max_aq_depth == g_nvmf_tprt->opts.max_aq_depth);
set_thread(0);
spdk_nvmf_tgt_add_transport(g_nvmf_tgt, g_nvmf_tprt,
_add_transport_done, 0);
poll_thread(0);
/* Add transport again - should get error */
spdk_nvmf_tgt_add_transport(g_nvmf_tgt, g_nvmf_tprt,
_add_transport_done_dup_err, 0);
poll_thread(0);
/* create transport with bad args/options */
#ifndef SPDK_CONFIG_RDMA
CU_ASSERT(spdk_nvmf_transport_create(SPDK_NVMF_TRTYPE_RDMA, &opts) == NULL);
#endif
CU_ASSERT(spdk_nvmf_transport_create(998, &opts) == NULL);
opts.max_io_size = 1024 ^ 3;
CU_ASSERT(spdk_nvmf_transport_create((enum spdk_nvme_transport_type) SPDK_NVMF_TRTYPE_FC,
&opts) == NULL);
opts.max_io_size = 999;
opts.io_unit_size = 1024;
CU_ASSERT(spdk_nvmf_transport_create((enum spdk_nvme_transport_type) SPDK_NVMF_TRTYPE_FC,
&opts) == NULL);
}
static void
port_init_cb(uint8_t port_handle, enum spdk_fc_event event_type, void *arg, int err)
{
CU_ASSERT(err == 0);
CU_ASSERT(port_handle == 2);
g_fc_port_handle = port_handle;
}
static void
create_fc_port_test(void)
{
struct spdk_nvmf_fc_hw_port_init_args init_args = { 0 };
struct spdk_nvmf_fc_port *fc_port = NULL;
int err;
SPDK_CU_ASSERT_FATAL(g_nvmf_tprt != NULL);
init_args.port_handle = 2;
init_args.io_queue_cnt = spdk_min(MAX_FC_UT_HWQPS, spdk_env_get_core_count());
init_args.ls_queue_size = 100;
init_args.io_queue_size = 100;
init_args.io_queues = (void *)lld_q;
set_thread(0);
err = spdk_nvmf_fc_master_enqueue_event(SPDK_FC_HW_PORT_INIT, (void *)&init_args, port_init_cb);
CU_ASSERT(err == 0);
poll_thread(0);
fc_port = spdk_nvmf_fc_port_lookup(g_fc_port_handle);
CU_ASSERT(fc_port != NULL);
}
static void
online_fc_port_test(void)
{
struct spdk_nvmf_fc_port *fc_port;
struct spdk_nvmf_fc_hw_port_online_args args;
int err;
SPDK_CU_ASSERT_FATAL(g_nvmf_tprt != NULL);
fc_port = spdk_nvmf_fc_port_lookup(g_fc_port_handle);
SPDK_CU_ASSERT_FATAL(fc_port != NULL);
set_thread(0);
args.port_handle = g_fc_port_handle;
err = spdk_nvmf_fc_master_enqueue_event(SPDK_FC_HW_PORT_ONLINE, (void *)&args, port_init_cb);
CU_ASSERT(err == 0);
poll_threads();
set_thread(0);
if (err == 0) {
uint32_t i;
for (i = 0; i < fc_port->num_io_queues; i++) {
CU_ASSERT(fc_port->io_queues[i].fc_poll_group != 0);
CU_ASSERT(fc_port->io_queues[i].fc_poll_group != 0);
CU_ASSERT(fc_port->io_queues[i].fc_poll_group->hwqp_count != 0);
}
}
}
static void
create_poll_groups_test(void)
{
unsigned i;
SPDK_CU_ASSERT_FATAL(g_nvmf_tprt != NULL);
for (i = 0; i < MAX_FC_UT_POLL_THREADS; i++) {
set_thread(i);
g_poll_groups[i] = spdk_nvmf_poll_group_create(g_nvmf_tgt);
poll_thread(i);
CU_ASSERT(g_poll_groups[i] != NULL);
}
set_thread(0);
}
static void
poll_group_poll_test(void)
{
unsigned i;
unsigned poll_cnt = 10;
struct spdk_nvmf_fc_port *fc_port = NULL;
SPDK_CU_ASSERT_FATAL(g_nvmf_tprt != NULL);
set_thread(0);
fc_port = spdk_nvmf_fc_port_lookup(g_fc_port_handle);
SPDK_CU_ASSERT_FATAL(fc_port != NULL);
for (i = 0; i < fc_port->num_io_queues; i++) {
fc_port->io_queues[i].lcore_id = 0;
}
for (i = 0; i < poll_cnt; i++) {
/* this should cause spdk_nvmf_fc_poll_group_poll to be called() */
poll_threads();
}
/* check if hwqp's lcore_id has been updated */
for (i = 0; i < fc_port->num_io_queues; i++) {
CU_ASSERT(fc_port->io_queues[i].lcore_id == poll_cnt);
}
}
static void
remove_hwqps_from_poll_groups_test(void)
{
unsigned i;
struct spdk_nvmf_fc_port *fc_port = NULL;
SPDK_CU_ASSERT_FATAL(g_nvmf_tprt != NULL);
fc_port = spdk_nvmf_fc_port_lookup(g_fc_port_handle);
SPDK_CU_ASSERT_FATAL(fc_port != NULL);
for (i = 0; i < fc_port->num_io_queues; i++) {
spdk_nvmf_fc_poll_group_remove_hwqp(&fc_port->io_queues[i]);
poll_threads();
CU_ASSERT(fc_port->io_queues[i].fc_poll_group == 0);
}
}
static void
destroy_transport_test(void)
{
unsigned i;
set_thread(0);
SPDK_CU_ASSERT_FATAL(g_nvmf_tprt != NULL);
for (i = 0; i < MAX_FC_UT_POLL_THREADS; i++) {
set_thread(i);
spdk_nvmf_poll_group_destroy(g_poll_groups[i]);
poll_thread(0);
}
SPDK_CU_ASSERT_FATAL(g_nvmf_tgt != NULL);
g_lld_fini_called = false;
spdk_nvmf_tgt_destroy(g_nvmf_tgt, NULL, NULL);
poll_threads();
CU_ASSERT(g_lld_fini_called == true);
}
static int
nvmf_fc_tests_init(void)
{
return 0;
}
static int
nvmf_fc_tests_fini(void)
{
free_threads();
return 0;
}
int main(int argc, char **argv)
{
unsigned int num_failures = 0;
CU_pSuite suite = NULL;
if (CU_initialize_registry() != CUE_SUCCESS) {
return CU_get_error();
}
suite = CU_add_suite("NVMf-FC", nvmf_fc_tests_init, nvmf_fc_tests_fini);
if (suite == NULL) {
CU_cleanup_registry();
return CU_get_error();
}
if (CU_add_test(suite, "Create Target & FC Transport",
create_transport_test) == NULL) {
CU_cleanup_registry();
return CU_get_error();
}
if (CU_add_test(suite, "Create Poll Groups",
create_poll_groups_test) == NULL) {
CU_cleanup_registry();
return CU_get_error();
}
if (CU_add_test(suite, "Create FC Port",
create_fc_port_test) == NULL) {
CU_cleanup_registry();
return CU_get_error();
}
if (CU_add_test(suite, "Online FC Port",
online_fc_port_test) == NULL) {
CU_cleanup_registry();
return CU_get_error();
}
if (CU_add_test(suite, "PG poll", poll_group_poll_test) == NULL) {
CU_cleanup_registry();
return CU_get_error();
}
if (CU_add_test(suite, "Remove HWQP's from PG's",
remove_hwqps_from_poll_groups_test) == NULL) {
CU_cleanup_registry();
return CU_get_error();
}
if (CU_add_test(suite, "Destroy Transport & Target",
destroy_transport_test) == NULL) {
CU_cleanup_registry();
return CU_get_error();
}
CU_basic_set_mode(CU_BRM_VERBOSE);
CU_basic_run_tests();
num_failures = CU_get_number_of_failures();
CU_cleanup_registry();
return num_failures;
}

1
test/unit/lib/nvmf/fc_ls.c/.gitignore vendored Normal file
View File

@ -0,0 +1 @@
fc_ls_ut

45
test/unit/lib/nvmf/fc_ls.c/Makefile Normal file
View File

@ -0,0 +1,45 @@
#
# BSD LICENSE
#
# Copyright (c) 2018 Broadcom. All Rights Reserved.
# The term "Broadcom" refers to Broadcom Limited and/or its subsidiaries.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions
# are met:
#
# * Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# * Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in
# the documentation and/or other materials provided with the
# distribution.
# * Neither the name of Intel Corporation nor the names of its
# contributors may be used to endorse or promote products derived
# from this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#
SPDK_ROOT_DIR := $(abspath $(CURDIR)/../../../../../)
include $(SPDK_ROOT_DIR)/mk/config.mk
CFLAGS += -I$(SPDK_ROOT_DIR)/test/common/lib -I$(SPDK_ROOT_DIR)/lib/nvmf
ifneq ($(strip $(CONFIG_FC_PATH)),)
CFLAGS += -I$(CONFIG_FC_PATH)
endif
TEST_FILE = fc_ls_ut.c
include $(SPDK_ROOT_DIR)/mk/spdk.unittest.mk

1125
test/unit/lib/nvmf/fc_ls.c/fc_ls_ut.c Normal file
View File

File diff suppressed because it is too large Load Diff

8
test/unit/unittest.sh
View File

@ -165,6 +165,14 @@ $valgrind $testdir/lib/ftl/ftl_md/ftl_md_ut
$valgrind $testdir/lib/ftl/ftl_io.c/ftl_io_ut $valgrind $testdir/lib/ftl/ftl_io.c/ftl_io_ut
fi fi
if [ -e $testdir/lib/nvmf/fc.c/fc_ut ]; then
$valgrind $testdir/lib/nvmf/fc.c/fc_ut
fi
if [ -e $testdir/lib/nvmf/fc_ls.c/fc_ls_ut ]; then
$valgrind $testdir/lib/nvmf/fc_ls.c/fc_ls_ut
fi
# local unit test coverage # local unit test coverage
if [ "$cov_avail" = "yes" ]; then if [ "$cov_avail" = "yes" ]; then
$LCOV -q -d . -c -t "$(hostname)" -o $UT_COVERAGE/ut_cov_test.info $LCOV -q -d . -c -t "$(hostname)" -o $UT_COVERAGE/ut_cov_test.info