ivampiresp/Spdk
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

nvme: copy 8 bytes at a time into virtualized CMB

Browse Source 
QEMU won't emulate wide (>8 byte) instructions on an MMIO
region.  We tried to fix this previously but it wasn't sufficient.
So this version adds an explicit copy function that will only
copy 8 bytes at a time from the source cmd pointer into the
SQ cmd point in the virtualized CMB.

Fixes issue #1458.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I804005a2e8cf3f7365c1608674f82a3f843e59ba

Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3135
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
This commit is contained in:
Jim Harris 2020-06-30 16:57:41 -07:00 committed by Tomasz Zawadzki
parent e762508854
commit 16364fdc14
1 changed files with 17 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

18
lib/nvme/nvme_pcie.c
View File

@ -1153,6 +1153,22 @@ nvme_pcie_qpair_construct(struct spdk_nvme_qpair *qpair,
return 0;
}
/* Used when dst points to MMIO (i.e. CMB) in a virtual machine - in these cases we must
* not use wide instructions because QEMU will not emulate such instructions to MMIO space.
* So this function ensures we only copy 8 bytes at a time.
*/
static inline void
nvme_pcie_copy_command_mmio(struct spdk_nvme_cmd *dst, const struct spdk_nvme_cmd *src)
{
uint64_t *dst64 = (uint64_t *)dst;
const uint64_t *src64 = (const uint64_t *)src;
uint32_t i;
for (i = 0; i < sizeof(*dst) / 8; i++) {
dst64[i] = src64[i];
}
}
static inline void
nvme_pcie_copy_command(struct spdk_nvme_cmd *dst, const struct spdk_nvme_cmd *src)
{
@ -1336,7 +1352,7 @@ nvme_pcie_qpair_submit_tracker(struct spdk_nvme_qpair *qpair, struct nvme_tracke
* virtual NVMe controller, the maximum access width is 8 Bytes for one time.
*/
if (spdk_unlikely((ctrlr->quirks & NVME_QUIRK_MAXIMUM_PCI_ACCESS_WIDTH) && pqpair->sq_in_cmb)) {
pqpair->cmd[pqpair->sq_tail] = req->cmd;
nvme_pcie_copy_command_mmio(&pqpair->cmd[pqpair->sq_tail], &req->cmd);
} else {
/* Copy the command from the tracker to the submission queue. */
nvme_pcie_copy_command(&pqpair->cmd[pqpair->sq_tail], &req->cmd);