Commit Graph

18835 Commits

Author SHA1 Message Date
Konrad Sztyber
75169d0dec nvmf/tcp: update pdu_in_use flag in write functions
This makes the flag indicate whether there's an outstanding PDU write
for a given request.  Additionally, it reduces the number of places we
need to update this flag.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Id7e587f84955b096c46bfbf88d4dd222214d4a6a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12254
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2022-04-19 11:15:45 +00:00
Konrad Sztyber
c676c0815d nvmf/tcp: use different callbacks for sending mgmt/req PDUs
This will make it possible to have some common handling in request's PDU
write completion.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Icaff38da0e47dd93327e3d8f09edd9fdba8f532e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12253
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2022-04-19 11:15:45 +00:00
Konrad Sztyber
37dc93b9ef nvmf/tcp: adjust assert for zcopy req complete
When an request using zcopy is completed, it might have an unreleased
zcopy_bdev_io attached in three cases:
 1) the request was a read,
 2) the request was a failed write,
 3) the qpair is being disconnected.

The last case was missing from the assertion.

Fixes #2425

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I5cbeaa198a1fd878c98caf148a0bc47060e35bca
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12263
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-04-19 11:14:56 +00:00
Konrad Sztyber
aa21240574 nvme/pcie: increase min admin queue size to 256
Now that IO qpairs can be created asynchronously, we need to make sure
that all the create IO CQ/SQ commands can be executed simultaneously.
It is pretty common to create multiple IO qpairs at the same time, e.g.
adding an NVMe bdev to an nvmf subsystem will create an IO qpair on each
poll group.  In that case, if the number of cores exceed the size of the
admin queue (actually it can be even lower due to outstanding AERs), we
might run out nvme_requests on the admin queue.

The chosen minimum value for the admin queue size, 256, should be enough
to cover most cases.

Fixes #2465

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I55c59aef64f3fdb33f7b4824d3e9beb403602633
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12270
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-04-19 08:18:34 +00:00
Yifan Bian
b287d3ad10 test/llvm_nvme_fuzz: add WRITE ZEROES and WRITE UNCORRECTABLE command test case
Change-Id: Ia52a7258fbbb342630ff763bd2a8b88358f58b91
Signed-off-by: Yifan Bian <yifan.bian@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12189
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-04-19 08:17:33 +00:00
Jim Harris
9097116e98 test/iscsi_tgt: add iptables entry to forward packets
Similar to commit 9035986e for the nvmf tests, we add
the same FORWARD entry here for iscsi_tgt tests.

Some distros or kernels do not automatically forward
packets across the bridge we set up in
create_veth_interfaces().  So add an iptables entry to
explicitly add a forwarding rule.

This allows me to run the iscsi_tgt tests in my
Ubuntu 22.04 VM.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I9e48c10971bdf752e8bd83e5494f4a2a75b06ccd
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12161
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-04-19 08:17:19 +00:00
Ben Walker
aef71e29a0 spdk_dd: Use IORING_SETUP_SQPOLL mode
This has better performance than interrupt mode, so it is a better
default.

Change-Id: Ia2e13058b9b36874cd402f97ef57db77192f596a
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12021
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-04-19 08:17:05 +00:00
Ben Walker
958ca17750 spdk_dd: Correctly allocate enough queue depth for both input and output
When using io_uring, both input and output operations can be outstanding
at the same time. Ensure there is enough queue depth.

Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: Ibd8a652c1be32e020a281529b50576c62ff12a98
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12201
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
2022-04-19 08:17:05 +00:00
Ben Walker
d5f7c9d3f7 spdk_dd: If the file size can't be detected, just use the input size
This lets us do testing on things like /dev/zero or /dev/null

Change-Id: Iec3a73c30ad58d13f6af3bf18c0932dcd177f8f2
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12020
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
2022-04-19 08:17:05 +00:00
Shuhei Matsumoto
2c13441ba8 nvme_rdma: Destroy qpair after qpair is actually disconnected
The RDMA transport can disconnect qpair asynchronously now.

Previously, we tried to release the resource of the qpair after disconnected.
However it did not work because it was done when deleting the qpair.
The admin qpair was not deleted in a ctrlr reset sequence.

This patch tries to satisfy the same aim again but by a different way.

Previously, we released the resource of the qpair before starting
actual disconnection process. This patch release the resource of the
qpair after the qpair is actually disconnected.

The related patches are:
b9518a5540
eb09178a59

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: Id6a814895a35b1589b781a91744ef872b42aaa69
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11783
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-04-18 18:35:29 +00:00
Shuhei Matsumoto
4b73223542 nvme_rdma: Wait until lingering qpair becomes quiet before completing disconnection
The code to handle the lingering qpair when deleting it was really
complicated.

The RDMA transport can connect or disconnect qpair asynchronously.

Then we can include the code to handle the lingering qpair into the
code to disconnect qpair now.

If the disconnected qpair is still busy, defer completion of the
disconnection until qpair becomes idle.

If poll group is not used, we can complete disconnection immediately
because cq is already destroyed.

The related data and unit test cases are not necessary anymore.
So delete them in this patch.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: Ic8f81143fcad0714ac9b7db862313aa8094eeefb
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11778
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-04-18 18:35:29 +00:00
Shuhei Matsumoto
20cf90801e nvme_rdma: Handle stale connection asynchronously
Include delayed disconnect/connect retries with finite times into
the state machine of asynchronous qpair connnection.

We do not need to call back to the common transport layer but
we need to do the following, clear rqpair->cq before starting disconnection
if qpair uses poll group, and clear qpair->transport_failure_reason after
disconnected.

Additionally locate the new state STALE_CONN before INITIALIZING
because cq is not ready to use for admin qpair when the state is
STALE_CONN.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: Ibc779a2b772be9506ffd8226d5f64d6d12102ff2
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11690
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2022-04-18 18:35:29 +00:00
Shuhei Matsumoto
77c4657140 nvme_rdma: Factor out destroying rdma qpair operation
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I18e166a726cca69f13e7c5818eba57f478726286
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11689
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
2022-04-18 18:35:29 +00:00
Shuhei Matsumoto
aa36c18196 nvme_rdma: Pass callback to ctrlr_disconnect_qpair() via a parameter
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I06cbb9739286d1928ad9fc07de3715a449914d75
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11688
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-04-18 18:35:29 +00:00
Shuhei Matsumoto
75d38a301d nvme: poll_group_process_completions() returns -ENXIO if any qpair failed
TCP transport already does it but was not documented clearly.

RDMA and PCIe transports follow it and document it clearly.

Then we can check each qpair's state if
spdk_nvme_poll_group_process_completions() returns -ENXIO before
disconnected_qpair_cb() is called.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I2afe920cfd06c374251fccc1c205948fb498dd33
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11328
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-04-18 18:35:29 +00:00
Shuhei Matsumoto
9717b0c3df nvme_rdma: Connect and disconnect qpair asynchronously
Add three states, INITIALIZING, EXITING, and EXITED to the rqpair
state.

Add async parameter to nvme_rdma_ctrlr_create_qpair() and set it
to opts->async_mode for I/O qpair and true for admin qpair.

Replace all nvme_rdma_process_event() calls by
nvme_rdma_process_event_start() calls.

nvme_rdma_ctrlr_connect_qpair() sets rqpair->state to INITIALIZING
when starting to process CM events.

nvme_rdma_ctrlr_connect_qpair_poll() calls
nvme_rdma_process_event_poll() with ctrlr->ctrlr_lock if qpair is
not admin qpair.

nvme_rdma_ctrlr_disconnect_qpair() returns if qpair->async is true
or qpair->poll_group is not NULL before polling CM events, or polls
CM events until completion otherwise. Add comments to clarify why
we do like this.

nvme_rdma_poll_group_process_completions() does not process submission
for any qpair which is still connecting.

Change-Id: Ie04c3408785124f2919eaaba7b2bd68f8da452c9
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11442
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-04-18 18:35:29 +00:00
Konrad Sztyber
6e8e184bca scripts: fix PYTHONPATH references to scripts/ directory
After 7610bc38d, python modules are now located in the python/
directory.  That patch also updated PYTHONPATH assignment in most of the
scripts, but one of the perf script and some docs/messages were missed.

Change-Id: Iff3588526f2a382e599f5734d38fc32c15a7bf56
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12223
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-04-18 17:22:09 +00:00
Jim Harris
ac0c53ae58 env_dpdk: do not set RTE_MEMPOOL_F_NO_IOVA_CONTIG
This was added in patch 07526d85, back in March 2018.
This was before DPDK supported dynamic hugepage allocations.
Presumably this flag was added to reduce the amount of
memory lost due to mempool buffers that would otherwise
span an IOVA boundary (mostly typical with IOMMU off and
we are relying on physical addresses).

Removing it simplifies any code in SPDK that uses
mempool buffers for DMA operations, since it doesn't have
to worry about splitting buffers that span an IOVA
boundary - DPDK has already done it for us.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I49f6c1407fad02acae7e07c9dd00cb0449bd3554
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12277
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2022-04-15 08:25:54 +00:00
Jaroslaw Chachulski
6d1e0c0eec test/vhost: Fix vhost migration tc1 test
From qemu 6.1 The use of qemu-img create that depends on a backing
file now requires that an explicit backing format be provided.
This is for safety: if QEMU probes a different format than what you
thought, the data presented to the guest will be corrupt.

Fixes: #2466

Signed-off-by: Jaroslaw Chachulski <jaroslawx.chachulski@intel.com>
Change-Id: I56ae7c71097edb387b14c8a042408cd6c50af4ee
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12268
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Kamil Godzwon <kamilx.godzwon@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-04-15 08:24:59 +00:00
Tomasz Zawadzki
0368340581 lib/vhost: consolidate successful and invalid request path
Both blk_request_finish() and invalid_blk_request()
acomplished the same thing, with variation on handled
statuses and debug logs.

Consolidating those two into single function will help
later on when replacing completion of request processing
to single callback.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Iae7b93db01bfd98819b2bb8fad9e11afcdb3a459
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12196
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2022-04-15 07:49:32 +00:00
Tomasz Zawadzki
4f95fd7be6 lib/vhost_blk: get bdev io_channels via vhost_blk functions
This patch adds vhost_blk_[get/put]_io_channel() to be used
by virtio_blk transports.

Functions related to vhost_user sessions were modified to
use it.

dummy_io_channel reference is managed at the vhost_blk
layer and as such continues to use the spdk_[get/put]_io_channel()
APIs. The description is updated to reflect its not specific to
vhost_user transport.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I6644198da83bfa0210c167e203d3875e96f1e7ea
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11101
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2022-04-15 07:49:32 +00:00
Tomasz Zawadzki
223f1f1446 lib/vhost: separate out vhost_user specific json config
The vhost_user_config_json() will be replaced with callback
to virtio_blk transport.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I6ea0ea38f505f0d354cd34ee5ab9cd3a858bd82e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9538
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-04-15 07:49:32 +00:00
Tomasz Zawadzki
6f89388ed3 lib/vhost: move vhost_user related fields from spdk_vhost_dev
spdk_vhost_dev structure should only contain generic fields
that are to be used by either vhost, vhost_blk or vhost_scsi
layer.

The vhost_user backend can hold its properties in
spdk_vhost_user_dev, which is maintained within rte_vhost.

Both structures contain references back to each other.
The reference in spdk_vhost_dev is a void pointer to
allow future transports to keep the reference
to their own structures.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I68640c524426d885c20242146365ba242fa9df8e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11813
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2022-04-15 07:49:32 +00:00
Or Gerlitz
bfcfdb7903 nvmf/rdma: Use spdk allocation scheme for RDMA requests and receives
In a similar manner for what we do for other per IO data-structures of cmds,
cpls and bufs, use the conventional huge-pages based spdk allocation scheme
for RDMA requests and receives.

Change-Id: I4c2e86e928106e78c053f24915e2a9ce1a200c78
Signed-off-by: Or Gerlitz <ogerlitz@nvidia.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12273
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2022-04-15 07:48:23 +00:00
Or Gerlitz
5edb8edca7 nvmf/rdma: use LIFO practice for incoming queue
To maximize cache locality, use lifo and not fifo when managing objects
which are used per IO such as the RDMA receive elements queue.

Change-Id: Id8917558acc1bec29943fcbae6afe6b072bde6ac
Reported-by: Jim Harris <james.r.harris@intel.com>
Signed-off-by: Or Gerlitz <ogerlitz@nvidia.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12272
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-04-15 07:48:23 +00:00
zhangduan
87cfed8442 sock: Add ack_timeout to spdk_sock_opts
Due to the same reason as transport_ack_timeout for
RDMA transport, TCP transport also needs ack timeout.
This timeout in msec will make TCP socket to wait for
ack util closes connection.

Signed-off-by: zhangduan <zhangd28@chinatelecom.cn>
Change-Id: I81c0089ac0d4afe4afdd2f2c7e5bff1790f59199
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12214
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-04-14 08:34:29 +00:00
paul luse
37b68d7287 accel: cleanup by getting rid of capabilties enum
In support of upcoming patches and to greatly simplify things,
the capabilites enum which held bit positions for each opcode
has been removed.  Only the opcodes enum remains and thus only
opcodes are used throughout.  For the capabiltiies bitmap a helper
function is added to convert from opcode to bit position.  Right
now it is used in the IO path but in upcoming patches that goes away
and the conversion is only done at init time.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Ic4ad15b9f24ad3675a7bba4831f4e81de9b7bc70
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11949
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-04-14 08:32:50 +00:00
Shuhei Matsumoto
0d29a988be bdev: Use spdk_for_each_bdev() for bdev_get_io_stat and get_bdevs RPCs
spdk_for_each_bdev() can be applied simply to rpc_bdev_get_bdevs()
because the callback function rpc_dump_bdev_info() is synchronous.

spdk_for_each_bdev() cannot be applied simply to rpc_bdev_get_iostat().
The factored-out callback function _bdev_get_device_stat() is
asynchronous. Add desc pointer to struct spdk_bdev_io_stat and open
before and close after executing spdk_bdev_get_device_stat().

Replace spdk_bdev_get_by_name() by spdk_bdev_open_ext() when a bdev name
is specified.

spdk_bdev_register() checks if the name of each bdev is set and each bdev
is opened while collecting its stats now. Hence it is not possible that
spdk_bdev_get_name() returns NULL. Simplify rpc_bdev_get_iostat_cb()
based on this fact.

Furthermore, we want to fail the RPC for all failures. The callback
function to spdk_bdev_get_device_stat() is executed after stack
unwinding if successful. Defer starting RPC response until it is ensured
that all spdk_bdev_get_device_stat() calls will succeed or there is no bdev.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I7b036d6d707c49d19c8922a159b12b5b5ce7ca41
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12089
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-04-14 08:32:12 +00:00
Ziv Hirsch
e749fa9c27 nvmf: fix buffer overflow on admin commands
When req->iovcnt is bigger than 1, `memset(req->data, 0, req->length)` is wrong.

Signed-off-by: Ziv Hirsch <zivhirsch13@gmail.com>
Change-Id: Ie53eba686b4c5889bbde3b3644d51acbef303b42
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12216
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-04-14 08:31:35 +00:00
Karol Latecki
20ec3ed66f scripts/vagrant: add missing Fedora 35 image name
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I910e1b9d23673fd5876bca60371051b4ac888ba4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12228
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
2022-04-13 14:41:49 +00:00
Karol Latecki
712c904a2f scripts/vagrant: change default Fedora 34 box
Use "generic" Fedora34 from Roboxes, as we did for
all other Fedora Vagrant boxes.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I208264c50a067b26c9ec5da2250f27881ba8431d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12227
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
2022-04-13 14:41:49 +00:00
Karol Latecki
a38452186c scripts/vagrant: Add Fedora 36 as an option
Change-Id: Ie25212a3cd4a523b8fc6c1bbee6e03b8f05a8583
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12226
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
2022-04-13 14:41:49 +00:00
Krzysztof Karas
f9e4e1fd62 spdk_top: add pop-up windows' response to interface resize
Make pop-up details windows handle interface resize and
redraw themselves in the center of spdk_top window.

Change type of check_resize_interface() from void to int.

Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Change-Id: Ib7d8e17c794eb4aab593a6057a1809ea678d633c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7398
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-04-13 12:47:16 +00:00
Krzysztof Karas
59f2f01f22 spdk_top: make checking interface resize into a function
Creates new function to check interface resize.
This will be used in the next patch.

Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Change-Id: I147145be2db4fdab6f97b67baf3c2936abf7f6a3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7499
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-04-13 12:47:16 +00:00
Krzysztof Karas
0ce6762b57 spdk_top: redraw tabs after exiting popups
Adds the following after exiting popups:
-redrawing tab menu
-tab and core window refresh after closing thread pop-up
-update_panels() to refresh column names and lines
-redrawing tab contents

The goal of this patch is purely to improve visual
aspects of spdk_top.

Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Change-Id: I16952b75dd8e79521a41a8c2bda6fa62d43ccec8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7397
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-04-13 12:47:16 +00:00
Krzysztof Karas
fb558e0dff spdk_top: set minimal time on refresh rate
Set minimal required time for data gathering thread
to give display thread time to redraw all windows.
This is useful when user-defined refresh rate is set
to 0. With minimal required time hardcoded here, we
avoid multiple data refresh calls during one
display refresh (because drawing on screen takes
more time).

Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Change-Id: I44ae808bc25eed36de60eccce30c8ecdeca5088d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7396
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-04-13 12:47:16 +00:00
Krzysztof Karas
65da9a26d3 spdk_top: add dynamic refreshing when core popup is active
Changes refreshing of cores tab to work from within cores pop-up
refresh function. This works the same way as refreshing of
THREADS and POLLERS pop-ups and tabs.

Added a handle for core window and core info to display_thread().
The purpose is to allow threads popup to refresh already active
cores popup. Otherwise, there are artifacts left where core popup
was displayed.

Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Change-Id: I6471d5001989870a0c073f771cb8ed5acc05445e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7394
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-04-13 12:47:16 +00:00
Krzysztof Karas
05c45b39e0 spdk_top: move draw_core_win_content()
Move draw_core_win_content() before display_thread(), because
it is going to be used there in the next patch.

Change-Id: I2dee246f14bc95f7dcdd1641830114919b300630
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11237
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-04-13 12:47:16 +00:00
Krzysztof Karas
6a0c7766bb spdk_top: separate core popup content drawing
Separate drawing of core popup content from the rest
of the popup's functionality.

Change-Id: I60641d347528b08e2809d9678ca306d9db5bb823
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11236
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-04-13 12:47:16 +00:00
Krzysztof Karas
d18ec6e392 spdk_top: make pollers tab refresh with pollers pop-up active
Changes refreshing of pollers tab to be done from within poller
pop-up display function. As mentioned before, NCURSES does not
behave predictably, when different parts of screen are refreshed
from within multiple functions at a time, so the "control" of
refreshing has to be passed to one function.

Adds refreshing of pollers pop-up.

Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Change-Id: I37dcedc495c314462b252774d97410158e1b895d

Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7393
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-04-13 12:47:16 +00:00
Krzysztof Karas
c82148a5f3 spdk_top: separate drawing poller popup content
Separate drawing of popup content from the rest of
window functionality.

Change-Id: I00f4de51beb9e3bb61e4003dae8a341408466ff6
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11235
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-04-13 12:47:16 +00:00
Krzysztof Karas
409cc5bdd4 spdk_top: modify refreshing of display
Changes calling refresh on different parts of spdk_top window.
refresh() function copies the virtual screen contents to the terminal.
With constant refreshes the terminal would blink. To fix this refresh()
is replaced with wnoutrefresh() calls, which only refreshes
the virtual screen. After all changes to the terminal are ready, one
refresh() is enough to draw everything on screen.

Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Change-Id: I16b526a1cac8f68dee833ae775cfddee131851d0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7286
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-04-13 12:47:16 +00:00
Krzysztof Karas
53add5e240 spdk_top: make threads pop-up window refresh itself and active tab
Adds refreshing of threads and cores tab when thread pop-up is active.

Adds refreshing of threads pop-up and moved thread_win allocation and
poller number checking inside while loop.

Dynamic refresing of core pop-up with thread pop-up active
will be added in future patches.

Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Change-Id: I916f2b64a81db22e5caa32948094b4fce0b6577c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7391
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-04-13 12:47:16 +00:00
Krzysztof Karas
216a7c3d0b spdk_top: add a helper function to check if thread exists
This check is going to be called in two places in the next patch.
The purpose is to eliminate duplicating code.

Change-Id: I1d739395453c083479c104e2774b1e0c01962bd1
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11234
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-04-13 12:47:16 +00:00
Jim Harris
92f0be87a0 iscsi: use EVP APIs for md5 calculations
OpenSSL 3.0 deprecated the MD5_xxx APIs, so switch
the md5 code in the iscsi library to use the EVP
APIs recommended by OpenSSL instead.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ic5e3cd6e30ebc8b027f0715434cc3be045f1b770
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12240
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2022-04-13 08:33:51 +00:00
Jim Harris
7778bc3a33 vhost: copy virtio_blk_outhdr to local struct
Some SeaBIOS versions are not aligning virtio_blk_outhdr
on 8-byte boundary, causing ubsan failures.  To be safe,
let's just make sure on our end that we only access a
properly aligned structure by copying this small (16-byte)
data structure to a local structure variable.

Fixes issue #2452.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iacad72c3a1759fb8dc5ba411272a34d93ef2a6fc
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12238
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-04-13 08:33:51 +00:00
yupeng
df80e8ffc6 bdev/raid: stop the raid bdev in raid_bdev_destruct
The raid bdev may receive IO requests after the raid layer
invokes spdk_bdev_unregister function. This patch moves the
raid_bdev->module->stop to the raid_bdev_destruct function.
Then there will be no IO after the raid_bdev->module->stop
is invoked.
Below is a way to reproduce this issue:
(1) create a malloc bdev
sudo ./scripts/rpc.py bdev_malloc_create --name malloc0 128 4096
(2) create a concat bdev base on the malloc bdev
sudo ./scripts/rpc.py bdev_raid_create --name concat0 \
--raid-level concat --base-bdevs malloc0 --strip-size-kb 4
(3) create a lvstore base on the concat bdev
sudo ./scripts/rpc.py bdev_lvol_create_lvstore \
--cluster-sz 4194304 --clear-method unmap concat0 lvs0
(4) remove the concat bdev
sudo ./scripts/rpc.py bdev_raid_delete concat0
In the step(4), the spdk app will crash because an IO request
is sent to the concat bdev after the concat_stop is invoked.

Signed-off-by: Peng Yu <yupeng0921@gmail.com>
Change-Id: I76d6964ee9528d4590ed86e9c5c28d53e85da32f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12221
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-04-13 08:33:19 +00:00
Changpeng Liu
1bac8afd02 nvmf/vfio-user: unlink created files
Only FDs are used for passing them to another process,
we can unlink them after creation.

Here we only unlink the files created in vfio-user,
and there is still one file created via libvfio-user,
it will be fixed via
https://github.com/nutanix/libvfio-user/issues/660.

Partly fix issue #2449.

Change-Id: Ie27640e0cb85f44596e9d0ad5a2b67adf0419f5c
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12195
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: John Levon <levon@movementarian.org>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-04-12 13:59:57 +00:00
Changpeng Liu
c47b7b0276 nvme/vfio-user: use API to setup BAR0 doorbells
We can use lib/vfio-user API to setup BAR0 doorbells,
existing implementation is redundant.

Change-Id: Ib880d167c84c6b8482bf1a35559a34c939f6a02d
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12211
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: John Levon <levon@movementarian.org>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-04-12 07:24:22 +00:00
Pawel Piatek
04b16168cf post_process: fix pylint W1732,W1705,W1722
Fix pylint findings:
- R1732 consider-using-with
- R1705 no-else-return
- R1722 consider-using-sys-exit

Signed-off-by: Pawel Piatek <pawelx.piatek@intel.com>
Change-Id: If409c34f2ab8712e3fb1ac25b1c297d7e75c38fb
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12139
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-04-12 07:24:15 +00:00