Commit Graph

14554 Commits

Author SHA1 Message Date
Karol Latecki
097e6e16b9 nvme/hotplug.sh: Copy external DPDK libs into test VM
Need to select proper path for passing libs into the VM
in case we're building with custom DPDK.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I97d301c70adee31b727c6b6673eadac3cbde9817
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4975
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-30 16:13:59 +00:00
Alexey Marchuk
9b19abae3c sock/posix: Disable zcopy send by default
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Change-Id: I4825c681d742946dfcf5bdc209356194766a15cd
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4978
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-10-30 14:51:34 +00:00
Alexey Marchuk
d296fcd8d9 nvme_tcp: Fix icreq/icresp handing with zcopy enabled.
There is a problem with TCP zcopy enabled:
1. TCP initiator sends icreq and start polling a qpair. Polling of qpair
actively calls nvme_tcp_read_pdu function
2. nvme_tcp_read_pdu: qpair is in NVME_TCP_PDU_RECV_STATE_AWAIT_PDU_CH state,
it reads 8 bytes of common PDU header. It determines the type of the PDU
and finds the size of PDU_PSH header.
3. nvme_tcp_read_pdu: qpair is in NVME_TCP_PDU_RECV_STATE_AWAIT_PDU_PSH state.
It should read 120 bytes of icresp PDU. The number of bytes which needs to be
read is pdu->psh_len - pdu->psh_valid_bytes. qpair receives 120 bytes
(the full PDU) and calls nvme_tcp_pdu_psh_handle -> nvme_tcp_icresp_handle.
Here we check that we haven't yet received buffer reclaim notification and
simply return from this function. At the same time we continue to poll the qpair.
4. nvme_tcp_read_pdu: qpair is in NVME_TCP_PDU_RECV_STATE_AWAIT_PDU_PSH state
and tries to read data from a socket again. The number of bytes is
pdu->psh_len - pdu->psh_valid_bytes. But now pdu->psh_len == pdu->psh_valid_bytes,
so we call nvme_tcp_read_data with zero length.
readv with zero length is commonly used to check errors on the socket,
but in our case there is no errors and readv returns 0.
5. nvme_tcp_read_data treats zero as error and return NVME_TCP_CONNECTION_FATAL.

Fix is to handle icresp, but leave qpair in INITIALIZING state until
we receive acknowledgement for icreqsend_ack. We also move qpair to
NVME_TCP_PDU_RECV_STATE_AWAIT_PDU_READY recv_state so recv_pdu
will be zerofied and qpair will try to read a common PDU header.
But since it is not initialized yet, it won't receive anything
from the target.

Fixes issue #1633

Change-Id: I22cedefe530a8ac3b51495988ed6265d8fad15bb
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4969
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-30 09:05:35 +00:00
Simon A. F. Lund
f875da4019 examples/nvme_fio_plugin: update ZNS section of README
Signed-off-by: Simon A. F. Lund <simon.lund@samsung.com>
Change-Id: I99c82490811314201b53dde59e586538835e0840
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4950
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Niklas Cassel <niklas.cassel@wdc.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-29 22:15:40 +00:00
Simon A. F. Lund
aae84b1f0e examples/nvme_fio_plugin: add plugin-option 'initial_zone_reset'
Added plugin-option 'initial_zone_reset', providing the option to reset
all zones on all namespaces with the Zoned Command Set enabled upon
initialization.
The default is not to reset. The option is exposed even when the ZBD
plumbing is not available. However, it will then inform the user that
ZBD/ZNS is not supported instead of resetting.

The plugin-option provides a short-term solution to an observed issue
with consecutive invocations of fio exhausting maximum-active-resources.
A longer-term solution would be to add a 'max_active_zones' limit in fio
and ensure that fio does not exceed that limit.

Signed-off-by: Simon A. F. Lund <simon.lund@samsung.com>
Change-Id: I65341c028a97657370b315fb298bf97651b9bffd
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4949
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Niklas Cassel <niklas.cassel@wdc.com>
2020-10-29 22:15:40 +00:00
Simon A. F. Lund
528ad3b3cf examples/nvme_fio_plugin: move completion-helpers
Preparation patch for the addition of the 'initial_zone_reset' plugin-option.

Signed-off-by: Simon A. F. Lund <simon.lund@samsung.com>
Change-Id: I768fc207b74cfa2a516009e10fc2a4646d06ba72
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4948
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Niklas Cassel <niklas.cassel@wdc.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-29 22:15:40 +00:00
Jim Harris
f896aa6f10 env_dpdk: add rte_rcu library dependency
rte_hash depends on rte_rcu starting in upcoming
DPDK 20.11 release.  rte_rcu was only added in
DPDK 19.05 release, so we need to check if it
exists before linking it.

Fixes issue #1661.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I7e343c6f964b03cc62484b57803a3bad00f80288
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4947
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-10-29 10:39:05 +00:00
Jim Harris
f2196fcafb dpdk: move submodule to build rte_rcu library
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ie38573774612ac7ec7cf23367d3233124acd1a66
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4953
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-10-29 10:39:05 +00:00
Jim Harris
d792356a5a test/external_code: use variable to hold DPDK lib list
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ie4a489926695cc56125f0e18796071f7730190f1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4946
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-10-29 10:39:05 +00:00
Maciej Szwed
67109ff88b changelog: Add information regarding scheduler implementation
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: Id389ffb2c6091add92fb2849fac21a0472c8a404
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4857
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-29 08:11:44 +00:00
Michael Haeuptle
7fc48a5ffc nvme: break completion loop when ctrlr is invalid
This fixes #1423 where the completion loop never
breaks when the NVMe ctrlr is no longer present.
This condition can happen during a hot remove.

Signed-off-by: Michael Haeuptle <michael.haeuptle@hpe.com>
Change-Id: Ia238c8aeae720832068de28ce4d34a9d233344fb
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4831
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Vasuki Manikarnike <vasuki.manikarnike@hpe.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-10-29 08:11:35 +00:00
Jim Harris
ddf86600bb nvme: continue probing ctrlrs even if one fails
It is possible that a single probe_ctx could be used
to probe multiple newly attached nvme controllers.  If
one of those controllers is removed during this process,
the rest of the controllers do not get probed and can
even get stuck in a zombie state.

It is better to just continue with probing the rest of
the controllers.

Fixes issue #1611.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I4156ee8b50e8d52cfeee7224f210a58bb773e939

Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4945
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michael Haeuptle <michaelhaeuptle@gmail.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Vasuki Manikarnike <vasuki.manikarnike@hpe.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-10-29 08:11:17 +00:00
Michael Haeuptle
4409007906 nvmf: Fixes double triggering of association timer
Fixes issue #1635.

Under rare circumstances, the CC.en and CC.shn are both set
which then results in setting the association timer twice.
This scenario was observed during hot plug testing when the
initiator tries to reset the subsystem that contains the
removed device.
The end result is that when the ctrlr is destructed, then
one of the timers can still fire and access freed memory.

Change-Id: Ie5880ab325a28f19361f73712bdeb5b58894ee68
Signed-off-by: Michael Haeuptle <michael.haeuptle@hpe.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4935
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-10-29 08:11:12 +00:00
Maciej Wawryk
c4c37f1cf1 Modifications for using universal qcow2 image in tests
We want to replace few qcow2 images with one universal.
This commit contains:
 - change password in autotest.sh
 - change image path
 - change image name
 - use snapshot mode in hotplug.sh instead of copying base image

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I75c457fe75f005b0ab43ca909be7886529ed115b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4551
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2020-10-28 15:07:56 +00:00
Niklas Cassel
1e83b640aa examples/nvme_fio_plugin: fix zone reporting
All zone management receive helper functions (including
spdk_nvme_zns_report_zones()) are implemented to match the parameters of
the zone management receive function in the ZNS specification.

The documentation for spdk_nvme_zns_report_zones() states:
"param partial_report If true, nr_zones field in the zone report indicates
the number of zone descriptors that were successfully written to the zone
report. If false, nr_zones field in the zone report indicates the number
of zone descriptors that match the report_opts criteria."
This matches the description of the "Partial Report" bit in the ZNS spec.

Since the FIO function parse_zone_info() calls the io_ops->report_zones()
function multiple times, until all zones have been reported, it expects
the return from this function to represent the number of zones that were
successfully reported.

By setting the partial_report bit to false, the controller will return
the total number of zones, and since spdk_fio_report_zones() loops until
idx < report->nr_zones, and writes to zbdz[idx], the current code will
overwrite heap memory, since idx will take on index values that are out
of bounds for the memory allocated by the FIO function parse_zone_info().

Therefore, spdk_fio_report_zones() has to set the partial_report bit to
true when calling the NVMe level function spdk_nvme_zns_report_zones().

Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Change-Id: I8846711bfed4faadac0315b450158293cefa36f4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4871
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Simon A. F. Lund <simon.lund@samsung.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-10-28 15:07:37 +00:00
Niklas Cassel
00d197dba2 examples/nvme/identify: print ZNS data structures if supported
Print the Zoned Namespace Command Set Specific data structures,
if the namespace/controller supports them.

spdk_nvme_zns_ctrlr_get_data() returns NULL for a controller
that does not support the ZNS specific controller data struct.

Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Change-Id: I0acd2695976fc598b61591989f612db35ac821db
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4839
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-10-28 15:07:10 +00:00
Simon A. F. Lund
414500c9eb examples/nvme_fio_plugin: fix log-msgs. in report_zones() and reset_wp()
In spdk_fio_report_zones(), log_err did not prefix messages with
"spdk/nvme", making it hard to determine who dumped the error-message.
In spdk_fio_reset_wp() log_err described the wrong function.

This change fixes the above.

Signed-off-by: Simon A. F. Lund <simon.lund@samsung.com>
Change-Id: I41df6d451e88942806c8b5a3cf9a0902d98cb186
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4916
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Niklas Cassel <niklas.cassel@wdc.com>
2020-10-28 15:06:44 +00:00
Simon A. F. Lund
12a44d4745 examples/nvme_fio_plugin: fix reset_wp()
When _reset_wp() received a range to reset, then the loop kept resetting
the first zone in the range.
Also, the processing of command-completion were re-using the same
'completion' state, thus a previous completion would short-circuit
command-completion such that it would never be processed.

This change fixes that.

Also, the reset-loop assumes that the given offset is a valid zone-start
LBA, a check is added to verify that and return -EINVAL if it is not.

Signed-off-by: Simon A. F. Lund <simon.lund@samsung.com>
Change-Id: I1a1e4be2e1f67c2d8fecb5fc36a211b2dbb5a921
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4915
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Niklas Cassel <niklas.cassel@wdc.com>
2020-10-28 15:06:44 +00:00
Simon A. F. Lund
906c2adb86 examples/nvme_fio_plugin: help the user with max_open_zones constraints
When a device has resource-limitations such as the
maximum-open-resources (mor) and this threshold is exceeded, then IO
will fail upon completion. Such behavior is not the most user-friendly
way to tell the user that they should provide a value for the
fio-parameter 'max_open_zones'.

This change provides an arguably more user-friendly approach by checking
whether the device is limited and in case it is:

* Provide a default value for 'max_open_zones', inform the user, and
continue
* Verify 'max_open_zones' and in case of error inform the user and
return error

Signed-off-by: Simon A. F. Lund <simon.lund@samsung.com>
Change-Id: I76cb045d560b9ec5701d97b82a62947af11960b6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4914
Reviewed-by: Niklas Cassel <niklas.cassel@wdc.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-28 15:06:44 +00:00
Changpeng Liu
2c9b5b5af5 env/memory: use stack variable when unmapping the dma region
When enable Werror compile option with new kernel(v5.8), there is
following error reported due to the <linux/vfio.h> data structure
change(added a uint8_t data[] variable in new kernel), we can just
put the 'unmap' at the end of the data structure to fix the issue,
I think it's better to just use a stack variable instead.

CC lib/env_dpdk/memory.o
memory.c:63:36: error: field 'unmap' with variable sized type 'struct vfio_iommu_type1_dma_unmap' not
at the end of a struct or class is a GNU extension [-Werror,-Wgnu-variable-sized-type-not-at-end]
struct vfio_iommu_type1_dma_unmap unmap;
^
1 error generated.

Change-Id: Icf73a3c48a301e74b92b9ae2e2d8715262b2d056
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4925
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-10-28 15:06:12 +00:00
yidong0635
a474889bc6 rocksdb/env_spdk: Fix unused warning.
Fuction takes one parameter to print,
others are unused.

spdk/lib/rocksdb/env_spdk.cc: In function
 ‘void rocksdb::base_bdev_event_cb(spdk_bdev_event_type, spdk_bdev*, void*)’:
/spdk/lib/rocksdb/env_spdk.cc:666:70:
error: unused parameter ‘bdev’ [-Werror=unused-parameter]
666 | base_bdev_event_cb(enum spdk_bdev_event_type type, struct spdk_bdev *bdev,
      |                                                    ~~~~~~~~~~~~~~~~~~^~~~
/home/yidong/spdk/lib/rocksdb/env_spdk.cc:667:12:
error: unused parameter ‘event_ctx’ [-Werror=unused-parameter]
  667 |      void *event_ctx)
      |      ~~~~~~^~~~~~~~~
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Change-Id: Ic1cf45443ab1dcdf38d1b9c6bdea2905e94df19c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4929
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
2020-10-28 12:49:10 +00:00
Liu Xiaodong
e2f773aafc reactor: check calloc failure in gather_metrics
A round of _reactors_scheduler_gather_metrics should be stopped
when there is calloc failure.

Change-Id: Ic2220c561abb07a849ea37d3c88af3f6d5d1ffa1
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4882
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-27 16:20:32 +00:00
Liu Xiaodong
b45788036f lib/rocksdb: remove redundant linked blobfs_bdev
blobfs_bdev lib is already added into BLOCKDEV_MODULES_LIST
so it shouldn't be included by application who already
uses BLOCKDEV_MODULES_LIST or ALL_MODULES_LIST.

Fixes issue: #1654

Change-Id: I46a272e4593e19cf14c3ed8b2965797443c37a0d
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4876
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-10-27 16:20:25 +00:00
yidong0635
1e2b38ba78 example/perf: Fix warning about scanbuild.
rc has been assigned to -1 directly, so it's no used here.

Issue:
accel_perf.c:681:7: warning: Although the value
stored to 'rc' is used in the enclosing expression,
the value is never actually read from 'rc'
        if ((rc = spdk_app_parse_args(argc, argv, &opts, "o:q:t:yw:P:f:", NULL, parse_args,
             ^    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 warning generated.

Signed-off-by: yidong0635 <dongx.yi@intel.com>
Change-Id: I1d0a4447c4655bde97fa18d50b694809fc9010ea
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4825
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2020-10-27 08:42:32 +00:00
yidong0635
eb9577f8d5 module/nvme: Fix warning about scanbuild.
Analyzer treats these as warning, they are unused.

Issue:
bdev_nvme_rpc.c:706:7: warning: Although the value
stored to 'rc' is used in the enclosing expression,
the value is never actually read from 'rc'
        if ((rc = spdk_nvme_ctrlr_reset(firm_ctx->ctrlr)) != 0) {
             ^    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
bdev_nvme_rpc.c:851:8: warning: Although the value
stored to 'rc' is used in the enclosing expression,
the value is never actually read from 'rc'
         if ((rc = spdk_bdev_open(bdev2, true, NULL, NULL, &desc)) != 0) {
              ^    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2 warnings generated.
nvme_rpc.c:375:10: warning: Although the value stored
to 'rc' is used in the enclosing expression, the value is
never actually read from 'rc'
        return rc = val->type == SPDK_JSON_VAL_STRING ? -ENOMEM : -EINVAL;
                ^    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 warning generated.

Signed-off-by: yidong0635 <dongx.yi@intel.com>
Change-Id: I1d58db97b00ac2d102c1a46d7852c2ce3f559fda
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4824
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-10-27 08:42:32 +00:00
yidong0635
fd661859ed nvmf/ctrlr: Fix scanbuild warning.
Issue:
ctrlr.c:1851:13: warning: Although the value stored
to 'copy_len' is used in the enclosing expression,
the value is never actually read from 'copy_len'
        next_pos = copy_len = 0;
                   ^          ~

Signed-off-by: yidong0635 <dongx.yi@intel.com>
Change-Id: Ie6d402cd4ad206ae5713deb15e2d03929bac94f4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4823
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Community-CI: Mellanox Build Bot
2020-10-27 08:42:32 +00:00
yidong0635
5c91e9d9e4 event/app: Fix warning about unused value.
Issue:
app.c:700:7: warning: Although the value stored to 'rc' is
used in the enclosing expression, the value is never actually read from 'rc'
        if ((rc = app_setup_signal_handlers(opts)) != 0) {
             ^    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Signed-off-by: yidong0635 <dongx.yi@intel.com>
Change-Id: I4912d37c8300c88afc7668b792039683addc94c0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4822
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-10-27 08:42:32 +00:00
yidong0635
2a53883af9 env_dpdk/memory: Fix warning of scanbuild.
We can verify readdir(dir) directly.
And this fixes issue :
memory.c:1233:23: warning: Although the value stored to 'd'
 is used in the enclosing expression, the value is never
actually read from 'd'
        while (count < 3 && (d = readdir(dir)) != NULL) {
                             ^   ~~~~~~~~~~~~

Signed-off-by: yidong0635 <dongx.yi@intel.com>
Change-Id: I34b191c9f89d8ae67697ec8e331f9c7c78c16ab2
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4819
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-10-27 08:42:32 +00:00
Liu Xiaodong
36d2e93ea1 vhost-blk: extract per queue vdev worker
Change-Id: Id852585f3eb3b61f5a074336c73e300f020c594e
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4863
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-10-27 08:40:28 +00:00
Liu Xiaodong
d2db3cdf8f vhost: extract vhost_session_vq_used_signal
Change-Id: I249e3e0df45f394c03aedfa7fa2a960de7419a58
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4862
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-10-27 08:40:28 +00:00
Liu Xiaodong
6a1ec6a640 vhost: extract session_vq_io_stats_update
Change-Id: Id0349d571c974e963fbbf080bd6e2fe113b13622
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4769
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-10-27 08:40:28 +00:00
Allen Zhu
3e2c9092d4 rpc/bdev.py: pass zero values to SPDK when allowed
In bdev_nvme_set_options command, timeout_us/retry_count/arbitration_burst/low_priority_weight/
medium_prioity_weight/high_prioity_weight/nvme_ioq_poll_period_us/io_queue_requests can be 0,
which should be passed in the rpc command.

Signed-off-by: Allen Zhu <allenz@mellanox.com>
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Signed-off-by: Evgeniy Kochetov <evgeniik@mellanox.com>
Change-Id: I2ebff51b7b7de418c5ac8fbd0265444fd27e1e10
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3816
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2020-10-27 08:40:12 +00:00
Allen Zhu
96ab62802c rpc/nvmf.py: pass zero values to SPDK when allowed
in_capsule_data_size/buf_cache_size/sock_priority/max_namespaces can be 0,
which should be passed in nvmf_create_transport/nvmf_create_subsystem commands.

Signed-off-by: Allen Zhu <allenz@mellanox.com>
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Signed-off-by: Evgeniy Kochetov <evgeniik@mellanox.com>
Change-Id: Ib557cf9f20f7ec2c0b3c31156cd79dbd670ce7e7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3815
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2020-10-27 08:40:12 +00:00
Alexey Marchuk
f600ca4c3c nvmf/tcp: Repack spdk_nvmf_tcp_qpair structure
Move data used in IO path to the beginning of the structure,
eliminate several holes

Change-Id: I45202f31c888fba32307a8b8ae2b62e7e601a32f
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4760
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-10-27 08:40:12 +00:00
Alexey Marchuk
a85057ea07 nvmf/tcp: Repack spdk_nvmf_tcp_req structure
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Change-Id: I4ca230a8a5ad01bb72cece4ce4cf543db144fcce
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4759
Community-CI: Mellanox Build Bot
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-27 08:40:12 +00:00
Alexey Marchuk
c1fbbfbe56 nvmf/tcp: Align recv_buf_size to MIN_SOC_PIPE_SIZE
If the user decided to disable ICD then we have several side effects:
1. SPDK prints several warnings/errors
2. SPDK doesn't create recv pipe and doesn't set SO_RCVBUF socket option.

I think that we should not rely on ICD only when we create recv pipe or
set SO_RCVBUF since data may be transferred in sgls via R2T/H2C and
we still need recv_pipe and SO_RCVBUF for better performance.
Alternative option is to set recv_buf_size as a maximum between
ICD and io_unit_size

Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Change-Id: Ida71ecc099f9a9355e4617f13315a341872d1cb3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4755
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-10-27 08:40:12 +00:00
Alexey Marchuk
4fe47d6ff4 rpc: Add new parameter 'control_msg_num' to 'nvmf_create_transport'
This parameter represents the number of control messages to be
allocated per poll group, specific for TCP transport.
The new parameter can't be zero.

Change-Id: I8ae198c0b46e9a5850a80492aa6260f0c6ef885e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4829
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-10-27 08:40:12 +00:00
Alexey Marchuk
85fa43241b nvmf/tcp: Support ICD for fabric/admin commands
According to the SPEC we should support up to 8192 bytes
of ICD for admin and fabric commands. Transport configuration
parameter in_capsule_data_size is applied to all qpair types -
admin and IO. Also we allocate resources when we get a connection
request, so we don't know qpair type at this moment.
Create a list of buffer in TCP poll group to support ICD up
to 8192 bytes when configuration ICD is less than this value.
The number of elements in this pool is hardcoded, it is planned
to add a new configuration parameter later.

Fixes issue #1569
Change-Id: I8589e3e2ea95d515f5503c6de7c1ee40aaf7b6da
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4754
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-10-27 08:40:12 +00:00
Jim Harris
c31ad66893 event: deprecate opts.config_file member
Just always put the config file name in json_config_file,
since we now only support JSON.

If user specifies both -c and --json, it will just take
the latter of the two.  This is similar to if the user
specified --json twice.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Idc21d73acf0e190eda57a7b0c5d9bcfa14e87030
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4858
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2020-10-26 22:25:47 +00:00
Liu Xiaodong
a3c3c0b538 thread: fix warning caused by intr
Fixes issue: #1650

Change-Id: I8935d439fb7d1d1c896ef297baa53db0d2cd538f
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4869
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-26 11:49:03 +00:00
Liu Xiaodong
47962dc7a3 bdev_aio: fix interrupt mode notify error
Change-Id: Ie4492aa33028e8090da96e2b592b20293d694120
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4865
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Community-CI: Mellanox Build Bot
2020-10-26 11:49:03 +00:00
Alexey Marchuk
e385cafa72 nvme: Don't log an error when we can't resubmit all requests
In TCP NVME initiator with zero copy enabled requests might be
completed asynchronously - out of qpair_process_completions
context. At the same time we calculate requests completed
asynchronously so that generic NVME layer can resubmit
queued requests after calling qpair_process_requests (or
poll_group_process_requests).
But there is a time gap between async request complete and
qpair_process_completions and the user can submit new IO
thereby decrease the number of free TCP requests. That means
that there might be less free requests than we excpected when
we try to resubmit queued requests.
The solution is change ERRLOG to DEBUG log since it is not a
fatal case.

Change-Id: If045ecd331cc6693e8ef450d8e15432dfa5d8812
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4859
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2020-10-26 11:48:46 +00:00
Tomasz Zawadzki
214b9847d0 version: SPDK 21.01 pre
This is first commit that should go into latest SPDK
after the code freeze for SPDK 20.10.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ie7dc39553206ba747dc83ab6af7ddc2fc25d71eb
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4854
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-10-23 23:21:14 +00:00
Vitaliy Mysak
a2596f4dc2 event: add scheduler_set RPC
Add RPC that allows to change scheduler at runtime.

Change-Id: I008670f5e936bc25a0fbc923b826277d15343273
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3958
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-23 22:35:53 +00:00
Maciej Szwed
1b1e52cb47 event: Do not change scheduler while scheduling
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: Idb8cce814b90f7a7a4a06c81507ce79da91e312c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4856
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-23 22:35:53 +00:00
Maciej Szwed
b309e8cefa event: Scheduler plugin for using governors
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I2edc6e31eb91fff9e1b7f0a3a05cf7cd5dc79a58
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4163
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-10-23 22:35:53 +00:00
Maciej Szwed
68493f45af event: Add DPDK based governor
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: Ide0ac38497bc500886e26fd219cb120113896966
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4152
Tested-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-10-23 22:35:53 +00:00
Maciej Szwed
63933afd24 dpdk: Move submodule to latest commit
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I4faa1e65898f50ea9bac0de7f666efd52f6426a4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4849
Tested-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-10-23 22:35:53 +00:00
Maciej Szwed
c6adf304bd event: governor implementation
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I9aaa84261144145933d227976a5f89dc89732267
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4028
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-10-23 22:35:53 +00:00
Vitaliy Mysak
2cffc80079 event: add rebalancing infrastructure
Features:
- does not delay thread creation
- does delay thread deletion
- singlethreaded, but can be made multithreaded.
  By being singlethreaded, we don't waste time,
  because reactors are not paused during rescheduling,
  but we do make statistics less up to date

Change-Id: Ie5a7e8569bc32b3fd4bb887804dfbc3f5c2ea858
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3899
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-23 22:35:53 +00:00