Commit Graph

13732 Commits

Author SHA1 Message Date
Jim Harris
d3b788e9ab nvme: continue probing ctrlrs even if one fails
It is possible that a single probe_ctx could be used
to probe multiple newly attached nvme controllers.  If
one of those controllers is removed during this process,
the rest of the controllers do not get probed and can
even get stuck in a zombie state.

It is better to just continue with probing the rest of
the controllers.

Fixes issue #1611.

Signed-off-by: Jim Harris <james.r.harris@intel.com>

Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4945 (master)

(cherry picked from commit ddf86600bb)
Change-Id: I4156ee8b50e8d52cfeee7224f210a58bb773e939
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4958
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-10-29 15:43:38 +00:00
Jim Harris
c64c931c52 env_dpdk: add rte_rcu library dependency
rte_hash depends on rte_rcu starting in upcoming
DPDK 20.11 release.  rte_rcu was only added in
DPDK 19.05 release, so we need to check if it
exists before linking it.

Fixes issue #1661.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4947 (master)

(cherry picked from commit f896aa6f10)
Change-Id: I7e343c6f964b03cc62484b57803a3bad00f80288
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4965
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-10-29 15:43:25 +00:00
Jim Harris
52d31fe78d dpdk: move submodule to build rte_rcu library
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4953 (master)

(cherry picked from commit f2196fcafb)
Change-Id: Ie38573774612ac7ec7cf23367d3233124acd1a66
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4964
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-10-29 15:43:25 +00:00
Jim Harris
3421560144 test/external_code: use variable to hold DPDK lib list
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4946 (master)

(cherry picked from commit d792356a5a)
Change-Id: Ie4a489926695cc56125f0e18796071f7730190f1
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4963
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-10-29 15:43:25 +00:00
Simon A. F. Lund
bfd7d22df5 examples/nvme_fio_plugin: fix log-msgs. in report_zones() and reset_wp()
In spdk_fio_report_zones(), log_err did not prefix messages with
"spdk/nvme", making it hard to determine who dumped the error-message.
In spdk_fio_reset_wp() log_err described the wrong function.

This change fixes the above.

Signed-off-by: Simon A. F. Lund <simon.lund@samsung.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4916 (master)

(cherry picked from commit 414500c9eb)
Change-Id: I41df6d451e88942806c8b5a3cf9a0902d98cb186
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4941
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Simon A. F. Lund <simon.lund@samsung.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-29 08:10:17 +00:00
Simon A. F. Lund
9c93d81be3 examples/nvme_fio_plugin: fix reset_wp()
When _reset_wp() received a range to reset, then the loop kept resetting
the first zone in the range.
Also, the processing of command-completion were re-using the same
'completion' state, thus a previous completion would short-circuit
command-completion such that it would never be processed.

This change fixes that.

Also, the reset-loop assumes that the given offset is a valid zone-start
LBA, a check is added to verify that and return -EINVAL if it is not.

Signed-off-by: Simon A. F. Lund <simon.lund@samsung.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4915 (master)

(cherry picked from commit 12a44d4745)
Change-Id: I1a1e4be2e1f67c2d8fecb5fc36a211b2dbb5a921
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4940
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Simon A. F. Lund <simon.lund@samsung.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-29 08:10:17 +00:00
Simon A. F. Lund
3a3cfb3292 examples/nvme_fio_plugin: help the user with max_open_zones constraints
When a device has resource-limitations such as the
maximum-open-resources (mor) and this threshold is exceeded, then IO
will fail upon completion. Such behavior is not the most user-friendly
way to tell the user that they should provide a value for the
fio-parameter 'max_open_zones'.

This change provides an arguably more user-friendly approach by checking
whether the device is limited and in case it is:

* Provide a default value for 'max_open_zones', inform the user, and
continue
* Verify 'max_open_zones' and in case of error inform the user and
return error

Signed-off-by: Simon A. F. Lund <simon.lund@samsung.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4914 (master)

(cherry picked from commit 906c2adb86)
Change-Id: I76cb045d560b9ec5701d97b82a62947af11960b6
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4939
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Simon A. F. Lund <simon.lund@samsung.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-29 08:10:17 +00:00
Niklas Cassel
8c6c009fce examples/nvme/identify: print ZNS data structures if supported
Print the Zoned Namespace Command Set Specific data structures,
if the namespace/controller supports them.

spdk_nvme_zns_ctrlr_get_data() returns NULL for a controller
that does not support the ZNS specific controller data struct.

Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4839 (master)

(cherry picked from commit 00d197dba2)
Change-Id: I0acd2695976fc598b61591989f612db35ac821db
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4942
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-29 08:10:03 +00:00
Niklas Cassel
4e9a9c4a22 examples/nvme_fio_plugin: fix zone reporting
All zone management receive helper functions (including
spdk_nvme_zns_report_zones()) are implemented to match the parameters of
the zone management receive function in the ZNS specification.

The documentation for spdk_nvme_zns_report_zones() states:
"param partial_report If true, nr_zones field in the zone report indicates
the number of zone descriptors that were successfully written to the zone
report. If false, nr_zones field in the zone report indicates the number
of zone descriptors that match the report_opts criteria."
This matches the description of the "Partial Report" bit in the ZNS spec.

Since the FIO function parse_zone_info() calls the io_ops->report_zones()
function multiple times, until all zones have been reported, it expects
the return from this function to represent the number of zones that were
successfully reported.

By setting the partial_report bit to false, the controller will return
the total number of zones, and since spdk_fio_report_zones() loops until
idx < report->nr_zones, and writes to zbdz[idx], the current code will
overwrite heap memory, since idx will take on index values that are out
of bounds for the memory allocated by the FIO function parse_zone_info().

Therefore, spdk_fio_report_zones() has to set the partial_report bit to
true when calling the NVMe level function spdk_nvme_zns_report_zones().

Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4871 (master)

(cherry picked from commit 1e83b640aa)
Change-Id: I8846711bfed4faadac0315b450158293cefa36f4
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4943
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-29 08:09:53 +00:00
Maciej Wawryk
fc9ae4f30f Modifications for using universal qcow2 image in tests
We want to replace few qcow2 images with one universal.
This commit contains:
 - change password in autotest.sh
 - change image path
 - change image name
 - use snapshot mode in hotplug.sh instead of copying base image

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4551 (master)

(cherry picked from commit c4c37f1cf1)
Change-Id: I75c457fe75f005b0ab43ca909be7886529ed115b
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4944
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-29 08:09:37 +00:00
Changpeng Liu
ea364da638 env/memory: use stack variable when unmapping the dma region
When enable Werror compile option with new kernel(v5.8), there is
following error reported due to the <linux/vfio.h> data structure
change(added a uint8_t data[] variable in new kernel), we can just
put the 'unmap' at the end of the data structure to fix the issue,
I think it's better to just use a stack variable instead.

CC lib/env_dpdk/memory.o
memory.c:63:36: error: field 'unmap' with variable sized type 'struct vfio_iommu_type1_dma_unmap' not
at the end of a struct or class is a GNU extension [-Werror,-Wgnu-variable-sized-type-not-at-end]
struct vfio_iommu_type1_dma_unmap unmap;
^
1 error generated.

Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4925 (master)

(cherry picked from commit 2c9b5b5af5)
Change-Id: Icf73a3c48a301e74b92b9ae2e2d8715262b2d056
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4938
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-29 08:09:24 +00:00
yidong0635
0e70cb48ab rocksdb/env_spdk: Fix unused warning.
Fuction takes one parameter to print,
others are unused.

spdk/lib/rocksdb/env_spdk.cc: In function
 ‘void rocksdb::base_bdev_event_cb(spdk_bdev_event_type, spdk_bdev*, void*)’:
/spdk/lib/rocksdb/env_spdk.cc:666:70:
error: unused parameter ‘bdev’ [-Werror=unused-parameter]
666 | base_bdev_event_cb(enum spdk_bdev_event_type type, struct spdk_bdev *bdev,
      |                                                    ~~~~~~~~~~~~~~~~~~^~~~
/home/yidong/spdk/lib/rocksdb/env_spdk.cc:667:12:
error: unused parameter ‘event_ctx’ [-Werror=unused-parameter]
  667 |      void *event_ctx)
      |      ~~~~~~^~~~~~~~~
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4929 (master)

(cherry picked from commit a474889bc6)
Change-Id: Ic1cf45443ab1dcdf38d1b9c6bdea2905e94df19c
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4933
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-10-29 08:09:09 +00:00
Liu Xiaodong
494848e3a9 reactor: check calloc failure in gather_metrics
A round of _reactors_scheduler_gather_metrics should be stopped
when there is calloc failure.

Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4882 (master)

(cherry picked from commit e2f773aafc)
Change-Id: Ic2220c561abb07a849ea37d3c88af3f6d5d1ffa1
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4923
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-28 15:03:17 +00:00
Alexey Marchuk
623d5cc456 nvme: Don't log an error when we can't resubmit all requests
In TCP NVME initiator with zero copy enabled requests might be
completed asynchronously - out of qpair_process_completions
context. At the same time we calculate requests completed
asynchronously so that generic NVME layer can resubmit
queued requests after calling qpair_process_requests (or
poll_group_process_requests).
But there is a time gap between async request complete and
qpair_process_completions and the user can submit new IO
thereby decrease the number of free TCP requests. That means
that there might be less free requests than we excpected when
we try to resubmit queued requests.
The solution is change ERRLOG to DEBUG log since it is not a
fatal case.

Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4859 (master)

(cherry picked from commit e385cafa72)
Change-Id: If045ecd331cc6693e8ef450d8e15432dfa5d8812
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4872
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-10-28 08:09:03 +00:00
Liu Xiaodong
0bcafaea56 lib/rocksdb: remove redundant linked blobfs_bdev
blobfs_bdev lib is already added into BLOCKDEV_MODULES_LIST
so it shouldn't be included by application who already
uses BLOCKDEV_MODULES_LIST or ALL_MODULES_LIST.

Fixes issue: #1654

Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4876 (master)

(cherry picked from commit b45788036f)
Change-Id: I46a272e4593e19cf14c3ed8b2965797443c37a0d
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4922
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-28 08:08:49 +00:00
Allen Zhu
b5005497db rpc/nvmf.py: pass zero values to SPDK when allowed
in_capsule_data_size/buf_cache_size/sock_priority/max_namespaces can be 0,
which should be passed in nvmf_create_transport/nvmf_create_subsystem commands.

Signed-off-by: Allen Zhu <allenz@mellanox.com>
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Signed-off-by: Evgeniy Kochetov <evgeniik@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3815 (master)

(cherry picked from commit 96ab62802c)
Change-Id: Ib557cf9f20f7ec2c0b3c31156cd79dbd670ce7e7
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4921
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Allen Zhu <allenz@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-28 08:08:37 +00:00
Alexey Marchuk
839af8867e nvmf/tcp: Align recv_buf_size to MIN_SOC_PIPE_SIZE
If the user decided to disable ICD then we have several side effects:
1. SPDK prints several warnings/errors
2. SPDK doesn't create recv pipe and doesn't set SO_RCVBUF socket option.

I think that we should not rely on ICD only when we create recv pipe or
set SO_RCVBUF since data may be transferred in sgls via R2T/H2C and
we still need recv_pipe and SO_RCVBUF for better performance.
Alternative option is to set recv_buf_size as a maximum between
ICD and io_unit_size

Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4755 (master)

(cherry picked from commit c1fbbfbe56)
Change-Id: Ida71ecc099f9a9355e4617f13315a341872d1cb3
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4920
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-28 08:08:37 +00:00
Alexey Marchuk
bf602f12ca nvmf/tcp: Support ICD for fabric/admin commands
According to the SPEC we should support up to 8192 bytes
of ICD for admin and fabric commands. Transport configuration
parameter in_capsule_data_size is applied to all qpair types -
admin and IO. Also we allocate resources when we get a connection
request, so we don't know qpair type at this moment.
Create a list of buffer in TCP poll group to support ICD up
to 8192 bytes when configuration ICD is less than this value.
The number of elements in this pool is hardcoded, it is planned
to add a new configuration parameter later.

Fixes issue #1569
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4754 (master)

(cherry picked from commit 85fa43241b)
Change-Id: I8589e3e2ea95d515f5503c6de7c1ee40aaf7b6da
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4886
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-10-28 08:08:37 +00:00
Liu Xiaodong
ad97082bd0 bdev_aio: fix interrupt mode notify error
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4865 (master)

(cherry picked from commit 47962dc7a3)
Change-Id: Ie4492aa33028e8090da96e2b592b20293d694120
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4873
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-10-27 15:44:16 +00:00
Jim Harris
05216cb7bf event: deprecate opts.config_file member
Just always put the config file name in json_config_file,
since we now only support JSON.

If user specifies both -c and --json, it will just take
the latter of the two.  This is similar to if the user
specified --json twice.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Idc21d73acf0e190eda57a7b0c5d9bcfa14e87030
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4858
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
(cherry picked from commit c31ad66893)
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4894
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
2020-10-27 15:44:03 +00:00
Liu Xiaodong
69b16a000f thread: fix warning caused by intr
Fixes issue: #1650

Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4869 (master)

(cherry picked from commit a3c3c0b538)
Change-Id: I8935d439fb7d1d1c896ef297baa53db0d2cd538f
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4874
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-10-27 15:43:46 +00:00
Vitaliy Mysak
a2596f4dc2 event: add scheduler_set RPC
Add RPC that allows to change scheduler at runtime.

Change-Id: I008670f5e936bc25a0fbc923b826277d15343273
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3958
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-23 22:35:53 +00:00
Maciej Szwed
1b1e52cb47 event: Do not change scheduler while scheduling
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: Idb8cce814b90f7a7a4a06c81507ce79da91e312c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4856
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-23 22:35:53 +00:00
Maciej Szwed
b309e8cefa event: Scheduler plugin for using governors
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I2edc6e31eb91fff9e1b7f0a3a05cf7cd5dc79a58
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4163
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-10-23 22:35:53 +00:00
Maciej Szwed
68493f45af event: Add DPDK based governor
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: Ide0ac38497bc500886e26fd219cb120113896966
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4152
Tested-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-10-23 22:35:53 +00:00
Maciej Szwed
63933afd24 dpdk: Move submodule to latest commit
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I4faa1e65898f50ea9bac0de7f666efd52f6426a4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4849
Tested-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-10-23 22:35:53 +00:00
Maciej Szwed
c6adf304bd event: governor implementation
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I9aaa84261144145933d227976a5f89dc89732267
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4028
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-10-23 22:35:53 +00:00
Vitaliy Mysak
2cffc80079 event: add rebalancing infrastructure
Features:
- does not delay thread creation
- does delay thread deletion
- singlethreaded, but can be made multithreaded.
  By being singlethreaded, we don't waste time,
  because reactors are not paused during rescheduling,
  but we do make statistics less up to date

Change-Id: Ie5a7e8569bc32b3fd4bb887804dfbc3f5c2ea858
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3899
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-23 22:35:53 +00:00
Vitaliy Mysak
7148f33360 event: adopt static scheduler to the new API
Move current scheduler to it's own module
and make use of new API.

Change-Id: I4928aed82603d51de01194c9650709e814f7f61b
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4054
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-23 22:35:53 +00:00
Maciej Szwed
0d8a4af849 event: Add lw_thread helper functions required for scheduler
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: Ifd21adad6116d75170d5a6ff4b6d03470644feab
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4338
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-23 22:35:53 +00:00
Maciej Szwed
2a146cd936 event: define scheduler API
Add structures and function for new scheduler API.

Change-Id: I8e379db62189eb7c36092265c137f07368650e9b
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4053
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-23 22:35:53 +00:00
Tomasz Zawadzki
9e89645b61 CHANGELOG: add missing entries for 20.10
Added missing entries from browsing header and RPC changes.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I326e63cc14278adebd58d1dadd5e444dd2bfc554
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4853
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-23 18:24:12 +00:00
Tomasz Zawadzki
75fdd05eea CHANGELOG: fix formatting and typos for 20.10
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ibbe6293955e74c236f5a45936aadfcac9879f4fe
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4852
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-23 18:24:12 +00:00
Tomasz Zawadzki
078ef4e5e6 CHANGELOG: move command log entry to correct SPDK version section
Patch introduced during SPDK 20.10 development put this addition put
the changelog entry into SPDK 20.07 section.
(198a3ad) nvme: if no command set is requested, use the most reasonable default

Move it to the right place.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I0f3ddd1a551c8ba3ad2e272dea82ab2179063599
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4851
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-23 18:24:12 +00:00
Tomasz Zawadzki
c7df7a4ce5 CHANELOG: alphabetize 20.10 section
This patch does not add/modify/remove entries.

All sections are put in alphabetic order and lower cased.
Some of the repeated ones are consolidated.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I5691f2de1e8de5efa00002361904bd63bddf2ac4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4850
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-23 18:24:12 +00:00
Liu Xiaodong
147000aa7b interrupt_tgt: add lvol, blobfs and basic bdevs
Logical volume, blobstore and blobfs can directly run in
interrupt mode.
Some bdev modules don't have pollers, so they can be directly
runnning in interrupt mode.
interrupt_tgt can create lvol bdevs and some other type bdevs

Change-Id: I9ad068393c6ff31bb3aafb3e58a41b0e0aeec9af
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4276
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-23 16:23:48 +00:00
Liu Xiaodong
54a7d73ad1 nbd: apply interrupt
If interrupt mode is set, related poller functions will be
registered to interrupt_handler instead of poller.
interrupt_tgt can run IO with linux nbd.

Change-Id: I39ecf1efa10be76419fb0d25713ea457a5a53b37
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4274
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-10-23 16:23:48 +00:00
Liu Xiaodong
ad8c05dc7c bdev_aio: apply interrupt mode
If interrupt is set, related poller functions will be
registered to thread interrupt  instead of poller.
interrupt_tgt can create aio bdev now.

Change-Id: Id777a55c8b7dc7a220f94ab837f7039ecea4fcb4
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4273
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-10-23 16:23:48 +00:00
Liu Xiaodong
d8618de0df interrupt: add interrupt_tgt app
interrupt_tgt is similar with vhost_tgt, but it is constructed by
libaries supporting event interrupt mode with interrupt functions.
Some basic bdev libraries are linked in it.

"interrupt_tgt -E" will run the app in interrupt mode.
At start, it can work with SPDK RPC methods.

Change-Id: I3a9b253a4657758fc670af835e5bed5e53f412dc
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4271
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-23 16:23:48 +00:00
Liu Xiaodong
4bf6e4bb32 interrupt: apply fd_group in thd and reactor
Each reactor and each thread is assigned with one fd group.
At the same time, each thread is treated as one interrupt source
registered into its corresponding reactor.
The egrp function for reacotr is the only block point waiting
for events.

Change-Id: Id092e66591b07b445342f8ae16dc218d28887427
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4269
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-10-23 16:23:48 +00:00
Liu Xiaodong
7d19e50a54 reactor: extract reactor_post_process_lw_thread
Change-Id: I2761db384f78529525b7f1eb3c9c959a7b885ede
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4756
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-10-23 16:23:48 +00:00
Liu Xiaodong
c5d1e7f171 interrupt: new fd_group in lib/util
Interrupt is aiming to support interrupt
mode in SPDK application framework.
fd_group is implemented by epoll on linux.
fd_group can be inserted into thread and reactor
to do epoll_wait internally in order to avoid polling

Change-Id: I9077ad648a97fbd68fd46f43de2e16440bedab0b
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4268
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-23 16:23:48 +00:00
Maciej Wawryk
4ffb15c133 test/vhost: fix shared.sh test to run in vagrant box
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I6d255ce3250b97895bd2bb02256f92e14a3c4245
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4666
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-10-23 16:20:08 +00:00
Tomasz Zawadzki
02d4c2b32a ut/iscsi: remove unused legacy config files
The tests did not use the legacy files anyway.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Icffedb7cc7a8574f0f801883b2d9655515cab687
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4855
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-10-23 16:12:18 +00:00
Michal Berger
dec533bf17 pkgdep/git: Fix git version check
f62834a6bb forgot to actually retrieve version string. Also, use
common function for comparing the strings.

Change-Id: I022ab54ea5b4edf0b596776dd35ca8637506ee46
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4833
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-10-23 14:28:50 +00:00
Michal Berger
62df2c1e81 pkgdep/apt-get: Add libpmemblk-dev package
libpmemblk-dev - provides /usr/include/libpmemblk.h (--with-pmdk)

Change-Id: Ie92dc1006cde972e7f77630577de18a016f4b3be
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4816
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-10-23 14:28:50 +00:00
Michal Berger
22213bb777 pkgdep/apt-get: Move ubuntu-dependent checks to os/ubuntu
Change-Id: I11b4bb2a0db20e6f86749cd951221a3697a72ed6
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4815
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-10-23 14:28:50 +00:00
Karol Latecki
17f3add0a4 autobuild.sh: exclude custom dpdk from scanbuild path
Change-Id: Icc668defa2670561f1160a78f68e76452e0862fc
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4774
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-23 13:47:43 +00:00
Karol Latecki
b7e4207b29 autobuild.sh: cherry pick dpdk fix for disabling qat_asym driver
Use the same fix that we already have in spdk/dpdk
repository:
c2c273d5c8

Change-Id: I136d3e90a048c039cb101a190f10dabd043875ab
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4828
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-10-23 13:47:43 +00:00
Ben Walker
2ec88c4f8c nvmf: Don't fail a property set command with invalid bits
If the initiator sends a property set command to a valid register, but
the value contains bits that are reserved by the NVMe specification,
don't fail the whole command. Just log an error message.

Previously, any valid bits set would take effect but then the command
would also fail. That confuses the initiator and thinks it must retry
the full property set.

Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: I566bc68c4469b9f41c69902e276f825c86683075
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4814
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-10-23 13:47:35 +00:00