Commit Graph

13096 Commits

Author SHA1 Message Date
Jacek Kalwas
92f29271d5 nvmf: add aqa property rsvd bits validation
Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: I824544f8b8faa040a5e0a6802c8ff611d3cb3b86
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3249
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2020-07-31 08:23:30 +00:00
Jacek Kalwas
e5bf2a1966 nvmf: fix aqa property validation
Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: I4de16c80fda5a39cfcdb15badb0d37acd1a5a21c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3102
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-07-31 08:23:30 +00:00
GangCao
51e8c2362b RPC: update the error message for current RPC state
This is to fix below issue:
https://github.com/spdk/spdk/issues/1516

Change-Id: Ibd7b4692b3d94fb7131aa0763d7fb7f6298d4101
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3565
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2020-07-31 08:21:37 +00:00
Tomasz Zawadzki
f6fcba38e8 test/nvmf: only disable test cases in top-level script
Some of the tests are skipped when using Soft-RoCE.
Test executed as part of run_test should never
skip a test, as it results in false assumption
that it was executed. See supplemental logs in CI output.

Before this patch bdevperf and srq_overwhelm (if executed),
resulted in misreporting this.

This patch moves all check for Soft-RoCE to top-level
nvmf test script.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I6d8941bbadaca2dbf8ab178e30e87d3786fea94c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3597
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-07-31 08:21:23 +00:00
Tomasz Zawadzki
31d2240873 test/nvmf: only check for rxe when testing RDMA
Using Soft-RoCE is only relevant when testing
RDMA transport. For all other cases, we might
treat the IP address as non-rxe.

This simplifies the checks a little bit in couple places.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I996afb364070fbe107bbe97207952f99e4089308
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3596
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-07-31 08:21:23 +00:00
Seth Howell
79aac78398 test/nvmf: don't use Soft-RoCE in Shutdown tests.
Soft-RoCE sometimes returns non-spec compliant values
in the WC entries after the qpair has been disconnected.
This can result in segmentation faults. In order to avoid
this, don't test shutdown on Soft-RoCE.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I604ffaf38aa8da5efa349df0b1b51e7e391d17f8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3582
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-07-31 08:21:23 +00:00
Jim Harris
144068a962 doc: fix raid_level parameter for bdev_raid_create RPC
This parameter is now a string, not an integer.

Fixes issue #1518.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I5c187af69d7cdf1f60573a07bc1bacb58b5b78b5

Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3581
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-07-31 08:21:16 +00:00
Changpeng Liu
da60639f86 rocksdb: call spdk_app_stop before exit the process
For some other reason when the host didn't have a block
device e.g: "Nvme0n1", we will call exit(1) directly in
the process, the ASAN will report memory leak issue, so
before exiting the process, we call spdk_app_stop to
do the cleanup.

Fix issue #1422.

Change-Id: I7b7a05807625586bd5b97dc31f41e2b86e1b323f
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3586
Community-CI: Mellanox Build Bot
Reviewed-by: JinYu <jin.yu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-07-31 08:21:10 +00:00
Alexey Marchuk
6967fec6df nvmf/rdma: Submit recv to SRQ when AER is released
Currently we don't resubmit receive request associated with AER
request to SRQ. This leads to reducing of SRQ elements and may
lead to non responsive NVMF target.

Fixes #1507

Change-Id: Ie96f8c4be0202ae973e561ebe5ea28688a6a3b72
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3558
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michael Haeuptle <michaelhaeuptle@gmail.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-07-30 18:20:30 +00:00
Vitaliy Mysak
8e5d5b8ff6 doc: add bdevperf doc section
Describe bdevperf tool usage and its config file.

Change-Id: I3648e9fcf6eb9e332dadda0d73f52740a19d5ad8
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3602
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-07-30 18:19:10 +00:00
Jin Yu
7ef6d8dd63 virtio: fix scsi double free issue
During virtio_pci_dev_probe, if enum_cb fails, hw needs
to be released. But in bdev_virtio, if vdev fails after
initialization, it will enter the bdev destruction process
which call the modern_destruct_dev function and hw will
be released during the process. So we will encounter the
problem of hw being released twice.

Change-Id: I1e8116283cfd810dfb050f8928f4ecd4bb2d815b
Signed-off-by: Jin Yu <jin.yu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3566
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2020-07-30 08:43:23 +00:00
Jin Yu
79c7744efb virtio: fix virtio hw double free issue
During virtio_pci_dev_probe, if enum_cb fails, hw needs
to be released. But in bdev_virtio, if vdev fails after
initialization, it will enter the bdev destruction process
which call the modern_destruct_dev function and hw will
be released during the process. So we will encounter the
problem of hw being released twice.

Change-Id: Ifba35284c072355ba0e10428b597a1894d32d59e
Signed-off-by: Jin Yu <jin.yu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3564
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2020-07-30 08:43:23 +00:00
Seth Howell
ebba5a0c58 make: add rpath to the shared library build option.
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I9f39ef73dcd9eaed7e5ca9431ab33aa139d9d268
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3361
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-07-30 08:43:14 +00:00
Ben Walker
a9be2ce64a env: Rename pci_device_rte_hotremove to pci_device_rte_dev_event
Give this a more generic name. We're going to be using these
events for more than just hotremove coming up.

Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: Ia5356e9ab809807ba4d85ecc212a496e96012bce
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3559
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-07-30 08:43:06 +00:00
yidong0635
0d73f143b8 lib/iscsi: Put bits NAND or OR together.
We can OR some of the bits ,and setting them one time.

Signed-off-by: yidong0635 <dongx.yi@intel.com>
Change-Id: Ie919ef66f3b8deea125df47d2194130b364c302a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3489
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2020-07-30 08:43:00 +00:00
Tomasz Zawadzki
1350922d09 bdev/ocf: take additional reference for ocf_cache
Fixes #1498

When shutting down the application, it was possible to
reference stale ocf_cache pointer. This was the case
when two or more vbdev_ocf devices were based on top
of single cache bdev.

This issue did not occur outside of the shutdown case,
since RPC only allows deletion of the vbdev_ocf.
This erases on disk metadata and next run of the application,
would not detect such vbdev_ocf.

Shutdown meanwhile works different, by first stopping
the instance of running "ocf_mngt_cache" and later detaching
"core" devices (the ones being cached). This prevented
erasing the on disk metadata and allowed for restarted
application to detect vbdev_ocf.
See patch (1292ef2) for details.

Since references to ocf_cache are copied between vbdev_ocf
[see start_cache()], the reference count inside ocf_cache
was limited to original ocf_mngt_cache_start() and
management queue creation. First call into ocf_mngt_cache_stop()
released all references to ocf_cache. Leaving other
vbdev_ocfs pointing to released memory.

This patch works around this issue by increasing ref cnt
on ocf_cache for each vbdev based on top of it.
It allows to call into ocf_mngt_cache_stop(), but not
release the memory for ocf_cache until last vbdev.

Note:
A proper redesign here is in order:
- either rearranging structures to be based around single ocf_cache,
rather than multiple vbdev_ocf instances
- better use of OCF API to reduce book keeping logic in vbdev

There are plans to implement detach/attach in RPC,
so it should be a focus during that effort.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I560a7fbb1c052bf53970e655bdb60803c561a252
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3574
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-07-30 04:02:08 +00:00
Tomasz Zawadzki
868ba17780 bdev/ocf: simplify check for running cache instance
There are additional conditions which SPDK tracks,
that are known before issuing call to OCF.

Two main ones are:
- if vbdev->ocf_cache was not yet created [start_cache()]
- if the cache bdev was opened [attach_base()]

Both happen for the first cache bdev once. Then for
consecutive vbdev_ocf on same cache bdev, reference
will be copied.

This call will simplify checking both conditions.
Calling into OCF with NULL or stale ocf_cache pointer,
rightly so will cause issues with ASAN.

Related #1498

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ib202c15bda4cbbffa1516c69168e8bfb80370047
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3573
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-07-30 04:02:08 +00:00
Ben Walker
e12a4f6ec8 Revert "thread: add spdk_env_get_primary_core"
This reverts commit 6194cb2e15.

It's unclear whether we need to add a new API for the env layer
for upcoming work. Nothing currently uses it. When we have a clear
need, we can add this back in.

Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: I174276799d650a1365b37a737271a54a796cd455
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3561
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-07-29 14:58:44 +00:00
Maciej Wawryk
35599b776f pkgdep/git: Fix QAT permision error during install
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I813d48674885c78369ec6a7d17c4e147e10f0c0b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3530
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-07-29 14:58:19 +00:00
Ziye Yang
d2a194c4f0 nvme/perf: Do not use IORING_SETUP_IOPOLL
This flag only works for local device. If the device from the kernel
is getting from remote (e.g., /dev/nvme2n1 is from NVMe-oF target),
then it will not work for those kernel devices while using
IORING_SETUP_IOPOLL flag.

Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: Ide396de9f53b884c4d12af64693293d57fac9523
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3531
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-07-29 08:06:11 +00:00
Jin Yu
b45f293d57 vhost-blk: add resize bdev support
This will allow us to resize the backend bdev of vhost-blk
and notify the guest OS that the capactiy of virtio-blk
disk has been resized.

The spdk api entry is `spdk_bdev_notify_blockcnt_change`.
Any bdev if used as vhost-blk backend may need to implement
a rpc that calls this function.

Related DPDK patch has been merged and release in 20.02.
https://www.mail-archive.com/dev@dpdk.org/msg153365.html

Change-Id: I961c61de0fc03e210d776035a40f3a4adfa9b4f3
Signed-off-by: Li Feng <fengli@smartx.com>
Signed-off-by: Jin Yu <jin.yu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1468
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-07-29 08:05:31 +00:00
Seth Howell
0b1799cd98 nvme/transport: addd assert for transport.
Silences a KW error.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: Ifd8d6088a22de7c230d48751be2b3991d0649778
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3553
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-07-29 07:37:26 +00:00
Seth Howell
7177e94973 changelog: update with new 20.07 features.
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: If0db6036203f95efcf2e4e1776e6f6fc9a68d6bf
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3540
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-07-29 07:37:00 +00:00
Maciej Wawryk
817d077a10 scripts/pkgdep: Add liburing to install_all_dependencies function
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I84cbd8c7a5563b704349c95df0c83513dcf91c45
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3541
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-07-29 07:36:50 +00:00
Ziye Yang
27ee283487 pkg dependency: Add the libaio package install.
Some images (especially cloud/containers) may do not
contain this package in default. So we need to add this package
install support.

Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I2620026534009e8b6c320ecf61b8970947d5a9fb
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3218
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2020-07-28 09:03:52 +00:00
Seth Howell
91bca45725 changelog: alphabetize and consolidate 20.07 section
In preparation for the upcoming release.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I8f118289612365a8d2be7baecb20450ae31068c8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3539
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-07-28 09:03:41 +00:00
paul luse
8d059e7a18 accel: add API to cancel a batch sequence
Added to the framework as well as all 3 engines.  Needed by apps
in the event that they have to fail following the creation of a
batch, allows them to tell the framework to forget about the batch
as they have no intent to send it.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Id94754ab1350e5a969a5fd2306bd59c38f0a0120
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3389
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-07-27 22:43:40 +00:00
Jim Harris
751e2812bc nvme: do not abort reqs in multi-process cleanup path
When a process cleans up IO qpairs from another crashed
process in a multi-process environment, we must not try to
abort reqs for that IO qpair.  Any reqs will contain callbacks
for the crashed process which we must not try to execute in
a different process.

Fixes issue #1509.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I5e58cce7bdb86e3feb4084733815c086901f867e

Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3536
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-07-27 22:42:07 +00:00
paul luse
62b3b171cb module/compress: add new parm to RPC for create compress vol
To specify the desired logical block size. Must be 4K or 512.
If no block size is provided a default of 0 means to use the
underlying bdev block size. For cases where something other
than 4K or 512 is desired, format the underlying device
accordingly and don't specify a logical block size on creation
of the compress vol.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I58b71e210cfa77b3237c0c454585c734e2e22aea
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3177
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-07-27 22:39:29 +00:00
Anil Veerabhadrappa
03fe6a77c8 nvmf/fc/ut: Add missing function stub
Fix the compilation error by adding spdk_nvmf_request_complete()
  function stub.

Signed-off-by: Anil Veerabhadrappa <anil.veerabhadrappa@broadcom.com>
Change-Id: I564cdb16238d5c45c50895b8b2a512096789ba38
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3523
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-07-27 20:45:21 +00:00
Alexey Marchuk
3d1d4fcf54 nvmf/rdma: Send ibv async event to the correct thread
Since rqpair->qpair.group is set to NULL when we remove the
qpair from poll group, we fail to send event to qpair's thread.
This patch adds a pointer to io_chaneel to spdk_nvmf_rdma_qpair
structure and a function to handle poll_group_remove transport
operation. In this function we get io_channel from nvmf_tgt,
this channel will be used to get a thread for sending
async event notification. This also guarantees that the thread
will be alive while we are destroying qpair.

Change-Id: I1222be9f9004304ba0a90edf6d56d316d014efda
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3475
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-by: Seth Howell <seth.howell@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-07-27 17:19:44 +00:00
Maciej Wawryk
8991773390 test/bdevperf: test config file
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I6a92345e1c3fae1f7f8b77bc68e0f715e7ac9ed9
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3291
Community-CI: Mellanox Build Bot
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-07-27 17:10:31 +00:00
Darek Stojaczyk
6b41a08654 dpdkbuild: build dpdk with meson+ninja
Makefile support in DPDK was deprecated and will be removed soon,
so switch to the officially supported way of building DPDK -
with meson and ninja. Two new tools. Basically, our Makefiles
will invoke meson+ninja for DPDK, no other SPDK components are
affected.

Apparently DPDK wanted to move away from an octopus-like config
system and the ideology behind meson configuration is simple now:
build everything by default. Some PMDs can be explicitly disabled
with meson command line, but all libraries (both static and shared
versions) and test apps are built unconditionally.

How long does it take to build minimal DPDK with meson? Too much.
On my machine half of the total build time is spent on libraries
we don't need at all. (I have some hacks up my sleeve to disable
building those libraries - see the subsequent patch.) As for the
official way of building a minimal DPDK, there was a patch [1]
on dpdk mailing list to introduce more specific configuration,
but it was rejected:

> We talked about this a few times in the past, and it was actually one
> of the design goals to _avoid_ replicating the octopus-like config
> system of the makefiles. That's because it makes the test matrix
> insanely complicated, not to mention the harm to user friendliness,
> among other things.
>
> If someone doesn't want to use a PMD, they can just avoid installing it
> - it's simple enough.
>
> Sorry, but from me it's a very strong NACK.

Let's not follow that direction, hack the DPDK build system instead.

As for advantages of meson+ninja over Makefiles? I can't find any.
It's another build system that does a lot for you with some functions,
magic options, and a built-in dependency system. It seems nice if you know
the syntax, but it's another component that you need to learn, debug,
and possibly find bugs in (there's a lot of github issues open for meson).
I would compare it to CMake.

As for changes in this patch: rather that explicitly disabling
PMDs we don't need, specify a list of PMDs we do need and disable
everything else found in ./dpdk/drivers/*. This way we won't have
to disable the new PMDs as they're added to DPDK.

Meson configuration also sets RTE_EAL_PMD_PATH #define to a valid directory
with built PMD shared libs. When it's set, DPDK dynamically loads all shared
libraries inside. The drivers there depend on DPDK shared libs and fail to
load in static SPDK builds, so we disable them altogether by unsetting
RTE_EAL_PMD_PATH in the meson-generated config file - just like
DPDK Makefiles did. EAL checks for RTE_EAL_PMD_PATH being empty and skips
loading any external PMDs then. We do it for both static and shared libs.
We specify all PMDs at build time for now, so there's just no need to load
them dynamically.

We have three more hacks in our submodule:
 * disable building dpdk apps by commenting-out a line in dpdk/meson.build
 * disable building unnecessary libs (build everything that spdk *may*
   need)
 * build isa-l compress pmd with `-L[...] -lisal`. DPDK expects to find
   libisal with pkg-config. We don't want to prepare a pkg-config file,
   so comment-out a failing check in another meson.build file and provide
   isa-l through CFLAGS and LDFLAGS.

We also need to make some changes to our test/external_code. First of
all, -ldpdk is no more. Meson build generates a pkg-config file with all
libs, but we'll switch to it in a separate patch - for now just specify
all -lrte_ libs one by one. -Wl,--no-as-needed has to be added to some
test cases, otherwise rte_mempool_ring isn't loaded. We don't use any
APIs from this library, it only has a static constructor that provides
a few callbacks used by rte_mempool_create(). Also, since DPDK now builds
both static and shared libraries, we need to add -Wl,-Bstatic to force
using static libswhere required. It's only needed for DPDK libs, but we
use it for SPDK libs as well since there's no harm.

As for performance:
$ ./configure --enable-debug --with-crypto --with-reduce
$ time make -j40 -C dpdkbuild all
with meson:
real    0m8.287s
user    1m7.983s
sys     0m10.548s

before, with the old DPDK makefiles:
real    0m20.232s
user    0m55.921s
sys     0m16.491s

The subsequent builds are much faster too:
$ time make -j40 -C dpdkbuild all
meson:
real    0m0.876s
user    0m0.663s
sys     0m0.217s

makefiles:
real    0m10.150s
user    0m11.740s
sys     0m6.772s

[1] http://inbox.dpdk.org/dev/1a07d1cd59d84dce84e56c10fdabf5e5504560a6.camel@debian.org/

Change-Id: Ic65db563014100bafb12e61ee0530cc2ae64401d
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1440
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-07-24 22:06:11 +00:00
Tomasz Zawadzki
c8c899f492 dpdk: update submodule to DPDK 20.05
With update to DPDK 20.05 EAL requires rte_telemetry,
external make tests were missing specifying this lib.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I9f5eaea46b47da48fc81b4fff0199cd75ff6af78
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3483
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-07-24 22:06:11 +00:00
Tomasz Zawadzki
bc803d6bcc autobuild: disable scan-build on DPDK
There is no need to perform scan-build on DPDK,
as changes there are outside of SPDK repository.

This prepares for next change in series that
updates submodule to DPDK 20.05, with it own set
of failures to address.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Iedb143c81339f0d33fec9a5cb58b48e2651b0a4f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3522
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-07-24 22:06:11 +00:00
Tomasz Zawadzki
e63845a6df ocf/test: disable persistent metadata test
related #1498

Updating DPDK 20.05 exposes issue listed in #1498.

Most likely the device removal path has to be re-worked
to accomodate multiple devices on the same cache instance.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I28f71ef2c2e1d9c2d19202cd33a932182bfb122c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3521
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-07-24 22:06:11 +00:00
Darek Stojaczyk
cdffd22581 env_dpdk: move NVMe PCI driver definition to the nvme lib
Now that drivers can be registered from upper layers there's
no need to keep them centralized inside env.

(check_format.sh complains that spdk_pci_nvme_get_driver() shouldn't
start with the spdk_ prefix - to workaround that we move the function
declaration from one place in env.h to another - that's enough to
convince check_format it really is a public function)

Change-Id: If86aebd6c997349569c71430ec815b413eb44ef8
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3187
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-07-24 21:42:26 +00:00
Darek Stojaczyk
f425f16385 env: add spdk_pci_driver_register() to the public API
This allows SPDK apps to register new PCI drivers outside of
the env layer, enabling SPDK as a whole with new use cases.

Change-Id: I0c998a9ec249c3ca610b7b3b8b6caf616b16f64c
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3185
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-07-24 21:42:26 +00:00
Darek Stojaczyk
3498c0a886 env: add spdk_pci_get_driver()
An alternative to spdk_pci_*_get_driver();

Change-Id: I20a80b3c655a37fb1c76da21c2b70d5678041fab
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3186
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-07-24 21:42:26 +00:00
Darek Stojaczyk
e6decf684a env_dpdk: add DPDK-independent driver information to spdk_pci_driver
We want to make struct spdk_pci_driver public, so add env-agnostic
fields that define a driver directly to that struct.
PCI driver registration (potentially in upper layers) will only use
spdk_pci_id-s and spdk pci drv_flags, then those will be translated
to DPDK equivalents inside env_dpdk.

Change-Id: Ia24ecfc99ebf0f54f096eaf27bca5ed9c0dfe01d
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3183
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-07-24 21:42:26 +00:00
paul luse
acee02acc9 module/accel/ioat: fix bug with preparing a fill command for batch
The accel_fw API taks a uint8_t for fill pattern, the ioat build
command expects a full 64.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I55b33706a9e48fe7efd57a06816e11fdd8b7fa4c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3322
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-07-24 19:38:46 +00:00
paul luse
0cecfcb19b examples/accel/perf: replace mempool of tasks with lists of tasks
For performance.  Also allocate the accurate number of tasks instead
of just blindly doubling to account for batching.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I4e5accfd97477f908368f16d3fc1d8f47896a4d6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3321
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-07-24 19:38:46 +00:00
paul luse
f7d25cf7c7 lib/accel: remove dead code
Not needed with new scheme around accel_task in prior patches.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Ic6f16654db62fe1dc51b0822f0ff92261da4494c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3212
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-07-24 19:38:46 +00:00
paul luse
e8463f873f examples/accel_perf: updates to match the new accel_fw API
No longer allocate accel_tasks or convert to and from application
tasks and accel tasks.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Id2d598d79bebcef522c76c125e65b100266ced57
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3210
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-07-24 19:38:46 +00:00
paul luse
ee7e31f9ae lib/accel: remove the need for the app to allocate an accel_task
This was sort of a clunky interface requiring a couple of inline
functions in every app that wants to use the accel_fw moving
forward. By having the accel_fw public API accept a callback arg
instead of an accel_task combined with adding a pool of accel_tasks
in the accel_fw engine we can eliminate this.

After changing the parm to a cb_arg, changes were made to all accel_fw
interfaces to put cb_fn and cb_arg as the last parms in public and
private function calls.

Related bdev_malloc changes need to be in this patch in order to pass CI.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I2b75764e534562d91484a094c3352266156d8425
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3209
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-07-24 19:38:46 +00:00
Vitaliy Mysak
6e1528503a bdevperf: config file: threat -T as filename parameter
This will allow to override job config filename from CLI.

Note that -T is required when using job config that has
missing filenames, but then it is optional if job config is not used.

Change-Id: I7d392ff2d0554524cfcb5072182ec806013083de
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3452
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-07-24 18:53:47 +00:00
Vitaliy Mysak
74c1028c80 bdevperf: config file: handle 'rw' argument
Some global argument and some code from verify_test_params()
could be removed.

Change-Id: I923f5f23fafd31ff48049bd16476f01c801bfbb8
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3278
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2020-07-24 18:53:47 +00:00
Vitaliy Mysak
fb5c98cecd bdevperf: config file: support multiple bdevs per job
Change-Id: I2aab753eb092378fe61cba9e4ef54070248c4c25
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3252
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-07-24 18:53:47 +00:00
Vitaliy Mysak
13a0ac2e20 bdevperf: config file: simplify old constructors
Instead of calling bdevperf_construct_job() directly,
create a job_config instance that would express
the same semantics and then use
bdevperf_construct_config_jobs() to create and run actual jobs.
The goal is to unify methods and reduce code duplication.

Change-Id: I724e5788ea74868c9e9c8128d658e720d211f253
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3510
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-07-24 18:53:47 +00:00
Vitaliy Mysak
32b2d78a3c bdevperf: config file: handle 'offset' and 'length' parameters
This parameters allow for low cost refactoring
because with them we can simplify construct_multithread_jobs()
by creating instances of job_config during its invocation
and then handling those in construct_config_jobs().

Change-Id: I73057a9a65f58e48ac719f30e918d7a05ffc0f28
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3509
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
2020-07-24 18:53:47 +00:00