Commit Graph

18251 Commits

Author SHA1 Message Date
Konrad Sztyber
e1068a862c sma: configure crypto when attaching volumes
Crypto parameters are now checked and crypto is configured when a volume
is attached.  Since configuring crypto can lead to creating new bdevs on
top of the attached volume, each device manager will also need to be
changed to retrieve the bdev through the CryptoEngine.get_crypto_bdev()
interface.  This will be done in subsequent patches.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Idb5f804cf88aa5e34bbee6817acdb8f3a42a2320
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13870
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: <sebastian.brzezinka@intel.com>
2022-09-19 19:43:35 +00:00
Konrad Sztyber
cc3f842cd1 sma: initial crypto definitions
This patch defines the interface for crypto engines, which provide
support for configuring crypto on a given volume.  Only a single crypto
engine can be active at a time and it's selected in the "crypto" section
of the config file.  Similarly to device managers, external crypto
engines can be loaded from plugins.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Id942ef876e070816827d7ad1937eb510a85c8f8d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13869
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: <sebastian.brzezinka@intel.com>
2022-09-19 19:43:35 +00:00
Konrad Sztyber
b30edf643d sma: volume crypto interface
This interface will make it possible for users to configure data at rest
encryption for volumes exposed through SMA.  The crypto parameters are
part of volume parameters specified when a volume is attached.  Once a
volume is attached it is not possible to change it.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I795cfab67523ace47d366d50e33b253737716668
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13868
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-09-19 19:43:35 +00:00
Michal Berger
11d13b85e6 test/nvmf: Test different abort qd sizes under SPDK and kernel targets
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I79ede7670477ed0d8eb38fa3eced4a9b70c64620
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8962
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-09-19 13:13:42 +00:00
Tomasz Zawadzki
dcd1fb796c example/perf: move allocating ns_worker
Next patch in series will add another caller for
the allocate_ns_worker.

Change-Id: I60281b519265b27d661c2b1ca94e09dc68dbbf57
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14414
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2022-09-19 13:13:09 +00:00
Tomasz Zawadzki
b6e9e1abb4 example/perf: comment the ns to worker association
This documents the logic below and motivation behind
(70db0e1)nvme/perf: Add support for multiple cores per device.

Change-Id: I949b794355f871c0f995b946622dbe77295d922b
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14413
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2022-09-19 13:13:09 +00:00
Liu Xiaodong
762db2a4f4 vhost: register memtable once if unchanged
Move memtable register out of start_device, into
post_handler for vhost-msg SET_MEMTABLE;
And unregister memtable in destroy_connection
instead of destroy_device

If memtable info not changed in the msg, then we
don't need to register it multi times.

Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Change-Id: I0f8c76c1ee43b6f981d703beeba92da5dac4dbd6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14263
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2022-09-19 13:12:24 +00:00
Tomasz Zawadzki
69bcb185f3 bdev/xnvme: save xnvme bdevs from runtime
config_json callback is used to preserve all
bdevs that were created during application runtime.

xnvme bdev module was missing this callback.

While here, io_mechanism is now saved in the bdev_xnvme
structure for reference in the config_json.
Haven't seen an option to pull this from xnvme itself.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I1a88ad2bb761f589d214fec8f0690c38572824d6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14116
Reviewed-by: Michal Berger <michal.berger@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-09-19 13:12:07 +00:00
wanghailiangx
69b4e8b17f test/nvme: Replace -s with -o for "io size in bytes"
For most of apps, -o is for "io size in bytes",
-s("io size in bytes") was changed to -o a long time ago.
-s is now for "memory size in MB for DPDK".

Change-Id: I0977d4a7bc30a4f2e135707ff6e77b6254240fb9
Signed-off-by: wanghailiangx <hailiangx.e.wang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14533
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-09-19 13:11:55 +00:00
Krzysztof Karas
905e519adc autotest_common.sh: increase read timeout inside rpc_cmd
Currently we use 5 seconds as a timeout inside rpc_cmd's
"while read" loop, which is enough for a system with
a single NVMe. That value is insufficient for machines
with more devices and causes the timeout, erroring out
the tests (verified with the same number of devices,
as in the CI node with the most NVMes available - 4).

For that reason increase the read timeout to 15 seconds.

Fixes #2641

Change-Id: If7e9fd169a75ea16ec243c63821a4e5d70995063
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14525
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-09-19 13:11:30 +00:00
Jim Harris
88055b422f test/spdkcli: reduce amount of HUGEMEM memory requested
We do not need to force usage of 4096MB of memory for these
tests.  Let's just remove how that is being forced.

This allows us to also remove the setup.sh call from
the spdkcli test common.sh script.  Its only purpose
was to increase the amount of memory allocated to satisfy
the spdkcli tests, but now that the spdkcli tests aren't
forcing so much memory to be allocated, we can remove
that too.

Fixes issue #2694.

Note: these limits were in the original commit for the
test scripts - 7e8206c - but it is not clear why they
were added instead of just relying on DPDK dynamic
memory allocation..

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I2210765e69117205a039aa6bc468efa5d5d537cb
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14520
Reviewed-by: Michal Berger <michal.berger@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-09-19 13:11:06 +00:00
Changpeng Liu
f167986b6c test/vfio-user: skip NVMeoF target coremask for VMs
For vfio-user tests, SPDK NVMeoF target uses "0xF"
as the core mask, so we should avoid to use these
cores when starting VMs.

Fix issue #2600.

Change-Id: I1927cd70670b30d59944d5571bcfc37c9e7e35d6
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14485
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-09-19 12:47:37 +00:00
Konrad Sztyber
35a71c7748 sock/ssl: free SSL resources in case of failures
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ifba581c92269c675b03b67586b1fea0b096f9e95
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14511
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-09-16 13:33:17 +00:00
jun.ran
488d6e84c4 bdev_aio: handle unexpected res=0 correctly
If res = 0, do not pass this value directly to bdev io completion function,
otherwise it will be treated as a successful completion.
Fix issue: 2679

Signed-off-by: jun.ran <ranjunsh@163.com>
Change-Id: Ice525de1bb5b0ada9ac30d3b59e5c731419efe2f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14364
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-09-16 13:32:40 +00:00
Xinrui Mao
c3f628f141 lib/nbd:export bdev flush and trim ability
Fix mkfs fail when using lvol as backend of nbd.Predefined
NBD_FLAG_SEND_FLUSH and NBD_FLAG_SEND_TRIM are defined by default,
so the operations of trim and flush are supported,but in fact lvol
doesn't support trim and flush operations.Therefore add judgement for
NBD_FLAG_SEND_FLUSH and NBD_FLAG_SEND_TRIM to check.

Signed-off-by: Xinrui Mao <xinrui.mao@intel.com>
Change-Id: I3d21034d12a038c8fc694d3383028103239ea6bd
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14099
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2022-09-16 13:32:13 +00:00
Michal Berger
7a4cb7bc1c perf/vhost: Allow to pass multiple FIO configs to the benchmark script
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I96df5f21f04b9196083f91783f8dd9cc4c6920d1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13688
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-09-15 20:15:57 +00:00
Michal Berger
971451a84a perf/vhost: Allow to add extra fio config through $fio_extra_conf
$fio_extra_conf can either point at an existing file or be an
attached stdin the fio_conf() should read from.

Signed-off-by: Michal Berger <michallinuxstuff@gmail.com>
Change-Id: If902262b0100fb056491928b3f667ab15e6e0024
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12429
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-09-15 20:15:57 +00:00
Michal Berger
0ee37d256a perf/vhost: Add io_uring related options to fio configuration
https://trello.com/c/EFPM29gq/399-use-iouring-in-vhost-performance-tests

Signed-off-by: Michal Berger <michallinuxstuff@gmail.com>
Change-Id: I9bf8d9a121ffc1f52273fbe863bf9918e7399e59
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12428
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-09-15 20:15:57 +00:00
Michal Berger
008f8a8617 perf/vhost: Replace run_vhost_test.py with a simple bash wrapper
The wrapper glues execution of the perf/vhost tests in as simple form
as possible. The arguments are meant to be passed either via env or
the cmdline in a keyworded argument fashion. All defaults are
embedded inside the configuration which is built by the wrapper
itself.

https://trello.com/c/HSoRtQkO/401-deprecate-vhost-performance-python-helper-script

Signed-off-by: Michal Berger <michallinuxstuff@gmail.com>
Change-Id: Iad94bfdd65cb8871b819351ddae65e647621fb30
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12400
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-09-15 20:15:57 +00:00
Michal Berger
6872c67e4f perf/vhost: Add NUMA-aware cgroup setup for the VMs
Each VM will be placed under dedicated cgroup pinned to proper set of
cpus and mem nodes as defined in the provided configuration.

Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: Id5d284ee9cbde541b8344ab38c3f77aa5430b1b1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13704
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-09-15 20:15:57 +00:00
Michal Berger
3d5e27df7d test/cgroups: Add helper functions for more granular setup
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I6b6baae849a472599adc36a04d268a7a65979b41
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13703
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-09-15 20:15:57 +00:00
Karol Latecki
1f7041b3fb docker: read architecture from uname
Remove hardcoded "x86_64" and read architecture
using "uname".

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I155876df50939bd766f242496bb73c49332dc940
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14515
Reviewed-by: Michal Berger <michal.berger@intel.com>
Reviewed-by: Boris Glimcher <Boris.Glimcher@emc.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-09-15 20:12:48 +00:00
Karol Latecki
602a900006 docker: remove pre-install script
Remove "pre-install" script used for building
"build_base" image and move it's content into
Dockerfile. This results in resulting more build
layers, but makes rebuilding easier. We can now
rebuild or update the image starting at some later
cached layer instead of starting pre-install from
scratch every time.

Change-Id: I78b46cda4bb815ac8cf72c8251a3cc585b373acc
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14158
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Boris Glimcher <Boris.Glimcher@emc.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Michal Berger <michal.berger@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-09-15 20:12:48 +00:00
Karol Latecki
ba453fbecf docker: update base to Fedora 35
Update base container image to more recent Fedora 35
version.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I879febc67ffe4c522e5138518d4182b5db99a570
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14157
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: Michal Berger <michal.berger@intel.com>
2022-09-15 20:12:48 +00:00
Karol Latecki
e8a40ada16 docker: make sure root owns spdk repo directory
Make sure copied SPDK repository is fully owned by
root user when performing the image build.
This is to avoid git "safe.directory" feature
from compalining.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I4ead2dd23198f79707b240c1c7e7470a68980d85
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14156
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Michal Berger <michal.berger@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-09-15 20:12:48 +00:00
MengjinWu
48312019c8 nvme/tcp: Remove duplicate code in nvme_tcp_read_pdu
Signed-off-by: MengjinWu <mengjin.wu@intel.com>
Change-Id: I63f51ecba2b4d40579d2592d2c85a7aefdacf7e7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14503
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-09-15 19:25:02 +00:00
MengjinWu
31fc5f196f nvme/tcp: simplify state change function
state change function do not need to use swtich to do some work.

Do memset in state machine.

Signed-off-by: MengjinWu <mengjin.wu@intel.com>
Change-Id: Ie66454d8f31860f403171f20858a6b4a24e3c76f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14502
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
2022-09-15 19:25:02 +00:00
Aleksey Marchuk
7a7f21b6fe init: Avoid calling RPC methods twice
Some methods are allowed to be run in both
STARTUP and RUNTIME states and current implementation
calls such methods twice. That can be a problem
in some cases, so use the new spdk_rpc_get_method_state_mask
function to skip such methods in RUNTIME state.

Signed-off-by: Aleksey Marchuk <alexeymar@nvidia.com>
Change-Id: I0a109805db428f60072a8c82161805dcde763da7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14407
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-09-15 08:25:18 +00:00
Aleksey Marchuk
515419ac66 rpc: Add API to get method state mask
The new API will be used in the next patch
to prevent calling metods for the seconds time
when subsystem is initialized with config file

Signed-off-by: Aleksey Marchuk <alexeymar@nvidia.com>
Change-Id: I60ac8196e46ccb3b22b3af0607e1ba35a11a66a6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14406
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-09-15 08:25:18 +00:00
Damiano Cipriani
67bb33160a vbdev_lvol: Implement SEEK_[DATA,HOLE] io type
Two functions have been added to implement bdev io type
SEEK_DATA and SEEK_HOLE. Blob functions to find next
[un]allocated io_unit are used

Signed-off-by: Damiano Cipriani <damiano.cipriani@suse.com>
Change-Id: Ic3be3c0c86bd3010a23ba2681f0f00c62abcaaba
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14362
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-09-15 08:23:56 +00:00
Damiano
6defafc913 bdev: Add functions to [hole,data] seek
These functions start from a given offset and seek for next
data or for next hole. For bdevs that do not support seeking,
it is assumed that only data and no holes are present

Signed-off-by: Damiano Cipriani <damiano.cipriani@suse.com>
Change-Id: I6bc831970223333b25683f60ce3fcbbfebb5bb81
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14361
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Community-CI: Mellanox Build Bot
2022-09-15 08:23:56 +00:00
Damiano
d8a3dee1c1 blob: Add functions to find [un]allocated io_unit
These functions start from a given offset and seek for first
io_unit belonging to an allocated cluster or first io_unit
belonging to an unallocated cluster

Signed-off-by: Damiano Cipriani <damiano.cipriani@suse.com>
Change-Id: I0c632e2b3dfd2e96aa22e21796e25a36f2f55f9f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14360
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
2022-09-15 08:23:56 +00:00
Damiano Cipriani
ddf5a8da90 blobstore: Add function to get io_unit per cluster
This function returns the number of io_units per cluster

Signed-off-by: Damiano Cipriani <damiano.cipriani@suse.com>
Change-Id: I8f33d24a63876a0a918830b9eeaa69a91ff21193
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14431
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Community-CI: Mellanox Build Bot
2022-09-15 08:23:56 +00:00
Boris Glimcher
2de485346e sock/ssl: move SSL_CTX creation to accept()
This will allow to have context per connection.
And free context when connection closes.

Fixes #2689

Change-Id: Ic4e9adfa3f1bd8574b9ccf75ff42c4f3bd442b26
Signed-off-by: Boris Glimcher <Boris.Glimcher@emc.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14443
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-09-14 07:45:09 +00:00
Boris Glimcher
35f7f0ce1e nvme/tcp: Allow to choose SSL socket implementation
Adding `psk` field to `spdk_nvme_ctrlr_opts`

Adding `psk` parameter to `bdev_nvme_attach_controller` RPC

Change-Id: Ie6f0d8b04ce472e6153934e985c026acded6cdfc
Signed-off-by: Boris Glimcher <Boris.Glimcher@emc.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14046
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-09-14 07:44:53 +00:00
Amir Haroush
ef65d8467c bdev/raid: fix flush on raid0
the calculation (offset_blocks + num_blocks - 1) didn't check if num_blocks is 0
which it is in case of flush.
in flush case, offset_blocks was also 0, so we got (-1) and we got big unsigned number.
just replace it with (offset_blocks + num_blocks - (num_blocks > 0)) to solve the issue.
NOTE this is only fixing the wrong math, there might be more issues that outside the scope of this commit
for example, if flush(x, y) called, no bdev implementation really use those values, and all bdevs just flush the whole disk
also, a convention is that fluch(0, 0) should flush it all (but like I said before, all bdevs flush the whole disk anyway)

Change-Id: I7e991653bc3050349dc155365b2c37ecc2d6b24c
Signed-off-by: Amir Haroush <amir.haroush@huawei.com>
Signed-off-by: Shai Fultheim <shai.fultheim@huawei.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13579
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-09-14 07:44:19 +00:00
Michal Berger
f352e9f477 pkgdep/git: Skip -pre tags
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I85daca195e0b7c272bc18a610b20e25408989d5b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14496
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Kamil Godzwon <kamilx.godzwon@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-09-14 07:43:47 +00:00
Jim Harris
a8074147b5 thread: note that spdk_thread_get_last_tsc(NULL) is allowed
The code has always supported this, it just wasn't
documented in the header file.  If caller passes NULL,
it will use the current thread to get the TSC.  This
is actually the most common case.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I39ba8260f0cb277b9272441bd336294c3327d5de
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14442
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
2022-09-14 04:35:20 +00:00
Kefu Chai
39ecb61ade event: pass "const struct option*" to spdk_app_parse_args()
before this change, we cannot pass a `const struct option*` to
spdk_app_parse_args() even the callee does not mutate the value pointed
by the pointer. in other words, we are not able to write something like:

static const option g_options[] = {...};
// ...
spdk_app_parse_args(argc, argv, &opts, "",
  g_options, app_parse_arg, app_usage);

after this change, the requirement of the type of the `option` argument
is relaxed, so we can pass a `const struct option*` to this function
now.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
Change-Id: I8794fcf92090f538743850a28ef4a2a8c357f121
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14082
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-09-13 10:48:58 +00:00
MengjinWu
12807c5bc6 lib/nvmf: Do one memset per new PDU recv
While waiting for a new PDU, target will not do too many useless
memcpy.

Signed-off-by: MengjinWu <mengjin.wu@intel.com>
Change-Id: Ie0825c2b1e44444b210040c4a1761010e0e4cfe5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14444
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2022-09-13 07:29:38 +00:00
Changpeng Liu
6045458cb6 test/bdevperf: apply g_abort to job->abort
Also give an error log when the g_timeout_in_sec parameter
isn't set but g_abort is set.

Fix issue #2643.

Change-Id: Iafd9e94952319b89891e37bc6e9e0db6892ddd7d
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14435
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-09-13 07:29:24 +00:00
Yifan Bian
c6824c7944 bdev/rbd: return a error value(-1) when create RBD bdev on non
existing RBD image

Fix issue #2690

Change-Id: Ia41b32a7f54be7e9a63c9b6284fba7d8ad4eebd6
Signed-off-by: Yifan Bian <yifan.bian@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14484
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2022-09-13 07:28:57 +00:00
Krzysztof Karas
b4e9ce466c spdk_top: Add status columns to THREADS and CORES tabs
We now have a Status column in POLLERS tab only. This patch aims
to add those status indicators to other columns to free user of
determining whether thread or core was busy themselves.

Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Change-Id: Ib1fb58c50f41591107ed0ea9c8fe2abcd5e54b5f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8414
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
2022-09-12 07:25:57 +00:00
yidong0635
1ad140676e test/aer: Complete requests before detaching.
The failed stack:
The calling processes:
	spdk_nvme_detach_async->nvme_ctrlr_detach_async->
	nvme_ctrlr_get_ref_count->nvme_ctrlr_proc_put_ref->
	nvme_ctrlr_remove_inactive_proc->nvme_ctrlr_get_current_process->
	nvme_ctrlr_remove_process

The proc->active_reqs list processing steps:
	spdk_nvme_ctrlr_process_admin_completions->
	spdk_nvme_qpair_process_completions->
	nvme_transport_qpair_process_completions->
	nvme_pcie_qpair_process_completions->
		*nvme_pcie_qpair_complete_tracker
		*nvme_pcie_qpair_insert_pending_admin_request
	and
		*nvme_pcie_qpair_complete_pending_admin_request
Issue assert:
	assert(STAILQ_EMPTY(&proc->active_reqs));
The assert means there are outstanding active requests in
proc->active_reqs list.

According the test app's coding style, it needs to foreach the dev and
complete the outstanding commands before spdk_nvme_detach_async
especially  it launched spdk_nvme_ctrlr_register_aer_callback .

Bring the aer tests back, to check it.

Fixes issue #2674.

Signed-off-by: yidong0635 <dongx.yi@intel.com>
Change-Id: Ic0d3946f8c8743155cf3067aea1c9b4676748d2f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14433
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2022-09-12 07:25:41 +00:00
Boris Glimcher
4f8bf4c380 sock/ssl: free SSL on socket close()
Fixes #2684

Change-Id: I0a91e7a70c53c8130921c70872b28d199d862346
Signed-off-by: Boris Glimcher <Boris.Glimcher@emc.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14432
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2022-09-12 07:23:10 +00:00
Kozlowski Mateusz
156b48b298 ftl: Trim tests
Unmap FIO and dd tests for FTL.

Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Change-Id: I6d01c4e729755c12d6c27159a7b12c575fc62e0a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13382
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-09-09 19:44:29 +00:00
Kozlowski Mateusz
630922e825 ftl: Add lazy unmap process
Since only L2P pages as a whole are marked as invalid during trim, the
specific L2P entries won't be updated until someone touches that page.
The unmap process will slowly invalidate pages during runtime, by paging
them in. This will allow compaction and relocation to benefit from the
trim as the user data gets invalidated.

Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Change-Id: I239b9adf0aaaeac58f440145f4ab78b0d78d98b0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13381
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-09-09 19:44:29 +00:00
Kozlowski Mateusz
b3e5d8a723 ftl: Add recovery and restart path for trim
Restores necessary metadata and sets L2P during clean/dirty shutdown recovery
process.

Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Change-Id: Iaa44025250b44f424ac9de5859d1db82900ecaa9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13380
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-09-09 19:44:29 +00:00
Kozlowski Mateusz
2c7c8b6ceb ftl: Add rpc functionality for unmap
Trim is now also available as a management operation via RPC.

Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Change-Id: I05b778a611e9809a14bfed50b01986bb4649a35c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13379
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-09-09 19:44:29 +00:00
Kozlowski Mateusz
66fe5f75bb ftl: Unmap functionality
Adds ability to send trim commands to FTL - only 4MiB aligned requests (both
for offset and length of request) will be processed. During a trim
operation an L2P page (containing 1024 4B entries, 1 per user LBA; which
is where the 4MiB alignment comes from) will be marked as unmapped.
After this point any L2P access to that page will actually set the
entries themselves as FTL_ADDR_INVALID. This is done to make the trim as
fast as possible, since for large requests it's probable that most of
the L2P pages aren't actually in DRAM.

Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Change-Id: I4a04ee9498a2a6939af31b06f2e45d2b7cccbf19
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13378
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-09-09 19:44:29 +00:00