Commit Graph

1778 Commits

Author SHA1 Message Date
Michal Berger
b43a0112dd scripts/perf: Add script for collecting system power statistics
This script is a wrapper around ipmitool which attempts to extract
power statistics from the BMC. It's mainly based around DCMI spec
to make sure it's common for most of the platforms that we use
in our CI. If given BMC does not conform to DCMI, there's a fallback
in place which attempts to find proper sensor in the SDR which may
report overall power usage of the underlying system.

By default, current ("NOW") reading is extracted in context of DCMI.
Different intervals may be defined, and when matched with supported
averaging time periods, special version of the command will be used
to extract additional avg, min, max information for given time (e.g.
current, max, min, avg readings from last 1m). For the SDR fallback,
only current reading can be extracted.

When terminated, script dumps all readings + their averaged value
into separate .txt files.

Change-Id: Ifc8c1defe862254864d69de7edf5cce635aff258
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14144
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-09-20 10:17:04 +00:00
Konrad Sztyber
cc3f842cd1 sma: initial crypto definitions
This patch defines the interface for crypto engines, which provide
support for configuring crypto on a given volume.  Only a single crypto
engine can be active at a time and it's selected in the "crypto" section
of the config file.  Similarly to device managers, external crypto
engines can be loaded from plugins.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Id942ef876e070816827d7ad1937eb510a85c8f8d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13869
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: <sebastian.brzezinka@intel.com>
2022-09-19 19:43:35 +00:00
Michal Berger
7a4cb7bc1c perf/vhost: Allow to pass multiple FIO configs to the benchmark script
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I96df5f21f04b9196083f91783f8dd9cc4c6920d1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13688
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-09-15 20:15:57 +00:00
Michal Berger
971451a84a perf/vhost: Allow to add extra fio config through $fio_extra_conf
$fio_extra_conf can either point at an existing file or be an
attached stdin the fio_conf() should read from.

Signed-off-by: Michal Berger <michallinuxstuff@gmail.com>
Change-Id: If902262b0100fb056491928b3f667ab15e6e0024
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12429
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-09-15 20:15:57 +00:00
Michal Berger
0ee37d256a perf/vhost: Add io_uring related options to fio configuration
https://trello.com/c/EFPM29gq/399-use-iouring-in-vhost-performance-tests

Signed-off-by: Michal Berger <michallinuxstuff@gmail.com>
Change-Id: I9bf8d9a121ffc1f52273fbe863bf9918e7399e59
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12428
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-09-15 20:15:57 +00:00
Michal Berger
008f8a8617 perf/vhost: Replace run_vhost_test.py with a simple bash wrapper
The wrapper glues execution of the perf/vhost tests in as simple form
as possible. The arguments are meant to be passed either via env or
the cmdline in a keyworded argument fashion. All defaults are
embedded inside the configuration which is built by the wrapper
itself.

https://trello.com/c/HSoRtQkO/401-deprecate-vhost-performance-python-helper-script

Signed-off-by: Michal Berger <michallinuxstuff@gmail.com>
Change-Id: Iad94bfdd65cb8871b819351ddae65e647621fb30
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12400
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-09-15 20:15:57 +00:00
Boris Glimcher
35f7f0ce1e nvme/tcp: Allow to choose SSL socket implementation
Adding `psk` field to `spdk_nvme_ctrlr_opts`

Adding `psk` parameter to `bdev_nvme_attach_controller` RPC

Change-Id: Ie6f0d8b04ce472e6153934e985c026acded6cdfc
Signed-off-by: Boris Glimcher <Boris.Glimcher@emc.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14046
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-09-14 07:44:53 +00:00
Kozlowski Mateusz
2c7c8b6ceb ftl: Add rpc functionality for unmap
Trim is now also available as a management operation via RPC.

Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Change-Id: I05b778a611e9809a14bfed50b01986bb4649a35c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13379
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-09-09 19:44:29 +00:00
gongwei
c8e594c2a0 bdev_iscsi: modify the timeout parameter name of iscsi opts
the current timeout parameter of the interface bdev_iscsi_set_opts is
duplicated with the timeout parameter of the JSONRPCClient parameter,
which may cause the iscsi timeout and JSONRPCClient parameters to
overwrite each other.

Signed-off-by: gongwei <gongwei833x@gmail.com>
Change-Id: I96604a7e1a495ac2e99518812297230680df42fd
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14306
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
2022-09-09 13:02:25 +00:00
Karol Latecki
7807960d46 scripts/nvmf_perf: load common drivers before test
Make sure needed drivers are loaded before we start
the test.

Change-Id: I93e16a2266f99e7bfa6fffaede460c6230659872
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13616
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Michal Berger <michal.berger@intel.com>
2022-09-09 12:59:13 +00:00
Karol Latecki
5d82cd2b08 scripts/nvmf_perf: update PCM measurements
Update PCM binary paths and executable names are these have
changed some time ago.
Update README.md with an explanation about int variables
for "pcm_settings" option.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I8270766ff2c429623a29e4c12427a3dd7e45f229
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13511
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Michal Berger <michal.berger@intel.com>
2022-09-09 12:59:13 +00:00
Karol Latecki
b0997a590a scripts/nvmf_perf: kill process after timeout
If SIGTERM does not work - use SIGKILL and clean up
any remaining socket files.

Change-Id: I1a4f2e49a8a43f3eed19e09e16d4ee3420cd31f7
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13739
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Michal Berger <michal.berger@intel.com>
2022-09-09 12:59:13 +00:00
Karol Latecki
1950f399e1 scripts/nvmf_perf: add num_jobs to configuration file
"num_jobs" is described as required parameter in README.md
but not included in example configuration file.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I1077d027d6c7dbee49afa1df4345b4dd478b5ad6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13615
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Michal Berger <michal.berger@intel.com>
2022-09-09 12:59:13 +00:00
Karol Latecki
750896ce36 scripts/nvmf_perf: use posix sockets
Use posix sockets by default when running tests.
For now we only want to test with this socket type
and ssl socket support may be added later if needed.

Change-Id: If8b125518cf1c01a841276b58fa7ca7799aeb229
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14353
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-09-06 07:16:45 +00:00
Kozlowski Mateusz
e7e5bc07b2 FTL: Add initial L2P cache logic
L2P cache allows for partial storing of L2P in memory, paging in and out
as necessary, lowering the total memory consumption.

Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Change-Id: I727fec9d2f0ade4ca73e872d62a2ec10cfdb0a88
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13353
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-09-02 17:40:09 +00:00
Konrad Sztyber
4cbd23e28b vmd: method for forcing a rescan
Added a new RPC, vmd_rescan, that forces the VMD driver to do a rescan
of all devices behind the VMD.  A device that was previously removed via
spdk_vmd_remove_device() will be found again during vmd_rescan.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ide87eb44c1d6d524234820dc07c78ba5b8bcd3ad
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13958
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tom Nabarro <tom.nabarro@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-09-01 08:48:32 +00:00
Konrad Sztyber
052ea0baac vmd: method for removing devices behind VMD
Added new RPC, vmd_remove_device, that allows users to remove a PCI
device managed by the VMD library simulating a hot-remove.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ifb84818ce8d147d1d586b52590527e85fe9c10de
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13957
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tom Nabarro <tom.nabarro@intel.com>
2022-09-01 08:48:32 +00:00
Konrad Sztyber
f0441b29db vmd: rename enable_vmd RPC to vmd_enable
The new name is consistent with the naming scheme of
<subsystem>_<action> that all of our other RPCs use.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I2cae7af5715add8eba26501cd192a6ac4884ec69
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13952
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Tom Nabarro <tom.nabarro@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-09-01 08:48:32 +00:00
Pawel Piatek
d21dc1501f scripts/vagrant: add support for FreeBSD 13
Signed-off-by: Pawel Piatek <pawelx.piatek@intel.com>
Change-Id: I719df0334dc786418e9eee05390c03f5f0b99434
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13622
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Michal Berger <michal.berger@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
2022-09-01 08:28:20 +00:00
Kozlowski Mateusz
0e33da4974 ftl: fast shutdown
Adds API for fast shutdown - the ability for FTL to skip most
of the metadata persists made during clean shutdown, and relying
on their representation in shared memory instead. This allows for
faster update of SPDK (or just FTL, assuming no metadata changes),
with downtime reduction from 2-5 seconds to 500-1000 ms (for
14TiB+800GiB base and cache drives).

Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Change-Id: I5999d31698a81512db8d5893eabee7b505c80d06
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13348
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-08-30 14:48:50 +00:00
Karol Latecki
7dfb090f95 pkgdep: silence hash in markdownlint installation
Do not print messages in case rake or bundler
are not found.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I92145a409af0ae95bc9ae194c7830feabd3a4641
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14155
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Kamil Godzwon <kamilx.godzwon@intel.com>
Reviewed-by: Michal Berger <michal.berger@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-08-29 11:54:08 +00:00
Michal Berger
c8f62d792a test: Enable xnvme in autotest builds
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I32594d94eb397dfe1fcec867fd0caf67a07b208c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14118
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-08-29 11:40:08 +00:00
gongwei
1519aa4715 bdev_iscsi: support iscsi timeout setting
Change-Id: I189ae677dfb13feb834b73fd1f55bf2545679213
Signed-off-by: gongwei <gongwei833x@gmail.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14110
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Community-CI: Mellanox Build Bot
2022-08-25 07:43:24 +00:00
0xe0f
2e283fcb67 bdev/daos: introduction of daos bdev
This commmit introduces a new bdev type backed up by DAOS DFS.

Design wise this bdev is a file named as the bdev itself in the DAOS POSIX
container that uses daos event queue per io channel.
Having an event queue per io channel is showing the best IO throughput.
The implementation uses the independent pool and container connections per
device's channel for the best IO throughput.

The semantic of usage is the same as any other bdev type.

To build SPDK with daos support, daos-devel package has to be installed.
The current supported DAOS version is v2.X, please see the installatoin and
setup guide here: https://docs.daos.io/v2.0/

$ ./configure --with-daos

To run it, the target machine should have daos_agent up and running, as
well as the pool and POSIX container ready to use, please see the
detailed requirements here: https://docs.daos.io/v2.0/admin/hardware/.

To export bdev over tcp:

$ ./nvmf_tgt &
$ ./scripts/rpc.py nvmf_create_transport -t TCP -u 2097152 -i 2097152

$ ./scripts/rpc.py bdev_daos_create daosdev0 <pool-label> <cont-label>
1048576 4096

$ ./scripts/rpc.py nvmf_create_subsystem nqn.2016-06.io.spdk1:cnode1 -a -s
SPDK00000000000001 -d SPDK_Virtual_Controller_1
$ ./scripts/rpc.py nvmf_subsystem_add_ns nqn.2016-06.io.spdk1:cnode1
daosdev0
$ ./scripts/rpc.py nvmf_subsystem_add_listener nqn.2016-06.io.spdk1:cnode1
-t tcp -a <IP> -s 4420

On the initiator side, make sure that `nvme-tcp` module is loaded then
connect drives, for instance:

$ nvme connect-all -t tcp -a 172.31.91.61 -s 4420
$ nvme list

Signed-off-by: Denis Barakhtanov <denis.barahtanov@croit.io>
Change-Id: I51945465122e0fb96de4326db742169419966806
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12260
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-08-23 07:15:13 +00:00
Jim Harris
e36f0d363e nvme/pcie, nvme/tcp: add cb_arg context tracepoint argument
This allows mapping an nvme_request back to the
nvme_bdev_io.

This requires bumping up the max number of arguments per
tracepoint.  5 was previously chosen as max since it
exactly fit in 64 bytes (1 cacheline) when all
arguments were stored as uint64_t, but now that we
support uint32_t arguments we can afford extra
arguments when some of them are uint32_t.  I've
bumped it to 8 so we can avoid having to touch
this value multiple times if we find some cases
where we need 7 or 8 args.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ie2ef5e59d10549860b47542e68c1c34efa63047f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13995
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2022-08-19 11:06:31 +00:00
Michal Berger
8309d224b9 scripts/vagrant: Add missing help entry for the zns option
This option determines if underlying nvme device must be zoned.

Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I65cb91db81d4c5ce694010db2e79e8af389687ab
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14008
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-08-16 21:48:18 +00:00
Michal Berger
c4acef02aa scripts/vagrant: Allow to configure active metadata size for nvme
The argument is treated as bool and the actual value of the metadata
size is set to 64B.

https://trello.com/c/neiHoXlr/448-add-extra-nvme-drive-to-vms-to-accommodate-for-new-ftl-tests

Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: Icd96903a5fa186373de1c647e3885f0162ffd579
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14007
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-08-16 21:48:18 +00:00
Jim Harris
43a3984c6c configure: add CONFIG_HAVE_ARC4RANDOM
glibc 2.36 added arc4random(), which breaks
the SPDK iSCSI build since it always implements its
own arc4random() implementation on non-FreeBSD OS
(meaning always on Linux).

So instead add a CONFIG_HAVE_ARC4RANDOM and remove
the explicit FreeBSD dependency - this will work on
FreeBSD as well as Linux with >= glibc 2.36.

Also fix check_format.sh, so that it does not
enforce spdk/stdinc.h checks on code snippets in
the configure file.

Fixes issue #2637.

Reported-by: Karl Bonde Torp <k.torp@samsung.com>
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iab9da8ae30d62a56869530846372ffddf7138eed
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14028
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
2022-08-16 10:25:17 +00:00
Boris Glimcher
6212597bda sock/ssl: Add psk_key and psk_identity options to spdk_sock_impl_opts
Note, this change only sets defaults for the ID/KEY,
more specific use cases like NVMe/TCP may set the ID and KEY on a per connection basis.

Also simplify PSK identity string, that isn't NVMe focused.
NVMe libraries using this will need to construct more complicated
identity strings and pass them to the sock layer.

Example:
  rpc.py sock_impl_set_options -i ssl --psk-key 4321DEADBEEF1234
  rpc.py sock_impl_set_options -i ssl --psk-identity psk.spdk.io

  ./build/examples/perf --psk-key 4321DEADBEEF1234 --psk-identity psk.spdk.io

  ./build/examples/hello_sock --psk-key 4321DEADBEEF1234 --psk-identity psk.spdk.io

Change-Id: I1cb5b0b706bdeafbccbc71f8320bc8e2961cbb55
Signed-off-by: Boris Glimcher <Boris.Glimcher@emc.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13759
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2022-08-15 16:52:28 +00:00
Michal Berger
7f1b530ac0 scripts/vagrant: Remove some outdated comments
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I5c8f0b47afd38aab2ac0b45ea7f4f8e9f4c8fc54
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13930
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
2022-08-12 09:00:15 +00:00
Michal Berger
8aee39f906 scripts/vagrant: Add support for loading extra Vagrantfiles
This allows our setup to be more flexible - extra Vagrantfiles can
provide additional configuration which may override|extend existing
configuration without having a support for specific set of options
on our end.

CI will attempt to use this functionality to mitigate the following
issue:

  https://github.com/spdk/spdk/issues/2628

Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I80cbe67b2fe3618f93e9233e0586417b208f987b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13929
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-08-12 09:00:15 +00:00
paul luse
efa33b8590 lib/accel: add RPC to enable override of opcode to engine
Docs explaining how to use the RPC are in the next patch in the
series.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I7dab8fdbeb90cdfde8b3e916ed6d19930ad36e66
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12848
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-08-10 07:25:29 +00:00
Jim Harris
813b92af44 setup.sh: add some more app names to the shm cleanup list
accel_perf and bdev_svc were not on the list.

Fixes issue #2483.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I9f758c1871f42c10e2c92b2fb6cd676830a1b3a0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13877
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michal Berger <michal.berger@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-08-05 10:48:42 +00:00
paul luse
44cbea402e lib/accel: Add new RPC to get valid engine info.
The RPC provides a list of initialized engine names along with
that engine's supported operations.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I59f9e5cb7aa51a6193f0bd2ec31e543a56c12f17
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13745
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2022-08-03 07:53:21 +00:00
paul luse
c6ecddcc1c lib/accel: add RPC to get list of OP codes per module
In prep for upcoming patch that will provide an RPC to override
and automatic assignment of an op code to an engine.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I17d4b962fb376a77f97ce051a513679d0fba698e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12829
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-08-03 07:53:21 +00:00
Artur Paszkiewicz
be90ea6e7b FTL: Add bdev_ftl_create and delete rpc definitions
Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Change-Id: I0837028fbe349e8df7f05fb3c9db1f4682f04679
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13301
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-08-02 19:00:42 +00:00
Milosz Linkiewicz
d5d6efd8a8 sma: nvmf/vfiouser device manager implementation
This patch utilize generic sma implementation by adding vfiouser
devices manager. It's allow to expose virtualized block devices to
QEMU instances or other arbitrary processes.

Max device capacity depend on available `pci-bridge`
```yaml
devices:
  - name: 'vfiouser'
    params:
      buses:
        - name: 'pci.spdk.0'
          count: 32
        - name: 'pci.spdk.1'
          count: 32
      qmp_addr: 127.0.0.1
      qmp_port: 9090
```

Signed-off-by: Milosz Linkiewicz <milosz.linkiewicz@intel.com>
Signed-off-by: Sebastian Brzezinka <sebastian.brzezinka@intel.com>
Change-Id: I5ab43f4b877c371fa16a4daf4212ac2686991bd4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13004
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-08-01 08:45:58 +00:00
Michal Berger
eb66239458 scripts/gen_nvme: Add option to define number of local bdevs to use
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I2474328a1aa0d72526e272430a8396e8e3fadfe1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13761
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-07-28 06:56:05 +00:00
Kamil Godzwon
bcf9753b15 scripts/pkgdep: DAOS dependency repository has changed
After recent DAOS 2.0.3 release (2022-07-14) repository has
changed its location.

Signed-off-by: Kamil Godzwon <kamilx.godzwon@intel.com>
Change-Id: I4ad97844fb529c8dde5e39b751c3d92c69b8c26d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13683
Reviewed-by: Michal Berger <michal.berger@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
2022-07-22 07:28:20 +00:00
Boris Glimcher
806744b7c8 sock: Add ktls and tls_version to spdk_sock_impl_opts
Since `sock_impl_opts` was added to `sock_opts`
Can remove `ktls` and `tls_version` from spdk_sock_opts

Example:
  rpc.py sock_impl_set_options -i ssl --enable-ktls
  rpc.py sock_impl_set_options -i ssl --disable-ktls
  rpc.py sock_impl_set_options -i ssl --tls-version=12

  ./build/examples/perf --enable-ktls
  ./build/examples/perf --disable-ktls
  ./build/examples/perf --tls-version=12

Check kTLS statistics here: /proc/net/tls_stat

Change-Id: Icf7ee822bad92fda149710be77feb77fc8d4f163
Signed-off-by: Boris Glimcher <Boris.Glimcher@emc.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13510
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-07-22 06:41:39 +00:00
Michal Berger
269737b902 scripts/fio-wrapper: Simplify check for bdev devices
While testing fedora35 under 5.18 kernel it was noticed that the
sysfs dir structure changed comparing to older versions of the
kernel. To make sure proper devices can be found regardless of
the kernel version simplify the process by just looking up
the model of the ctrl given nvme block device is connected to.

Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I7ff55c8d8c50acd2a117bed2a31ddb9890615e18
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13674
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Kamil Godzwon <kamilx.godzwon@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
2022-07-18 13:00:17 +00:00
Kamil Godzwon
10abf16de2 vagrant: add Fedora36 support
Signed-off-by: Kamil Godzwon <kamilx.godzwon@intel.com>
Change-Id: Iced5b9d20671f27c5fad975cded0e44beff74f4f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12996
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-07-18 11:51:23 +00:00
Krishna Kanth Reddy
6f338d4bf3 bdev : xNVMe BDEV module implementation
This implementation of xNVMe BDEV module supports the char-device / ioctl-over-uring,
along with the "regular" io_uring, libaio, POSIX aio, emulated aio (via threadpools) etc.

Code changes done :
a. Addition of xNVMe submodule to SPDK
b. Modification of RPC scripts to Create / Delete xNVMe BDEVs
c. Implementation of xNVMe BDEV module

Signed-off-by: Krishna Kanth Reddy <krish.reddy@samsung.com>
Change-Id: If814ca1c784124df429d283015a6570068b44f87
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11161
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-07-15 12:52:13 +00:00
Konrad Sztyber
9e03923b95 rpc: fix zerocopy_threshold in sock_impl_set_options
This shouldn't be an on/off option, but should expect an integer value
specyfing the number of bytes.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I888af3253ad619a6c5f86e6d35454dafe8fc09a0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13540
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2022-07-05 07:11:28 +00:00
Jaroslaw Chachulski
400aadf83f scripts/vagrant: Add ubuntu 22.04 lts support.
Add support for ubuntu 22.04 lts.

Signed-off-by: Jaroslaw Chachulski <jaroslawx.chachulski@intel.com>
Change-Id: I6f598bbf1961c8266620acb69a388e94ed794d8f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12988
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-07-05 07:10:56 +00:00
Michal Berger
e8863cb76b configure: Allow to define custom libdir location
This is done mainly to align the final location with the one listed
in the .pc files created for each lib.

Signed-off-by: Michal Berger <michallinuxstuff@gmail.com>
Change-Id: I99d6875c9e26f8202723dd6d73bd865b6478bcf9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12978
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-07-04 07:25:39 +00:00
Jaroslaw Chachulski
642f8b3da5 scripts/rpc: remove default values from bdev functions
Removing default options from parser command to prevent overwrite by
argparse the defaults defined in the .c files.

Signed-off-by: Jaroslaw Chachulski <jaroslawx.chachulski@intel.com>
Change-Id: I39026e0866f2a9aecda6588bb3f91ed87676ddd1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13461
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-07-04 07:23:26 +00:00
Jaroslaw Chachulski
cd79f1a210 scripts/rpc: remove default value from bdev_ocf_create
Removing default options from parser command to prevent overwrite by
argparse the defaults defined in the .c files.

Signed-off-by: Jaroslaw Chachulski <jaroslawx.chachulski@intel.com>
Change-Id: I9c1ba26e2b867b91fd5f7cd91edbff1844f08c79
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12979
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2022-07-04 07:23:26 +00:00
Jaroslaw Chachulski
ba81cb169e scripts/rpc: remove default values from bdev_split_create and
nvmf_get_subsystems functions

Removing default options from parser command to prevent overwrite by
argparse the defaults defined in the .c files.

Signed-off-by: Jaroslaw Chachulski <jaroslawx.chachulski@intel.com>
Change-Id: Ibce4122aed1f6c4e22b6286f52f4295a4cc13e0c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13497
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-07-04 07:23:26 +00:00
Jaroslaw Chachulski
169bab1f1a scripts/rpc: remove default value from iscsi functions
Removing default options from parser command to prevent overwrite by
argparse the defaults defined in the .c files.

Signed-off-by: Jaroslaw Chachulski <jaroslawx.chachulski@intel.com>
Change-Id: I11ef4947324dfba796fc16da566c0b8a1e606a00
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12922
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-07-04 07:23:26 +00:00