Commit Graph

1764 Commits

Author SHA1 Message Date
Kozlowski Mateusz
e7e5bc07b2 FTL: Add initial L2P cache logic
L2P cache allows for partial storing of L2P in memory, paging in and out
as necessary, lowering the total memory consumption.

Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Change-Id: I727fec9d2f0ade4ca73e872d62a2ec10cfdb0a88
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13353
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-09-02 17:40:09 +00:00
Konrad Sztyber
4cbd23e28b vmd: method for forcing a rescan
Added a new RPC, vmd_rescan, that forces the VMD driver to do a rescan
of all devices behind the VMD.  A device that was previously removed via
spdk_vmd_remove_device() will be found again during vmd_rescan.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ide87eb44c1d6d524234820dc07c78ba5b8bcd3ad
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13958
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tom Nabarro <tom.nabarro@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-09-01 08:48:32 +00:00
Konrad Sztyber
052ea0baac vmd: method for removing devices behind VMD
Added new RPC, vmd_remove_device, that allows users to remove a PCI
device managed by the VMD library simulating a hot-remove.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ifb84818ce8d147d1d586b52590527e85fe9c10de
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13957
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tom Nabarro <tom.nabarro@intel.com>
2022-09-01 08:48:32 +00:00
Konrad Sztyber
f0441b29db vmd: rename enable_vmd RPC to vmd_enable
The new name is consistent with the naming scheme of
<subsystem>_<action> that all of our other RPCs use.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I2cae7af5715add8eba26501cd192a6ac4884ec69
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13952
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Tom Nabarro <tom.nabarro@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-09-01 08:48:32 +00:00
Pawel Piatek
d21dc1501f scripts/vagrant: add support for FreeBSD 13
Signed-off-by: Pawel Piatek <pawelx.piatek@intel.com>
Change-Id: I719df0334dc786418e9eee05390c03f5f0b99434
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13622
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Michal Berger <michal.berger@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
2022-09-01 08:28:20 +00:00
Kozlowski Mateusz
0e33da4974 ftl: fast shutdown
Adds API for fast shutdown - the ability for FTL to skip most
of the metadata persists made during clean shutdown, and relying
on their representation in shared memory instead. This allows for
faster update of SPDK (or just FTL, assuming no metadata changes),
with downtime reduction from 2-5 seconds to 500-1000 ms (for
14TiB+800GiB base and cache drives).

Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Change-Id: I5999d31698a81512db8d5893eabee7b505c80d06
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13348
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-08-30 14:48:50 +00:00
Karol Latecki
7dfb090f95 pkgdep: silence hash in markdownlint installation
Do not print messages in case rake or bundler
are not found.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I92145a409af0ae95bc9ae194c7830feabd3a4641
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14155
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Kamil Godzwon <kamilx.godzwon@intel.com>
Reviewed-by: Michal Berger <michal.berger@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-08-29 11:54:08 +00:00
Michal Berger
c8f62d792a test: Enable xnvme in autotest builds
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I32594d94eb397dfe1fcec867fd0caf67a07b208c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14118
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-08-29 11:40:08 +00:00
gongwei
1519aa4715 bdev_iscsi: support iscsi timeout setting
Change-Id: I189ae677dfb13feb834b73fd1f55bf2545679213
Signed-off-by: gongwei <gongwei833x@gmail.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14110
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Community-CI: Mellanox Build Bot
2022-08-25 07:43:24 +00:00
0xe0f
2e283fcb67 bdev/daos: introduction of daos bdev
This commmit introduces a new bdev type backed up by DAOS DFS.

Design wise this bdev is a file named as the bdev itself in the DAOS POSIX
container that uses daos event queue per io channel.
Having an event queue per io channel is showing the best IO throughput.
The implementation uses the independent pool and container connections per
device's channel for the best IO throughput.

The semantic of usage is the same as any other bdev type.

To build SPDK with daos support, daos-devel package has to be installed.
The current supported DAOS version is v2.X, please see the installatoin and
setup guide here: https://docs.daos.io/v2.0/

$ ./configure --with-daos

To run it, the target machine should have daos_agent up and running, as
well as the pool and POSIX container ready to use, please see the
detailed requirements here: https://docs.daos.io/v2.0/admin/hardware/.

To export bdev over tcp:

$ ./nvmf_tgt &
$ ./scripts/rpc.py nvmf_create_transport -t TCP -u 2097152 -i 2097152

$ ./scripts/rpc.py bdev_daos_create daosdev0 <pool-label> <cont-label>
1048576 4096

$ ./scripts/rpc.py nvmf_create_subsystem nqn.2016-06.io.spdk1:cnode1 -a -s
SPDK00000000000001 -d SPDK_Virtual_Controller_1
$ ./scripts/rpc.py nvmf_subsystem_add_ns nqn.2016-06.io.spdk1:cnode1
daosdev0
$ ./scripts/rpc.py nvmf_subsystem_add_listener nqn.2016-06.io.spdk1:cnode1
-t tcp -a <IP> -s 4420

On the initiator side, make sure that `nvme-tcp` module is loaded then
connect drives, for instance:

$ nvme connect-all -t tcp -a 172.31.91.61 -s 4420
$ nvme list

Signed-off-by: Denis Barakhtanov <denis.barahtanov@croit.io>
Change-Id: I51945465122e0fb96de4326db742169419966806
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12260
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-08-23 07:15:13 +00:00
Jim Harris
e36f0d363e nvme/pcie, nvme/tcp: add cb_arg context tracepoint argument
This allows mapping an nvme_request back to the
nvme_bdev_io.

This requires bumping up the max number of arguments per
tracepoint.  5 was previously chosen as max since it
exactly fit in 64 bytes (1 cacheline) when all
arguments were stored as uint64_t, but now that we
support uint32_t arguments we can afford extra
arguments when some of them are uint32_t.  I've
bumped it to 8 so we can avoid having to touch
this value multiple times if we find some cases
where we need 7 or 8 args.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ie2ef5e59d10549860b47542e68c1c34efa63047f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13995
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2022-08-19 11:06:31 +00:00
Michal Berger
8309d224b9 scripts/vagrant: Add missing help entry for the zns option
This option determines if underlying nvme device must be zoned.

Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I65cb91db81d4c5ce694010db2e79e8af389687ab
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14008
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-08-16 21:48:18 +00:00
Michal Berger
c4acef02aa scripts/vagrant: Allow to configure active metadata size for nvme
The argument is treated as bool and the actual value of the metadata
size is set to 64B.

https://trello.com/c/neiHoXlr/448-add-extra-nvme-drive-to-vms-to-accommodate-for-new-ftl-tests

Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: Icd96903a5fa186373de1c647e3885f0162ffd579
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14007
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-08-16 21:48:18 +00:00
Jim Harris
43a3984c6c configure: add CONFIG_HAVE_ARC4RANDOM
glibc 2.36 added arc4random(), which breaks
the SPDK iSCSI build since it always implements its
own arc4random() implementation on non-FreeBSD OS
(meaning always on Linux).

So instead add a CONFIG_HAVE_ARC4RANDOM and remove
the explicit FreeBSD dependency - this will work on
FreeBSD as well as Linux with >= glibc 2.36.

Also fix check_format.sh, so that it does not
enforce spdk/stdinc.h checks on code snippets in
the configure file.

Fixes issue #2637.

Reported-by: Karl Bonde Torp <k.torp@samsung.com>
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iab9da8ae30d62a56869530846372ffddf7138eed
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14028
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
2022-08-16 10:25:17 +00:00
Boris Glimcher
6212597bda sock/ssl: Add psk_key and psk_identity options to spdk_sock_impl_opts
Note, this change only sets defaults for the ID/KEY,
more specific use cases like NVMe/TCP may set the ID and KEY on a per connection basis.

Also simplify PSK identity string, that isn't NVMe focused.
NVMe libraries using this will need to construct more complicated
identity strings and pass them to the sock layer.

Example:
  rpc.py sock_impl_set_options -i ssl --psk-key 4321DEADBEEF1234
  rpc.py sock_impl_set_options -i ssl --psk-identity psk.spdk.io

  ./build/examples/perf --psk-key 4321DEADBEEF1234 --psk-identity psk.spdk.io

  ./build/examples/hello_sock --psk-key 4321DEADBEEF1234 --psk-identity psk.spdk.io

Change-Id: I1cb5b0b706bdeafbccbc71f8320bc8e2961cbb55
Signed-off-by: Boris Glimcher <Boris.Glimcher@emc.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13759
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2022-08-15 16:52:28 +00:00
Michal Berger
7f1b530ac0 scripts/vagrant: Remove some outdated comments
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I5c8f0b47afd38aab2ac0b45ea7f4f8e9f4c8fc54
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13930
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
2022-08-12 09:00:15 +00:00
Michal Berger
8aee39f906 scripts/vagrant: Add support for loading extra Vagrantfiles
This allows our setup to be more flexible - extra Vagrantfiles can
provide additional configuration which may override|extend existing
configuration without having a support for specific set of options
on our end.

CI will attempt to use this functionality to mitigate the following
issue:

  https://github.com/spdk/spdk/issues/2628

Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I80cbe67b2fe3618f93e9233e0586417b208f987b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13929
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-08-12 09:00:15 +00:00
paul luse
efa33b8590 lib/accel: add RPC to enable override of opcode to engine
Docs explaining how to use the RPC are in the next patch in the
series.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I7dab8fdbeb90cdfde8b3e916ed6d19930ad36e66
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12848
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-08-10 07:25:29 +00:00
Jim Harris
813b92af44 setup.sh: add some more app names to the shm cleanup list
accel_perf and bdev_svc were not on the list.

Fixes issue #2483.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I9f758c1871f42c10e2c92b2fb6cd676830a1b3a0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13877
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michal Berger <michal.berger@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-08-05 10:48:42 +00:00
paul luse
44cbea402e lib/accel: Add new RPC to get valid engine info.
The RPC provides a list of initialized engine names along with
that engine's supported operations.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I59f9e5cb7aa51a6193f0bd2ec31e543a56c12f17
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13745
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2022-08-03 07:53:21 +00:00
paul luse
c6ecddcc1c lib/accel: add RPC to get list of OP codes per module
In prep for upcoming patch that will provide an RPC to override
and automatic assignment of an op code to an engine.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I17d4b962fb376a77f97ce051a513679d0fba698e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12829
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-08-03 07:53:21 +00:00
Artur Paszkiewicz
be90ea6e7b FTL: Add bdev_ftl_create and delete rpc definitions
Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Change-Id: I0837028fbe349e8df7f05fb3c9db1f4682f04679
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13301
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-08-02 19:00:42 +00:00
Milosz Linkiewicz
d5d6efd8a8 sma: nvmf/vfiouser device manager implementation
This patch utilize generic sma implementation by adding vfiouser
devices manager. It's allow to expose virtualized block devices to
QEMU instances or other arbitrary processes.

Max device capacity depend on available `pci-bridge`
```yaml
devices:
  - name: 'vfiouser'
    params:
      buses:
        - name: 'pci.spdk.0'
          count: 32
        - name: 'pci.spdk.1'
          count: 32
      qmp_addr: 127.0.0.1
      qmp_port: 9090
```

Signed-off-by: Milosz Linkiewicz <milosz.linkiewicz@intel.com>
Signed-off-by: Sebastian Brzezinka <sebastian.brzezinka@intel.com>
Change-Id: I5ab43f4b877c371fa16a4daf4212ac2686991bd4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13004
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-08-01 08:45:58 +00:00
Michal Berger
eb66239458 scripts/gen_nvme: Add option to define number of local bdevs to use
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I2474328a1aa0d72526e272430a8396e8e3fadfe1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13761
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-07-28 06:56:05 +00:00
Kamil Godzwon
bcf9753b15 scripts/pkgdep: DAOS dependency repository has changed
After recent DAOS 2.0.3 release (2022-07-14) repository has
changed its location.

Signed-off-by: Kamil Godzwon <kamilx.godzwon@intel.com>
Change-Id: I4ad97844fb529c8dde5e39b751c3d92c69b8c26d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13683
Reviewed-by: Michal Berger <michal.berger@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
2022-07-22 07:28:20 +00:00
Boris Glimcher
806744b7c8 sock: Add ktls and tls_version to spdk_sock_impl_opts
Since `sock_impl_opts` was added to `sock_opts`
Can remove `ktls` and `tls_version` from spdk_sock_opts

Example:
  rpc.py sock_impl_set_options -i ssl --enable-ktls
  rpc.py sock_impl_set_options -i ssl --disable-ktls
  rpc.py sock_impl_set_options -i ssl --tls-version=12

  ./build/examples/perf --enable-ktls
  ./build/examples/perf --disable-ktls
  ./build/examples/perf --tls-version=12

Check kTLS statistics here: /proc/net/tls_stat

Change-Id: Icf7ee822bad92fda149710be77feb77fc8d4f163
Signed-off-by: Boris Glimcher <Boris.Glimcher@emc.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13510
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-07-22 06:41:39 +00:00
Michal Berger
269737b902 scripts/fio-wrapper: Simplify check for bdev devices
While testing fedora35 under 5.18 kernel it was noticed that the
sysfs dir structure changed comparing to older versions of the
kernel. To make sure proper devices can be found regardless of
the kernel version simplify the process by just looking up
the model of the ctrl given nvme block device is connected to.

Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I7ff55c8d8c50acd2a117bed2a31ddb9890615e18
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13674
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Kamil Godzwon <kamilx.godzwon@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
2022-07-18 13:00:17 +00:00
Kamil Godzwon
10abf16de2 vagrant: add Fedora36 support
Signed-off-by: Kamil Godzwon <kamilx.godzwon@intel.com>
Change-Id: Iced5b9d20671f27c5fad975cded0e44beff74f4f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12996
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-07-18 11:51:23 +00:00
Krishna Kanth Reddy
6f338d4bf3 bdev : xNVMe BDEV module implementation
This implementation of xNVMe BDEV module supports the char-device / ioctl-over-uring,
along with the "regular" io_uring, libaio, POSIX aio, emulated aio (via threadpools) etc.

Code changes done :
a. Addition of xNVMe submodule to SPDK
b. Modification of RPC scripts to Create / Delete xNVMe BDEVs
c. Implementation of xNVMe BDEV module

Signed-off-by: Krishna Kanth Reddy <krish.reddy@samsung.com>
Change-Id: If814ca1c784124df429d283015a6570068b44f87
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11161
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-07-15 12:52:13 +00:00
Konrad Sztyber
9e03923b95 rpc: fix zerocopy_threshold in sock_impl_set_options
This shouldn't be an on/off option, but should expect an integer value
specyfing the number of bytes.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I888af3253ad619a6c5f86e6d35454dafe8fc09a0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13540
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2022-07-05 07:11:28 +00:00
Jaroslaw Chachulski
400aadf83f scripts/vagrant: Add ubuntu 22.04 lts support.
Add support for ubuntu 22.04 lts.

Signed-off-by: Jaroslaw Chachulski <jaroslawx.chachulski@intel.com>
Change-Id: I6f598bbf1961c8266620acb69a388e94ed794d8f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12988
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-07-05 07:10:56 +00:00
Michal Berger
e8863cb76b configure: Allow to define custom libdir location
This is done mainly to align the final location with the one listed
in the .pc files created for each lib.

Signed-off-by: Michal Berger <michallinuxstuff@gmail.com>
Change-Id: I99d6875c9e26f8202723dd6d73bd865b6478bcf9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12978
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-07-04 07:25:39 +00:00
Jaroslaw Chachulski
642f8b3da5 scripts/rpc: remove default values from bdev functions
Removing default options from parser command to prevent overwrite by
argparse the defaults defined in the .c files.

Signed-off-by: Jaroslaw Chachulski <jaroslawx.chachulski@intel.com>
Change-Id: I39026e0866f2a9aecda6588bb3f91ed87676ddd1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13461
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-07-04 07:23:26 +00:00
Jaroslaw Chachulski
cd79f1a210 scripts/rpc: remove default value from bdev_ocf_create
Removing default options from parser command to prevent overwrite by
argparse the defaults defined in the .c files.

Signed-off-by: Jaroslaw Chachulski <jaroslawx.chachulski@intel.com>
Change-Id: I9c1ba26e2b867b91fd5f7cd91edbff1844f08c79
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12979
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2022-07-04 07:23:26 +00:00
Jaroslaw Chachulski
ba81cb169e scripts/rpc: remove default values from bdev_split_create and
nvmf_get_subsystems functions

Removing default options from parser command to prevent overwrite by
argparse the defaults defined in the .c files.

Signed-off-by: Jaroslaw Chachulski <jaroslawx.chachulski@intel.com>
Change-Id: Ibce4122aed1f6c4e22b6286f52f4295a4cc13e0c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13497
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-07-04 07:23:26 +00:00
Jaroslaw Chachulski
169bab1f1a scripts/rpc: remove default value from iscsi functions
Removing default options from parser command to prevent overwrite by
argparse the defaults defined in the .c files.

Signed-off-by: Jaroslaw Chachulski <jaroslawx.chachulski@intel.com>
Change-Id: I11ef4947324dfba796fc16da566c0b8a1e606a00
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12922
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-07-04 07:23:26 +00:00
Jaroslaw Chachulski
9518bbc260 scripts/rpc: remove default value from nvmf_create_subsystem function
Removing default options from parser command to prevent overwrite by
argparse the defaults defined in the .c files.

Signed-off-by: Jaroslaw Chachulski <jaroslawx.chachulski@intel.com>
Change-Id: I3e009139e10bbd2c9aa94e88ffdeb3a6570c4110
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12911
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2022-07-04 07:23:26 +00:00
Jaroslaw Chachulski
16772e6de3 scripts/rpc: remove default value from bdev_nvme_send_cmd function
Removing default options from parser command to prevent overwrite by
argparse the defaults defined in the .c files.

Signed-off-by: Jaroslaw Chachulski <jaroslawx.chachulski@intel.com>
Change-Id: I8db8ab436e15a313f8141ada14c5b60f2545266d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12894
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2022-07-04 07:23:26 +00:00
yupeng
4c59c6ac53 blobstore: add RPC for growing blobstore
Add num_md_pages_per_cluster_ratio parameter to bdev_lvol_create_lvstore.
Add bdev_lvol_grow_lvstore rpc.

Signed-off-by: Peng Yu <yupeng0921@gmail.com>
Change-Id: I43a129efbc1a08f9d0df027e80f63ab1393163a2
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9734
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-06-28 17:55:43 +00:00
Ben Walker
96d4bb6ac2 check_format: Re-enable naming convention check
This was temporarily disabled to work around a big style fix patch. It
can be enabled again now.

Change-Id: I9009d83798551545307cf5881bc1bbb87453e611
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13466
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-06-27 10:29:15 +00:00
Ben Walker
8dd1cd2104 check_format: For C files only, fix return type breaks
In SPDK, declarations have the return type on the same line. Definitions
have the return type on a separate line. Astyle has an option for
enforcing this. Unfortunately, it seems to have two bugs:

1) It doesn't work correctly at all on C++ files.
2) It often fails on functions that return enums, or long type names

Deal with 1) by adjusting the check_format.sh script to only tell astyle
to fix return type line breaks for C files and not C++. Deal with 2) by
adding a few typedefs to work around the problem.

Change-Id: Idf28281466cab8411ce252d5f02ab384166790c6
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13437
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
2022-06-27 09:33:48 +00:00
Ben Walker
8620c9a0d1 check_format: Temporarily disable naming convention check
This check isn't smart enough to handle the next big formatting patch,
so disable it and then re-enable later.

Change-Id: I2d73598d6b7c43f9356562161ed116bc01361166
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13465
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
2022-06-27 09:33:48 +00:00
Ben Walker
08477ef8d9 check_format: Split astyle call for C and C++
These are going to need different options in an upcoming patch.

Change-Id: I5d43f21887abefbae207b5cff23fafcfac4a33d7
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13464
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
2022-06-27 09:33:48 +00:00
Jaroslaw Chachulski
5960c74d04 scripts/rpc: remove default values from bdev_nvme_add_error_injection
Removing default options from parser command to prevent overwrite by
argparse the defaults defined in the .c files.

Signed-off-by: Jaroslaw Chachulski <jaroslawx.chachulski@intel.com>
Change-Id: I757b893a65a1c0e21310171fa8b89a87e28b87a4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12871
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-06-23 07:45:19 +00:00
Kamil Godzwon
14c0f94c88 test/fio: supress fio output files
verify.state files are not very useful because
we fail the tests at any first failure.
If there was something wrong with IO verification
then autotest.sh would fail.

Fio files with metrics output (IOPS, BW, latency, etc)
are also not very useful as per-patch tests are not
focused on benchmarking.

Fixes #2546

Signed-off-by: Kamil Godzwon <kamilx.godzwon@intel.com>
Change-Id: I33fbe2fb0749d840f08ffe0d606d206268140991
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13019
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-06-23 07:43:36 +00:00
Kamil Godzwon
eb341f672c pkgdep: do not install grpcio modules on Centos7
grpcio, grpcio-tools modules have compilation
problem on Centos7 and are not necessary under
this OS.

Signed-off-by: Kamil Godzwon <kamilx.godzwon@intel.com>
Change-Id: I2b48a3b1b2deafa983adf83424940ffa6ce7d20e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13070
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-06-21 12:59:16 +00:00
Denis Barakhtanov
386af793fd scripts/pkgdep.sh: Suse install DAOS bdev build dependencies
Signed-off-by: Denis Barakhtanov <denis.barahtanov@croit.io>
Change-Id: I851240a8c24c137aaf9c088ad294e96e349d5e97
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12688
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: <qun.wan@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Kamil Godzwon <kamilx.godzwon@intel.com>
2022-06-21 07:53:12 +00:00
Kamil Godzwon
e8945f926c scripts/pkgdep: adjusting packages for sles
Disabled installing markdownlint due to ruby
version issue.
Changed packages name according to packages
available from a repository.

Signed-off-by: Kamil Godzwon <kamilx.godzwon@intel.com>
Change-Id: Iea22d78dedad5386ab30d6bdd79cd7e961eaee72
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12756
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: <qun.wan@intel.com>
2022-06-21 07:53:12 +00:00
Karol Latecki
777afdf853 scripts/pkgdep.sh: install DAOS bdev build dependencies
Change-Id: I9b4032656839b5ebc5202b26b97b7882664263ab
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12393
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-06-21 07:53:12 +00:00
Jim Harris
9fe6fd9890 create_vbox.sh: cleanup help text formatting
Also added missing single letter options from the Usage
string.  Long options (such as --qemu-emulator) aren't
in the Usage string - leave them off for now, we may
want to consider just eliminating the Usage string
entirely rather than duplicating the options in
the Usage string and the option list below it.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I3d8b53f9395f25b07842d81a2d18c29a104b209e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13059
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2022-06-20 09:56:52 +00:00